From eric.des.courtis@REDACTED Wed Mar 1 00:07:58 2017 From: eric.des.courtis@REDACTED (Eric des Courtis) Date: Tue, 28 Feb 2017 18:07:58 -0500 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: Message-ID: Haha yes I know! But it's a reality that Amazon is down much less than most VPS services. On Tue, Feb 28, 2017 at 5:30 PM, Lo?c Hoguin wrote: > Great timing. :-) > > On 02/28/2017 11:29 PM, Eric des Courtis wrote: > >> In my experience cheap VPS services tend to be flaky. Amazon offers EC2 >> instance for free for one year. I doubt you can get a more reliable >> setup for the price. >> >> https://aws.amazon.com/s/dm/optimization/server-side-test/fr >> ee-tier/free_np/ >> >> Eric >> >> >> >> On Tue, Feb 28, 2017 at 3:20 PM, Caragea Silviu > > wrote: >> >> Most probably the cheapest solution: https://contabo.com/?show=vps 7 >> euro/month for 6 GB and 2 cores >> Silviu >> >> On Tue, Feb 28, 2017 at 7:04 PM, Leo > > wrote: >> >> >> >> 2017-02-28 17:14 GMT+01:00 Joe Armstrong > >: >> >> Hello, >> >> I was considering setting up a dedicated Erlang VPS for a >> small web >> site with a few simple Erlang applications. >> >> I was looking at the various hosting packages - and have to >> choose >> the amount of memory I need - any ideas how much resident >> memory I >> need to run a smallish server. >> >> I guess all I need is an OS + Erlang - what is a suitable >> choice of >> OS? - Do i really need GBytes of memory - it sounds a lot to >> me. >> >> Any advice on VPS providers? >> >> >> Maybe you can find some advice here: https://deploystack.io/ >> >> Leo >> >> >> >> >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > g.org> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfisher@REDACTED Wed Mar 1 00:08:53 2017 From: pfisher@REDACTED (Fisher, Paul) Date: Tue, 28 Feb 2017 23:08:53 +0000 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: , Message-ID: Best advise is to stay out of us-east-1... -- paul Fellow, Engineering ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Eric des Courtis Sent: Tuesday, February 28, 2017 5:07:58 PM To: Lo?c Hoguin; Erlang-Questions Questions Subject: Re: [erlang-questions] setting up a VPS for dedicated erlang server Haha yes I know! But it's a reality that Amazon is down much less than most VPS services. On Tue, Feb 28, 2017 at 5:30 PM, Lo?c Hoguin > wrote: Great timing. :-) On 02/28/2017 11:29 PM, Eric des Courtis wrote: In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. https://aws.amazon.com/s/dm/optimization/server-side-test/free-tier/free_np/ Eric On Tue, Feb 28, 2017 at 3:20 PM, Caragea Silviu >> wrote: Most probably the cheapest solution: https://contabo.com/?show=vps 7 euro/month for 6 GB and 2 cores Silviu On Tue, Feb 28, 2017 at 7:04 PM, Leo >> wrote: 2017-02-28 17:14 GMT+01:00 Joe Armstrong >>: Hello, I was considering setting up a dedicated Erlang VPS for a small web site with a few simple Erlang applications. I was looking at the various hosting packages - and have to choose the amount of memory I need - any ideas how much resident memory I need to run a smallish server. I guess all I need is an OS + Erlang - what is a suitable choice of OS? - Do i really need GBytes of memory - it sounds a lot to me. Any advice on VPS providers? Maybe you can find some advice here: https://deploystack.io/ Leo Cheers /Joe _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin https://ninenines.eu Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Wed Mar 1 00:52:07 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Tue, 28 Feb 2017 18:52:07 -0500 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: Message-ID: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> > In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. From brauningbar@REDACTED Wed Mar 1 02:47:23 2017 From: brauningbar@REDACTED (fran) Date: Tue, 28 Feb 2017 22:47:23 -0300 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: Message-ID: my stack Nitrogen web framework: best, simpler and capable web framework in the world using Cowboy webserver and in the vps, VPSDIME, a very cheap 7$: 6G, 4cpu, 30GB ssd, 2TB, 10gbps couldn't be more satisfied 2017-02-28 13:14 GMT-03:00, Joe Armstrong : > Hello, > > I was considering setting up a dedicated Erlang VPS for a small web > site with a few simple Erlang applications. > > I was looking at the various hosting packages - and have to choose > the amount of memory I need - any ideas how much resident memory I > need to run a smallish server. > > I guess all I need is an OS + Erlang - what is a suitable choice of > OS? - Do i really need GBytes of memory - it sounds a lot to me. > > Any advice on VPS providers? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roger@REDACTED Wed Mar 1 09:32:57 2017 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 1 Mar 2017 08:32:57 +0000 Subject: [erlang-questions] Send VM metrics to Grafana In-Reply-To: References: <9d68c485-d56c-5da3-c27f-a56d76ea8b98@khandkar.net> Message-ID: On 28 February 2017 at 20:43, Anthony Molinaro wrote: > I haven't done it, but isn't there a gen_event behind the error_logger you can somehow tap into to collect this? lager[1] hooks into error_logger to convert process crashes, etc. into error reports. If you want specific stats, you could take a look at how it does it. Or you could do what we do, which is simply to convert the warning/error/etc. lager events into metrics, by using a custom lager backend. You lose the per-process stats, but you still get a good overview. [1]: https://github.com/erlang-lager/lager From Lucas.Persson@REDACTED Wed Mar 1 10:39:59 2017 From: Lucas.Persson@REDACTED (Lucas Persson) Date: Wed, 1 Mar 2017 09:39:59 +0000 Subject: [erlang-questions] SSL issues with Firefox 51.0.1 and Chrome 56.0.2924.87 Message-ID: Hi I do have the same issue with FF and Chrome as and I did upgrade Erlang to 18 so that ssl_cipher:hash_algorithm(8) returns unassigned. But still the browser can not connect properly since it says it got an "net::ERR_INSECURE_RESPONSE" Checking the RFC for TLS1.2 then there isn't any hash_algorithm with value 8. Only 0-6 and 256 are valid values. What change in FF and Chrome made them send value 8? That was not the case in FF 50 and Chrome 55. The only "big change" in those browsers are that they do not trust any cert signed with SHA-1. The only log I get on the server side is: 2017-03-01T10:38:55.259 [error] [<0.434.0>|undefined] unknown_module: Ranch listener receiver_https had connection process started with cowboy_protocol:start_link/4 at <0.984.0> exit with reason: {closed,[{ranch_ssl,accept_ack,2,[{file,"src/ranch_ssl.erl"},{line,115}]},{cowboy_protocol,init,4,[{file,"src/cowboy_protocol.erl"},{line,91}]}]} 2017-03-01T10:38:55.259 [error] [emulator|undefined] unknown_module: Error in process <0.984.0> on node cepheid@REDACTED with exit value: {closed,[{ranch_ssl,accept_ack,2,[{file,"src/ranch_ssl.erl"},{line,115}]},{cowboy_protocol,init,4,[{file,"src/cowboy_protocol.erl"},{line,91}]}]} Thanks Lucas -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3587 bytes Desc: not available URL: From igor.clark@REDACTED Wed Mar 1 11:24:39 2017 From: igor.clark@REDACTED (Igor Clark) Date: Wed, 1 Mar 2017 10:24:39 +0000 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> Message-ID: <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> Bit late to this party but I really like https://www.scaleway.com/ - they're European (Paris/Amsterdam), and they provide cheap, decent-spec VPSs, as well as their "Bare metal" range which is own-design, custom-build, multi-tenant hardware. I use one with 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth at a decent fixed rate (300mbps on my package), so you don't get out-of-control bandwidth charge horror stories. And no, I don't work there, I just think they're really good :-) On 28/02/2017 23:52, Nathaniel Waisbrot wrote: >> In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. > > > The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From himars@REDACTED Wed Mar 1 11:30:23 2017 From: himars@REDACTED (Jack Tang) Date: Wed, 1 Mar 2017 18:30:23 +0800 Subject: [erlang-questions] Send VM metrics to Grafana In-Reply-To: References: <9d68c485-d56c-5da3-c27f-a56d76ea8b98@khandkar.net> Message-ID: Thank you for the hits, guys. We hook customized gen_event to error_logger like lager, and collect the crashed information. Also mnesia metrics are added. After some tests the code will be put to github :) On Wed, Mar 1, 2017 at 4:32 PM, Roger Lipscombe wrote: > On 28 February 2017 at 20:43, Anthony Molinaro > wrote: > > I haven't done it, but isn't there a gen_event behind the error_logger > you can somehow tap into to collect this? > > lager[1] hooks into error_logger to convert process crashes, etc. into > error reports. If you want specific stats, you could take a look at > how it does it. > > Or you could do what we do, which is simply to convert the > warning/error/etc. lager events into metrics, by using a custom lager > backend. You lose the per-process stats, but you still get a good > overview. > > [1]: https://github.com/erlang-lager/lager > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jack Tang http://www.linkedin.com/in/jacktang -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Wed Mar 1 11:35:39 2017 From: hugo@REDACTED (Hugo Mills) Date: Wed, 1 Mar 2017 10:35:39 +0000 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> Message-ID: <20170301103539.GH29429@carfax.org.uk> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: > Bit late to this party but I really like https://www.scaleway.com/ - > they're European (Paris/Amsterdam), and they provide cheap, > decent-spec VPSs, as well as their "Bare metal" range which is > own-design, custom-build, multi-tenant hardware. I use one with > 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth > at a decent fixed rate (300mbps on my package), so you don't get > out-of-control bandwidth charge horror stories. And no, I don't work > there, I just think they're really good :-) I'll second Scaleway. It's Just Worked for me. The other recommendation I've got is a small company called Bitfolk (http://bitfolk.com). More expensive than many of the larger operators, but the quality of the support is superb. (Disclaimer: I've known the owner of the company for many years; I own two Bitfolk VMs and manage a third). Hugo. > On 28/02/2017 23:52, Nathaniel Waisbrot wrote: > >>In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. > > > > > >The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. > > > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | We are all lying in the gutter, but some of us are hugo@REDACTED carfax.org.uk | looking at the stars. http://carfax.org.uk/ | PGP: E2AB1DE4 | Oscar Wilde -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From erlang@REDACTED Wed Mar 1 12:08:52 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 1 Mar 2017 12:08:52 +0100 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: <20170301103539.GH29429@carfax.org.uk> References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills wrote: > On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: >> Bit late to this party but I really like https://www.scaleway.com/ - >> they're European (Paris/Amsterdam), and they provide cheap, >> decent-spec VPSs, as well as their "Bare metal" range which is >> own-design, custom-build, multi-tenant hardware. I use one with >> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth >> at a decent fixed rate (300mbps on my package), so you don't get >> out-of-control bandwidth charge horror stories. And no, I don't work >> there, I just think they're really good :-) > > I'll second Scaleway. It's Just Worked for me. > > The other recommendation I've got is a small company called Bitfolk > (http://bitfolk.com). More expensive than many of the larger > operators, but the quality of the support is superb. (Disclaimer: I've > known the owner of the company for many years; I own two Bitfolk VMs > and manage a third). > > Hugo. > >> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: >> >>In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. >> > >> > >> >The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. >> > >> >_______________________________________________ >> >erlang-questions mailing list >> >erlang-questions@REDACTED >> >http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Hugo Mills | We are all lying in the gutter, but some of us are > hugo@REDACTED carfax.org.uk | looking at the stars. > http://carfax.org.uk/ | > PGP: E2AB1DE4 | Oscar Wilde > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Wed Mar 1 12:28:20 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 1 Mar 2017 12:28:20 +0100 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: Thanks for all your reply - I shall choose one of these and give it a try. It seems there is a lot of choice. Next problem. Given that I have decided on a VPS and payed the $$$ - what I now have is a raw machine and some kind of admin interface. The admin interface will (I guess) allow choice of an OS and set up a few basic things - at the end of which I assume I can do an SSH login and then I'm free to play. The next step is that I want to setup a whole load of things to make the machine minimally useful - install Erlang etc. All the installation commands and paths and environment variables and so will depend upon my choice of OS - it would be nice to just run a local script (on my machine at home) that automates as much as possible of this. But what I'd prefer to do is abstract away from the package manager and say Locally $ remote_install erlang If my remote machine was a linux machine this might cause an 'apt get command' to be issued remotely - if the remote machine was windows it would do a chocolatey command - it it were a mac it would do a brew command Is there anything remotely like this??? Has anybody any advice on the best way to proceed. Or do I have to write a long 'rsh' script :-( (And no I'm not looking for an expensive tool that does *everything* and has a 400 page manual - just something simple) Cheers /Joe On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong wrote: > On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills wrote: >> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: >>> Bit late to this party but I really like https://www.scaleway.com/ - >>> they're European (Paris/Amsterdam), and they provide cheap, >>> decent-spec VPSs, as well as their "Bare metal" range which is >>> own-design, custom-build, multi-tenant hardware. I use one with >>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth >>> at a decent fixed rate (300mbps on my package), so you don't get >>> out-of-control bandwidth charge horror stories. And no, I don't work >>> there, I just think they're really good :-) >> >> I'll second Scaleway. It's Just Worked for me. >> >> The other recommendation I've got is a small company called Bitfolk >> (http://bitfolk.com). More expensive than many of the larger >> operators, but the quality of the support is superb. (Disclaimer: I've >> known the owner of the company for many years; I own two Bitfolk VMs >> and manage a third). >> >> Hugo. >> >>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: >>> >>In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. >>> > >>> > >>> >The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. >>> > >>> >_______________________________________________ >>> >erlang-questions mailing list >>> >erlang-questions@REDACTED >>> >http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> Hugo Mills | We are all lying in the gutter, but some of us are >> hugo@REDACTED carfax.org.uk | looking at the stars. >> http://carfax.org.uk/ | >> PGP: E2AB1DE4 | Oscar Wilde >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From hugo@REDACTED Wed Mar 1 12:41:19 2017 From: hugo@REDACTED (Hugo Mills) Date: Wed, 1 Mar 2017 11:41:19 +0000 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: <20170301114119.GI29429@carfax.org.uk> On Wed, Mar 01, 2017 at 12:28:20PM +0100, Joe Armstrong wrote: > Thanks for all your reply - I shall choose one of these and give it a try. > It seems there is a lot of choice. > > Next problem. > > Given that I have decided on a VPS and payed the $$$ - what I now have > is a raw machine and some kind of admin interface. > > The admin interface will (I guess) allow choice of an OS and set up a few > basic things - at the end of which I assume I can do an SSH login and then > I'm free to play. > > The next step is that I want to setup a whole load of things to make the machine > minimally useful - install Erlang etc. > > All the installation commands and paths and environment variables and so > will depend upon my choice of OS - it would be nice to just run a local > script (on my machine at home) that automates as much as possible of this. > > But what I'd prefer to do is abstract away from the package manager and say > Locally > > $ remote_install erlang > > If my remote machine was a linux machine this might cause an 'apt get command' > to be issued remotely - if the remote machine was windows it would do > a chocolatey command - it it were a mac it would do a brew command > > Is there anything remotely like this??? > > Has anybody any advice on the best way to proceed. Or do I have to write > a long 'rsh' script :-( > > (And no I'm not looking for an expensive tool that does *everything* and has > a 400 page manual - just something simple) If you intend having a small number of choices for the whole system (i.e. only one Ubuntu configuration, only one Windows configuration, only one Fedora configuration), and simply want to be able to deploy that configuration repeatedly, then many VM providers will give you the ability to build deployment images. If you want more flexibility (i.e. you might want an Ubuntu configuration with Postgres installed one time, and one with RabbitMQ installed a second time), then you're looking more into the area of configuration management systems like Puppet or Chef or Ansible. I use Puppet -- it's a pain in the arse, but that's more to do with (a) the problem space: completely automating an installation and site-specific configuration for a bunch of arbitrary packages takes time and effort, and (b) the fact that I'm not really a practised sysadmin and don't much enjoy it. For simple things (installing packages, or managing basic configs of popular and well-known packages like Apache or Postgres), Puppet is pretty straightforward, and you can get _something_ working quickly. It's probably no more effort in the long term than writing and maintaining a bunch of complicated shell scripts, and ultimately more reliable. Hugo. > On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong wrote: > > On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills wrote: > >> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: > >>> Bit late to this party but I really like https://www.scaleway.com/ - > >>> they're European (Paris/Amsterdam), and they provide cheap, > >>> decent-spec VPSs, as well as their "Bare metal" range which is > >>> own-design, custom-build, multi-tenant hardware. I use one with > >>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth > >>> at a decent fixed rate (300mbps on my package), so you don't get > >>> out-of-control bandwidth charge horror stories. And no, I don't work > >>> there, I just think they're really good :-) > >> > >> I'll second Scaleway. It's Just Worked for me. > >> > >> The other recommendation I've got is a small company called Bitfolk > >> (http://bitfolk.com). More expensive than many of the larger > >> operators, but the quality of the support is superb. (Disclaimer: I've > >> known the owner of the company for many years; I own two Bitfolk VMs > >> and manage a third). > >> > >> Hugo. > >> > >>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: > >>> >>In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. > >>> > > >>> > > >>> >The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. > >>> > > >>> >_______________________________________________ > >>> >erlang-questions mailing list > >>> >erlang-questions@REDACTED > >>> >http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> -- Hugo Mills | Great oxymorons of the world, no. 7: hugo@REDACTED carfax.org.uk | The Simple Truth http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From roe.adrian@REDACTED Wed Mar 1 13:18:36 2017 From: roe.adrian@REDACTED (Adrian Roe) Date: Wed, 1 Mar 2017 12:18:36 +0000 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: <20170301114119.GI29429@carfax.org.uk> References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> <20170301114119.GI29429@carfax.org.uk> Message-ID: <08B4E6AD-79E3-4581-B965-04ECA74CE6AC@gmail.com> Joe I agree that tools such as Puppet etc are a good way to go if you are operating at scale but there is an element of 400 page manual about Puppet and other tools. It sounds to me as though your use-case would be well met with a simple shell script. We have one that builds a CentOS dev environment (so yum rather than apt-get) that would be very easy to tailor. More than happy to send you a copy, but there?s no rocket science to it! One bit of advice if you do go down the ?just a few shell scripts? route would be that you are well served to run all your sessions on the remote server in something like tmux or screen as that will allow your session to continue running should you have a temporary network break between you and the server. It?s so very frustrating if you get most of the way through an instal to have it randomly killed in some arbitrary state if your local network glitches? Adrian > On 1 Mar 2017, at 11:41, Hugo Mills wrote: > > On Wed, Mar 01, 2017 at 12:28:20PM +0100, Joe Armstrong wrote: >> Thanks for all your reply - I shall choose one of these and give it a try. >> It seems there is a lot of choice. >> >> Next problem. >> >> Given that I have decided on a VPS and payed the $$$ - what I now have >> is a raw machine and some kind of admin interface. >> >> The admin interface will (I guess) allow choice of an OS and set up a few >> basic things - at the end of which I assume I can do an SSH login and then >> I'm free to play. >> >> The next step is that I want to setup a whole load of things to make the machine >> minimally useful - install Erlang etc. >> >> All the installation commands and paths and environment variables and so >> will depend upon my choice of OS - it would be nice to just run a local >> script (on my machine at home) that automates as much as possible of this. >> >> But what I'd prefer to do is abstract away from the package manager and say >> Locally >> >> $ remote_install erlang >> >> If my remote machine was a linux machine this might cause an 'apt get command' >> to be issued remotely - if the remote machine was windows it would do >> a chocolatey command - it it were a mac it would do a brew command >> >> Is there anything remotely like this??? >> >> Has anybody any advice on the best way to proceed. Or do I have to write >> a long 'rsh' script :-( >> >> (And no I'm not looking for an expensive tool that does *everything* and has >> a 400 page manual - just something simple) > > If you intend having a small number of choices for the whole system > (i.e. only one Ubuntu configuration, only one Windows configuration, > only one Fedora configuration), and simply want to be able to deploy > that configuration repeatedly, then many VM providers will give you > the ability to build deployment images. > > If you want more flexibility (i.e. you might want an Ubuntu > configuration with Postgres installed one time, and one with RabbitMQ > installed a second time), then you're looking more into the area of > configuration management systems like Puppet or Chef or Ansible. I use > Puppet -- it's a pain in the arse, but that's more to do with (a) the > problem space: completely automating an installation and site-specific > configuration for a bunch of arbitrary packages takes time and effort, > and (b) the fact that I'm not really a practised sysadmin and don't > much enjoy it. For simple things (installing packages, or managing > basic configs of popular and well-known packages like Apache or > Postgres), Puppet is pretty straightforward, and you can get > _something_ working quickly. It's probably no more effort in the long > term than writing and maintaining a bunch of complicated shell > scripts, and ultimately more reliable. > > Hugo. > >> On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong > wrote: >>> On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills > wrote: >>>> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: >>>>> Bit late to this party but I really like https://www.scaleway.com/ - >>>>> they're European (Paris/Amsterdam), and they provide cheap, >>>>> decent-spec VPSs, as well as their "Bare metal" range which is >>>>> own-design, custom-build, multi-tenant hardware. I use one with >>>>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth >>>>> at a decent fixed rate (300mbps on my package), so you don't get >>>>> out-of-control bandwidth charge horror stories. And no, I don't work >>>>> there, I just think they're really good :-) >>>> >>>> I'll second Scaleway. It's Just Worked for me. >>>> >>>> The other recommendation I've got is a small company called Bitfolk >>>> (http://bitfolk.com ). More expensive than many of the larger >>>> operators, but the quality of the support is superb. (Disclaimer: I've >>>> known the owner of the company for many years; I own two Bitfolk VMs >>>> and manage a third). >>>> >>>> Hugo. >>>> >>>>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: >>>>>>> In my experience cheap VPS services tend to be flaky. Amazon offers EC2 instance for free for one year. I doubt you can get a more reliable setup for the price. >>>>>> >>>>>> >>>>>> The free tier is nice if you're interested in getting into Amazon (it's a frequently requested resume item). But I used the "free" tier, thought I was being careful, and got slapped with $60 in charges from network traffic before I could shut things down. There is no way to tell Amazon "I would rather go offline than pay $x", so a misconfigured cron job or traffic spike (DDoS?) that happens while you're asleep is basically guaranteed to cost you. >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> > > -- > Hugo Mills | Great oxymorons of the world, no. 7: > hugo@REDACTED carfax.org.uk | The Simple Truth > http://carfax.org.uk/ | > PGP: E2AB1DE4 | > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Wed Mar 1 13:21:49 2017 From: marco.molteni@REDACTED (Marco Molteni) Date: Wed, 1 Mar 2017 13:21:49 +0100 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: > On 1 Mar 2017, at 12:28, Joe Armstrong wrote: .. > But what I'd prefer to do is abstract away from the package manager and say > Locally > > $ remote_install erlang .. You are entering the realm of what is called orchestration or configuration management. I don't think there is a "simple script" do do what you want. On the other hand, last-generation configuration management tools like SaltStack and Ansible allow to do exactly that, and way more. You could even take a additional step back and provision the VM with APIs like OpenStack, but probably it would be over engineering for your use case. Since we are functional :-), an alternative approach (not exactly the same thing) is http://nixos.org/ and http://nixos.org/nix/, which should really excite any Erlanger :-) marco From donpedrothird@REDACTED Wed Mar 1 16:11:28 2017 From: donpedrothird@REDACTED (John Doe) Date: Wed, 1 Mar 2017 18:11:28 +0300 Subject: [erlang-questions] Send VM metrics to Grafana In-Reply-To: References: <9d68c485-d56c-5da3-c27f-a56d76ea8b98@khandkar.net> Message-ID: Just a warning - error_logger is an async gen_server process, so it could easily crash the VM if you can't control in and out rates of events. 2017-03-01 13:30 GMT+03:00 Jack Tang : > Thank you for the hits, guys. We hook customized gen_event to error_logger > like lager, and collect the crashed information. Also mnesia metrics are > added. After some tests the code will be put to github :) > > On Wed, Mar 1, 2017 at 4:32 PM, Roger Lipscombe > wrote: > >> On 28 February 2017 at 20:43, Anthony Molinaro >> wrote: >> > I haven't done it, but isn't there a gen_event behind the error_logger >> you can somehow tap into to collect this? >> >> lager[1] hooks into error_logger to convert process crashes, etc. into >> error reports. If you want specific stats, you could take a look at >> how it does it. >> >> Or you could do what we do, which is simply to convert the >> warning/error/etc. lager events into metrics, by using a custom lager >> backend. You lose the per-process stats, but you still get a good >> overview. >> >> [1]: https://github.com/erlang-lager/lager >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Jack Tang > > > http://www.linkedin.com/in/jacktang > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 3bep0n@REDACTED Wed Mar 1 16:35:46 2017 From: 3bep0n@REDACTED (Mykola) Date: Wed, 1 Mar 2017 17:35:46 +0200 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: Joe, I can recommend to take a look at Vagrant and play locally with provisioning. There is many different provisioners supported out-of-box, so you can choose a most comfortable/suitable one. I have started from shell scripts, then used Ansible, and actually using Puppet. Key feature for my case - Puppet executed on guest system, so I don't need Puppet installed at host system - only Puppet modules near to Vagrantfile. -- Mykola 2017-03-01 13:28 GMT+02:00 Joe Armstrong : > Thanks for all your reply - I shall choose one of these and give it a try. > It seems there is a lot of choice. > > Next problem. > > Given that I have decided on a VPS and payed the $$$ - what I now have > is a raw machine and some kind of admin interface. > > The admin interface will (I guess) allow choice of an OS and set up a few > basic things - at the end of which I assume I can do an SSH login and then > I'm free to play. > > The next step is that I want to setup a whole load of things to make the > machine > minimally useful - install Erlang etc. > > All the installation commands and paths and environment variables and so > will depend upon my choice of OS - it would be nice to just run a local > script (on my machine at home) that automates as much as possible of this. > > But what I'd prefer to do is abstract away from the package manager and say > Locally > > $ remote_install erlang > > If my remote machine was a linux machine this might cause an 'apt get > command' > to be issued remotely - if the remote machine was windows it would do > a chocolatey command - it it were a mac it would do a brew command > > Is there anything remotely like this??? > > Has anybody any advice on the best way to proceed. Or do I have to write > a long 'rsh' script :-( > > (And no I'm not looking for an expensive tool that does *everything* and > has > a 400 page manual - just something simple) > > Cheers > > /Joe > > > > On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong wrote: > > On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills wrote: > >> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: > >>> Bit late to this party but I really like https://www.scaleway.com/ - > >>> they're European (Paris/Amsterdam), and they provide cheap, > >>> decent-spec VPSs, as well as their "Bare metal" range which is > >>> own-design, custom-build, multi-tenant hardware. I use one with > >>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth > >>> at a decent fixed rate (300mbps on my package), so you don't get > >>> out-of-control bandwidth charge horror stories. And no, I don't work > >>> there, I just think they're really good :-) > >> > >> I'll second Scaleway. It's Just Worked for me. > >> > >> The other recommendation I've got is a small company called Bitfolk > >> (http://bitfolk.com). More expensive than many of the larger > >> operators, but the quality of the support is superb. (Disclaimer: I've > >> known the owner of the company for many years; I own two Bitfolk VMs > >> and manage a third). > >> > >> Hugo. > >> > >>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: > >>> >>In my experience cheap VPS services tend to be flaky. Amazon offers > EC2 instance for free for one year. I doubt you can get a more reliable > setup for the price. > >>> > > >>> > > >>> >The free tier is nice if you're interested in getting into Amazon > (it's a frequently requested resume item). But I used the "free" tier, > thought I was being careful, and got slapped with $60 in charges from > network traffic before I could shut things down. There is no way to tell > Amazon "I would rather go offline than pay $x", so a misconfigured cron job > or traffic spike (DDoS?) that happens while you're asleep is basically > guaranteed to cost you. > >>> > > >>> >_______________________________________________ > >>> >erlang-questions mailing list > >>> >erlang-questions@REDACTED > >>> >http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> -- > >> Hugo Mills | We are all lying in the gutter, but some of us > are > >> hugo@REDACTED carfax.org.uk | looking at the stars. > >> http://carfax.org.uk/ | > >> PGP: E2AB1DE4 | > Oscar Wilde > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spylik@REDACTED Wed Mar 1 18:08:49 2017 From: spylik@REDACTED (Oleksii Semilietov) Date: Wed, 1 Mar 2017 19:08:49 +0200 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: I believe Ansible the simplest way to do it. It much simpler than Chef or Puppet and works just on top of ssh. Also, in Haskel community quite popular Nix (https://nixos.org/nix/) with NixOps. I played with it few month ago and I think it worth to play with. I implemented simple Cowboy web application, then wrapped it as Nix package and module where I creating systemd service on port 80 and start it. Samples how to get it with Erlang could be found in my playground repo: https://github.com/spylik/zlr-nix But sure, Ansible is much simple. Cheers On 1 March 2017 at 13:28, Joe Armstrong wrote: > Thanks for all your reply - I shall choose one of these and give it a try. > It seems there is a lot of choice. > > Next problem. > > Given that I have decided on a VPS and payed the $$$ - what I now have > is a raw machine and some kind of admin interface. > > The admin interface will (I guess) allow choice of an OS and set up a few > basic things - at the end of which I assume I can do an SSH login and then > I'm free to play. > > The next step is that I want to setup a whole load of things to make the > machine > minimally useful - install Erlang etc. > > All the installation commands and paths and environment variables and so > will depend upon my choice of OS - it would be nice to just run a local > script (on my machine at home) that automates as much as possible of this. > > But what I'd prefer to do is abstract away from the package manager and say > Locally > > $ remote_install erlang > > If my remote machine was a linux machine this might cause an 'apt get > command' > to be issued remotely - if the remote machine was windows it would do > a chocolatey command - it it were a mac it would do a brew command > > Is there anything remotely like this??? > > Has anybody any advice on the best way to proceed. Or do I have to write > a long 'rsh' script :-( > > (And no I'm not looking for an expensive tool that does *everything* and > has > a 400 page manual - just something simple) > > Cheers > > /Joe > > > > On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong wrote: > > On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills wrote: > >> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: > >>> Bit late to this party but I really like https://www.scaleway.com/ - > >>> they're European (Paris/Amsterdam), and they provide cheap, > >>> decent-spec VPSs, as well as their "Bare metal" range which is > >>> own-design, custom-build, multi-tenant hardware. I use one with > >>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered bandwidth > >>> at a decent fixed rate (300mbps on my package), so you don't get > >>> out-of-control bandwidth charge horror stories. And no, I don't work > >>> there, I just think they're really good :-) > >> > >> I'll second Scaleway. It's Just Worked for me. > >> > >> The other recommendation I've got is a small company called Bitfolk > >> (http://bitfolk.com). More expensive than many of the larger > >> operators, but the quality of the support is superb. (Disclaimer: I've > >> known the owner of the company for many years; I own two Bitfolk VMs > >> and manage a third). > >> > >> Hugo. > >> > >>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: > >>> >>In my experience cheap VPS services tend to be flaky. Amazon offers > EC2 instance for free for one year. I doubt you can get a more reliable > setup for the price. > >>> > > >>> > > >>> >The free tier is nice if you're interested in getting into Amazon > (it's a frequently requested resume item). But I used the "free" tier, > thought I was being careful, and got slapped with $60 in charges from > network traffic before I could shut things down. There is no way to tell > Amazon "I would rather go offline than pay $x", so a misconfigured cron job > or traffic spike (DDoS?) that happens while you're asleep is basically > guaranteed to cost you. > >>> > > >>> >_______________________________________________ > >>> >erlang-questions mailing list > >>> >erlang-questions@REDACTED > >>> >http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> -- > >> Hugo Mills | We are all lying in the gutter, but some of us > are > >> hugo@REDACTED carfax.org.uk | looking at the stars. > >> http://carfax.org.uk/ | > >> PGP: E2AB1DE4 | > Oscar Wilde > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Oleksii D. Semilietov -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Wed Mar 1 22:06:47 2017 From: vans_163@REDACTED (Vans S) Date: Wed, 1 Mar 2017 21:06:47 +0000 (UTC) Subject: [erlang-questions] getting the error (after the fact) in an Erlang process? References: <802893271.4922308.1488402407727.ref@mail.yahoo.com> Message-ID: <802893271.4922308.1488402407727@mail.yahoo.com> Before I was doing something like: spawn_monitor(fun() -> {ok, F} = file:open("/hi", [write]), group_leader(F, self()) end). Now this processes error messages after a crash out go to the hi file. Now what I have is a [1] supervisor -> [1] gen_server, trap_exit, start_link -> [many] gen_statem When the gen_statem changes group_leader like that to a file, the errors do not go to the file. Is this because of how gen_*,trap_exit, and start_link work and whatnot? Does anyone know a way to keep this relationship but also have the error+stacktrace forwarded to the group leader? From dm.klionsky@REDACTED Thu Mar 2 09:19:21 2017 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Thu, 2 Mar 2017 11:19:21 +0300 Subject: [erlang-questions] setting up a VPS for dedicated erlang server In-Reply-To: References: <5FE0468C-256B-4D31-AC87-04D13AFA70FA@waisbrot.net> <9d320843-045f-8670-7ac0-de7316fb846d@gmail.com> <20170301103539.GH29429@carfax.org.uk> Message-ID: <58B7D589.9040601@gmail.com> I also recommend Ansible for its simplicity. On 03/01/2017 08:08 PM, Oleksii Semilietov wrote: > I believe Ansible the simplest way to do it. It much simpler than > Chef or Puppet and works just on top of ssh. > > Also, in Haskel community quite popular Nix (https://nixos.org/nix/) > with NixOps. > I played with it few month ago and I think it worth to play with. > I implemented simple Cowboy web application, then wrapped it as Nix > package and module where I creating systemd service on port 80 and > start it. > Samples how to get it with Erlang could be found in my playground > repo: https://github.com/spylik/zlr-nix > > But sure, Ansible is much simple. > > Cheers > > > On 1 March 2017 at 13:28, Joe Armstrong > wrote: > > Thanks for all your reply - I shall choose one of these and give > it a try. > It seems there is a lot of choice. > > Next problem. > > Given that I have decided on a VPS and payed the $$$ - what I now have > is a raw machine and some kind of admin interface. > > The admin interface will (I guess) allow choice of an OS and set > up a few > basic things - at the end of which I assume I can do an SSH login > and then > I'm free to play. > > The next step is that I want to setup a whole load of things to > make the machine > minimally useful - install Erlang etc. > > All the installation commands and paths and environment variables > and so > will depend upon my choice of OS - it would be nice to just run a > local > script (on my machine at home) that automates as much as possible > of this. > > But what I'd prefer to do is abstract away from the package > manager and say > Locally > > $ remote_install erlang > > If my remote machine was a linux machine this might cause an 'apt > get command' > to be issued remotely - if the remote machine was windows it would do > a chocolatey command - it it were a mac it would do a brew command > > Is there anything remotely like this??? > > Has anybody any advice on the best way to proceed. Or do I have to > write > a long 'rsh' script :-( > > (And no I'm not looking for an expensive tool that does > *everything* and has > a 400 page manual - just something simple) > > Cheers > > /Joe > > > > On Wed, Mar 1, 2017 at 12:08 PM, Joe Armstrong > wrote: > > On Wed, Mar 1, 2017 at 11:35 AM, Hugo Mills > wrote: > >> On Wed, Mar 01, 2017 at 10:24:39AM +0000, Igor Clark wrote: > >>> Bit late to this party but I really like > https://www.scaleway.com/ - > >>> they're European (Paris/Amsterdam), and they provide cheap, > >>> decent-spec VPSs, as well as their "Bare metal" range which is > >>> own-design, custom-build, multi-tenant hardware. I use one with > >>> 4-core/8GB/50GBSSD for ?11.99/month. And you get unmetered > bandwidth > >>> at a decent fixed rate (300mbps on my package), so you don't get > >>> out-of-control bandwidth charge horror stories. And no, I > don't work > >>> there, I just think they're really good :-) > >> > >> I'll second Scaleway. It's Just Worked for me. > >> > >> The other recommendation I've got is a small company called > Bitfolk > >> (http://bitfolk.com). More expensive than many of the larger > >> operators, but the quality of the support is superb. > (Disclaimer: I've > >> known the owner of the company for many years; I own two > Bitfolk VMs > >> and manage a third). > >> > >> Hugo. > >> > >>> On 28/02/2017 23:52, Nathaniel Waisbrot wrote: > >>> >>In my experience cheap VPS services tend to be flaky. Amazon > offers EC2 instance for free for one year. I doubt you can get a > more reliable setup for the price. > >>> > > >>> > > >>> >The free tier is nice if you're interested in getting into > Amazon (it's a frequently requested resume item). But I used the > "free" tier, thought I was being careful, and got slapped with $60 > in charges from network traffic before I could shut things down. > There is no way to tell Amazon "I would rather go offline than pay > $x", so a misconfigured cron job or traffic spike (DDoS?) that > happens while you're asleep is basically guaranteed to cost you. > >>> > > >>> >_______________________________________________ > >>> >erlang-questions mailing list > >>> >erlang-questions@REDACTED > >>> >http://erlang.org/mailman/listinfo/erlang-questions > > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > >> > >> -- > >> Hugo Mills | We are all lying in the gutter, but > some of us are > >> hugo@REDACTED carfax.org.uk | looking at the > stars. > >> http://carfax.org.uk/ | > >> PGP: E2AB1DE4 | Oscar Wilde > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Oleksii D. Semilietov > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- BR, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From christopher.meiklejohn@REDACTED Thu Mar 2 09:47:49 2017 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Thu, 2 Mar 2017 09:47:49 +0100 Subject: [erlang-questions] [ANN] Call for Papers, PMLDC 2017 Message-ID: Second Workshop on Programming Models and Languages for Distributed Computing Co-located with ECOOP 2017, Barcelona, Spain Date: July 20, 2017 Whether you are programming a rich web application in JavaScript that mutates state in the client?s browser, or you are building a massively deployed mobile application that will operate with client state at the device, it?s undeniable that you are building a distributed system! Two major challenges of programming distributed systems are concurrency and partial failure. Concurrency of operations can introduce accidental nondeterminism: computations may result in different outcomes with the same inputs given scheduling differences in the underlying system unless a synchronization mechanism is used to enforce some order. Synchronization is typically expensive, and reduces the efficiency of user applications. Partial failure, or the failure of one or more components in a distributed system at one time, introduces the challenge of knowing, when an operation fails, which components of the operation completed successfully. To solve these problems in practice on an unreliable, asynchronous network, atomic commit protocols and timeouts as failure detection are typically used. Because of these challenges, early approaches to providing programming abstractions for distributed computing that ignored them were inherently misguided: the canonical example being the Remote Procedure Call, still widely deployed in industry. The goal of this workshop is to discuss new approaches to distributed programming that provide efficient execution and the elimination of accidental nondeterminism resulting from concurrency and partial failure. It will bring together both practitioners and theoreticians from many disciplines: database theory, distributed systems, systems programming, programming languages, data-centric programming, web application development, and verification, to discuss the state-of-the-art of distributed programming, advancement of the state-of-the-art and paths forward to better application of theory in practice. The main objectives of this workshop are the following: * To review the state-of-the-art research in languages, models, and systems for distributed programming; * To identify areas of critical need where research can advance the state of the art; * To create a forum for discussion; * To present open problems from practitioners with an aim towards motivating academic research on relevant problems faced by industry. In the spirit of both ECOOP and Curry On, this workshop aims at favoring a multidisciplinary perspective by bringing together researchers, developers, and practitioners from both academia and industry. Submission Guidelines We solicit proposals for contributed talks. We recommend preparing proposals of 2 pages, in ACM 2 column SIGPLAN style, written in English and in PDF format. However, we will accept longer proposals or submissions to other conferences, under the understanding that PC members are only expected to read the first two pages of such longer submissions. Authors with accepted papers will have the opportunity to have their submission published on the ACM Digital Library. Workshop http://2017.ecoop.org/track/pmldc-2017-papers Submission Link https://easychair.org/conferences/?conf=pmldc2017 Important Dates Paper submission: Monday, 1 May 2017 Workshop date: Thursday, 20 July 2017 Authors notification: TBA Final version: TBA Organizing Committee Christopher Meiklejohn Universit? catholique de Louvain Heather Miller Ecole Polytechnique F?d?rale de Lausanne Program Committee Carlos Baquero Universidade do Minho Annette Bieniusa Technischen Universit?t Kaiserslautern Carla Ferreira Universidade Nova Lisboa Alexey Gotsman IMDEA Software Institute Tyler McMullen Fastly Rodrigo Rodrigues Instituto Superior T?cnico, University of Lisboa & INESC-ID Ali Shoker HASLab/INESC TEC & University of Minho KC Sivaramakrishnan University of Cambridge Zach Tellman Lacuna Peter Van Roy Universit? catholique de Louvain Leif Walsh Two Sigma Investments Jordan West NOVA-LINCS Hongseok Yang University of Oxford From raimo+erlang-questions@REDACTED Thu Mar 2 09:53:45 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 2 Mar 2017 09:53:45 +0100 Subject: [erlang-questions] getting the error (after the fact) in an Erlang process? In-Reply-To: <802893271.4922308.1488402407727@mail.yahoo.com> References: <802893271.4922308.1488402407727.ref@mail.yahoo.com> <802893271.4922308.1488402407727@mail.yahoo.com> Message-ID: <20170302085345.GA25143@erix.ericsson.se> On Wed, Mar 01, 2017 at 09:06:47PM +0000, Vans S wrote: > Before I was doing something like: > > spawn_monitor(fun() -> {ok, F} = file:open("/hi", [write]), group_leader(F, self()) end). > > Now this processes error messages after a crash out go to the hi file. > That redirects all process standard output to the file e.g io:format printouts. I did not know that the process exit message also lands there, a bit surprised actually. I would have guessed such messages were printed with error_logger, see below. > > Now what I have is a [1] supervisor -> [1] gen_server, trap_exit, start_link -> [many] gen_statem > > When the gen_statem changes group_leader like that to a file, the errors do not go to the file. > > Is this because of how gen_*,trap_exit, and start_link work and whatnot? > > Does anyone know a way to keep this relationship but also have the error+stacktrace forwarded to the > group leader? gen_* like like all processes adhering to OTP design principals print their events through error_logger: http://erlang.org/doc/man/error_logger.html This is to collect all system logs in one place. In production you often run the node via run_erl, which puts the system logs in rotating log files. Is this what you are aiming for? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tony.reix@REDACTED Thu Mar 2 12:23:34 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 11:23:34 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results Message-ID: Hi, In order to understand which Erlang tests and features do not work fine within our port of version 19.2 on AIX, we need some "official" Erlang tests results for current version (19.2.3). However, searching on Erlang web-site, I was unable to find such tests results. So, I have built and tested Erlang on 3 environments: AIX 6.1, Fedora25/PPC64, and Fedora25/x86_64. I hope I've correctly ran the tests, but I'm not sure. In short, using nearly the same procedure (use Fedora .spec file), but without HiPE-SMP-Threads for AIX, I've got: OK FAIL SKIP TOTAL ------------------------------------------ AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) FC25/PPC64 12,376 193 3,608 16,177 FC25/x86_64 12,407 288 3,475 16,170 I am really surprised by: - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all tests !! - how few failures I have on AIX, comparable to the amount of failures on FC25/x86_64. (However: - when trying with SMP and/or HiPE, I had hangs, so I'm first building/testing without them - we applied several AIX-specific patches) - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . (However: there are some more SKIPped tests: 133) Since result files contains about 600,000 lines, it is very difficult for me to understand where/what are the issues. I think we have understood that, in many cases, some environment is missing/failing and thus the test is skipped. I'd like to get some help about: - checking that I'm doing the Erlang configuration, build, and tests correctly, - finding some "official" Erlang tests results on Fedora / x86_64 - how to analyze these results and find the real issues. I have attached to this email the index.html file produced on Fedora25/x86_64 : /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html Thanks Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Thu Mar 2 13:37:31 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 2 Mar 2017 13:37:31 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: Did you run Erlang from the git repository/source tree or did you do "make release" and run it from there? A few tests cases will fail if you have not done "make release". (One example is xref_SUITE:md/1 in tools.) I think that the best way for us to help you is if you can share the names of the failed test cases in the following test suites: emulator_test stdlib_test kernel_test We should be able to tell you which ones that are most likely real bugs, and which ones that fail because of envioronmental issues. /Bj?rn On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang tests > results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all > tests !! > - how few failures I have on AIX, comparable to the amount of failures on > FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so I'm > first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult for me > to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From essen@REDACTED Thu Mar 2 14:09:37 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 2 Mar 2017 14:09:37 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: <9a6fe3b3-7945-9047-1429-bff3c09457f9@ninenines.eu> On 03/02/2017 01:37 PM, Bj?rn Gustavsson wrote: > Did you run Erlang from the git repository/source > tree or did you do "make release" and run it from > there? > > A few tests cases will fail if you have not done > "make release". (One example is > xref_SUITE:md/1 in tools.) When should we do this exactly? It is not mentioned here: https://github.com/erlang/otp/wiki/Running-tests Cheers, > I think that the best way for us to help you is > if you can share the names of the failed > test cases in the following test suites: > > emulator_test > stdlib_test > kernel_test > > We should be able to tell you which ones > that are most likely real bugs, and which > ones that fail because of envioronmental > issues. > > /Bj?rn > > On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony wrote: >> Hi, >> >> In order to understand which Erlang tests and features do not work fine >> within our port of version 19.2 on AIX, we need some "official" Erlang tests >> results for current version (19.2.3). >> However, searching on Erlang web-site, I was unable to find such tests >> results. >> >> So, I have built and tested Erlang on 3 environments: AIX 6.1, >> Fedora25/PPC64, and Fedora25/x86_64. >> I hope I've correctly ran the tests, but I'm not sure. >> >> In short, using nearly the same procedure (use Fedora .spec file), but >> without HiPE-SMP-Threads for AIX, I've got: >> >> OK FAIL SKIP TOTAL >> ------------------------------------------ >> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) >> FC25/PPC64 12,376 193 3,608 16,177 >> FC25/x86_64 12,407 288 3,475 16,170 >> >> I am really surprised by: >> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all >> tests !! >> - how few failures I have on AIX, comparable to the amount of failures on >> FC25/x86_64. >> (However: - when trying with SMP and/or HiPE, I had hangs, so I'm >> first building/testing without them >> - we applied several AIX-specific patches) >> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . >> (However: there are some more SKIPped tests: 133) >> >> Since result files contains about 600,000 lines, it is very difficult for me >> to understand where/what are the issues. >> I think we have understood that, in many cases, some environment is >> missing/failing and thus the test is skipped. >> >> I'd like to get some help about: >> - checking that I'm doing the Erlang configuration, build, and tests >> correctly, >> - finding some "official" Erlang tests results on Fedora / x86_64 >> - how to analyze these results and find the real issues. >> >> I have attached to this email the index.html file produced on >> Fedora25/x86_64 : >> >> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html >> >> Thanks >> >> Cordialement, >> >> Tony Reix >> >> Bull - ATOS >> IBM Coop Architect & Technical Leader >> Office : +33 (0) 4 76 29 72 67 >> 1 rue de Provence - 38432 ?chirolles - France >> www.atos.net >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- Lo?c Hoguin https://ninenines.eu From bjorn@REDACTED Thu Mar 2 14:40:08 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 2 Mar 2017 14:40:08 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: <9a6fe3b3-7945-9047-1429-bff3c09457f9@ninenines.eu> References: <9a6fe3b3-7945-9047-1429-bff3c09457f9@ninenines.eu> Message-ID: On Thu, Mar 2, 2017 at 2:09 PM, Lo?c Hoguin wrote: > On 03/02/2017 01:37 PM, Bj?rn Gustavsson wrote: >> >> A few tests cases will fail if you have not done >> "make release". (One example is >> xref_SUITE:md/1 in tools.) > > > When should we do this exactly? > After building Erlang/OTP, do "make release". Then you will have to run the Install script. On my Linux system the commands would be: make release cd release/x86_64-unknown-linux-gnu ./Install `pwd` When running the tests, started the released Erlang/OTP system like this: $ERL_TOP/release/x86_64-unknown-linux-gnu/bin/erl (Then do ts:install(), ts:run() etc as in the instruction.) > It is not mentioned here: https://github.com/erlang/otp/wiki/Running-tests It is not mentioned because there are only a few test cases that must be run in an installed system. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From hans.r.nilsson@REDACTED Thu Mar 2 15:39:05 2017 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Thu, 2 Mar 2017 15:39:05 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: Hi, I looked at the [crypto, public_key, ssh, ssl] group of applications. It does look very bad indeed, except for public_key. So let's start from the bottom (i.e. crypto). I need the following: 1) start your tested Erlang and call 1> crypto:info_lib(). -- result 1 --- 2> crypto:supports(). -- result 2 --- 3> What are the "result *" you get? 2) Mail me the crypto test results directory ..../ct_run.test_server@REDACTED/tests.crypto_test.logs /Hans On 03/02/2017 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang > tests results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of > all tests !! > - how few failures I have on AIX, comparable to the amount of failures > on FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so > I'm first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult > for me to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From tony.reix@REDACTED Thu Mar 2 15:41:40 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 14:41:40 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: , Message-ID: Hi Bj?rn, I got Erlang source from Fedora: # yumdownloader --source erlang I got: erlang-19.2.3-1.fc25.src.rpm It contains a .spec file, otp-OTP-19.2.3.tar.gz, and all other files (patches) needed for building Erlang. And I built Erlang by means of: rpmbuild -ba erlang-19.2.3-1.fc25.spec However, this .spec file does not build and launch the Erlang tests. I had to search Erlang documentation and find some commands. Main ones are: make release_tests make tests $ERL_TOP/bin/erl -pa $TESTROOT/test_server -s ts install -s ts run all_tests -s init stop This produced the results I provided in previous email. Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : bgustavsson@REDACTED [bgustavsson@REDACTED] de la part de Bj?rn Gustavsson [bjorn@REDACTED] Envoy? : jeudi 2 mars 2017 13:37 ? : REIX, Tony Cc : erlang-questions@REDACTED; GIRARDET, JEAN Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results Did you run Erlang from the git repository/source tree or did you do "make release" and run it from there? A few tests cases will fail if you have not done "make release". (One example is xref_SUITE:md/1 in tools.) I think that the best way for us to help you is if you can share the names of the failed test cases in the following test suites: emulator_test stdlib_test kernel_test We should be able to tell you which ones that are most likely real bugs, and which ones that fail because of envioronmental issues. /Bj?rn On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang tests > results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all > tests !! > - how few failures I have on AIX, comparable to the amount of failures on > FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so I'm > first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult for me > to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From semmitmondo@REDACTED Thu Mar 2 15:44:14 2017 From: semmitmondo@REDACTED (semmit mondo) Date: Thu, 2 Mar 2017 15:44:14 +0100 (CET) Subject: [erlang-questions] queue process with capacity Message-ID: Hi, I need to implement a queue service that has a capacity. This means that the `in` actions has to be able to block the producer when the queue is full, and also the `out` action has to be able to block the consumer when the queue is empty. Sending in an item and then receiving with "after infinity" in the producer seems to be a working approach. But my concern is that this way the producer process is blocked and so it is unable to respond to any system messages. If I do this in a supervised process, do I break the contract between OTP and that process? Aren't all the processes supposed to answer certain low level messages that is handled by gen_server without me to even know about? Shouldn't supervised processes be blocked only in the gen_server's main receive loop? I feel that receiving indefinitely is something I should avoid. But how can I implement the queue in an OTP friendly way? S. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Mar 2 15:43:48 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 2 Mar 2017 14:43:48 +0000 (UTC) Subject: [erlang-questions] getting the error (after the fact) in an Erlang process? In-Reply-To: <20170302085345.GA25143@erix.ericsson.se> References: <802893271.4922308.1488402407727.ref@mail.yahoo.com> <802893271.4922308.1488402407727@mail.yahoo.com> <20170302085345.GA25143@erix.ericsson.se> Message-ID: <1858684969.543923.1488465828474@mail.yahoo.com> > I did not know that the process exit message also lands there, > a bit surprised actually. Apparently so, and I was surprised going into the other direction, that an OTP app sends the message to error_logger instead of standard output. So far I found a temporary hacky solution to call erlang:get_stacktrace() when I get an EXIT signal or end up in terminate, then I print that to stdout of the process. I don't get the nice gen_* OTP error message but at least I get the error + stacktrace. > In production you often run the node via run_erl, which puts the system > logs in rotating log files. Is this what you are aiming for? I am aiming to have the gen* OTP error messages go to the stdout of the process that produced them, I wonder if this is at all possible? On Thursday, March 2, 2017 3:54 AM, Raimo Niskanen wrote: On Wed, Mar 01, 2017 at 09:06:47PM +0000, Vans S wrote: > Before I was doing something like: > > spawn_monitor(fun() -> {ok, F} = file:open("/hi", [write]), group_leader(F, self()) end). > > Now this processes error messages after a crash out go to the hi file. > That redirects all process standard output to the file e.g io:format printouts. I did not know that the process exit message also lands there, a bit surprised actually. I would have guessed such messages were printed with error_logger, see below. > > Now what I have is a [1] supervisor -> [1] gen_server, trap_exit, start_link -> [many] gen_statem > > When the gen_statem changes group_leader like that to a file, the errors do not go to the file. > > Is this because of how gen_*,trap_exit, and start_link work and whatnot? > > Does anyone know a way to keep this relationship but also have the error+stacktrace forwarded to the > group leader? gen_* like like all processes adhering to OTP design principals print their events through error_logger: http://erlang.org/doc/man/error_logger.html This is to collect all system logs in one place. In production you often run the node via run_erl, which puts the system logs in rotating log files. Is this what you are aiming for? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From tony.reix@REDACTED Thu Mar 2 16:26:44 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 15:26:44 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: , , Message-ID: Resending this email WITHOUT the attachment since it is too big to be sent without some approval. Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________ De : REIX, Tony Envoy? : jeudi 2 mars 2017 16:19 ? : Bj?rn Gustavsson Cc : erlang-questions@REDACTED; GIRARDET, JEAN Objet : RE:[erlang-questions] Desperately looking for official Erlang tests results Hi Bj?rn Compressed test results file from FC25/x86_64 is attached to this email: /tmp/erlang-19.2.3-1.fc25.tests.sh.res2.xz . About kernel_test, there are failed tests common to both FC25/x86_64 and AIX, and failed tests specific to each of the 2 OS. 1) On FC25/x86_64 : # grep "Testing tests.kernel_test" *tests*res2 | grep FAILED 1> Testing tests.kernel_test: *** FAILED test case 125 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 157 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 158 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 160 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 161 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 241 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 244 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 259 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 261 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 262 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 294 of 750 *** x86_64 only 1> Testing tests.kernel_test: *** FAILED test case 307 of 750 *** x86_64 only 1> Testing tests.kernel_test: *** FAILED {gen_tcp_api_SUITE,init_per_suite} *** 1> Testing tests.kernel_test: *** FAILED test case 384 of 750 *** x86_64 only 1> Testing tests.kernel_test: *** FAILED test case 417 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 561 of 750 *** x86_64 only 1> Testing tests.kernel_test: *** FAILED test case 562 of 750 *** x86_64 only 1> Testing tests.kernel_test: *** FAILED test case 606 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 607 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 612 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 613 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 627 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 629 of 750 *** 1> Testing tests.kernel_test: *** FAILED test case 630 of 750 *** 1> Testing tests.kernel_test: *** FAILED {seq_trace_SUITE,init_per_suite} *** Case 125: disk_log_SUITE:open_overwrite failed on line 1865 Reason: {badmatch,{ok,n}} 157,158: erl_boot_server_SUITE:start failed on line 83 Reason: {badmatch,{error,{badarg,["dorado-vm1","gandalf"]}}} 160: erl_boot_server_SUITE:add failed on line 153 Reason: {badmatch,{error,nxdomain}} 161: erl_boot_server_SUITE:delete failed on line 186 Reason: {badmatch,{error,nxdomain}} 241: file_SUITE:test_directory failed on line 1376 Reason: {badmatch,read} 244: file_SUITE:file_write_file_info failed on line 1524 Reason: {badmatch,ok} 259: file_SUITE:e_delete failed on line 2109 Reason: {badmatch,ok} 261: file_SUITE:e_make_dir failed on line 2240 Reason: {badmatch,ok} 262: file_SUITE:e_del_dir failed on line 2287 Reason: {badmatch,ok} 294: file_name_SUITE:make_icky_dir failed on line 642 Reason: {badmatch,{error,no_translation}} 307: gen_sctp_SUITE:ok failed on line 544 Reason: function_clause 384: gen_tcp_misc_SUITE:test_prio_fail failed on line 1871 Reason: {badmatch,ok} 417: gen_udp_SUITE:implicit_inet6 failed on line 733 Reason: not_ok 561: multi_load_SUITE:do_ensure_modules_loaded failed on line 282 Reason: {badmatch,false} 562: multi_load_SUITE:make_and_load failed on line 330 Reason: {badmatch,ok} 606: prim_file_SUITE:test_directory failed on line 801 Reason: {badmatch,read} 607: prim_file_SUITE:test_directory failed on line 801 Reason: {badmatch,read} 612: prim_file_SUITE:file_write_file_info failed on line 968 Reason: {badmatch,ok} 613: prim_file_SUITE:file_write_file_info failed on line 968 Reason: {badmatch,ok} 627: prim_file_SUITE:e_delete failed on line 1455 Reason: {badmatch,ok} 629: prim_file_SUITE:e_make_dir failed on line 1591 Reason: {badmatch,ok} 630: prim_file_SUITE:e_del_dir failed on line 1641 Reason: {badmatch,ok} 2) On AIX, I have these AIX-specific tests that fail: 1> Testing tests.kernel_test: *** FAILED test case 88 of 750 **** code_SUITE:big_boot_embedded failed on line 1119 Reason: {badmatch,{error,timeout}} 1> Testing tests.kernel_test: *** FAILED test case 220 of 750 *** file_SUITE:expect failed on line 217 Reason: function_clause 1> Testing tests.kernel_test: *** FAILED test case 284 of 750 *** erlang:hd failed Reason: badarg 1> Testing tests.kernel_test: *** FAILED test case 285 of 750 *** rlang:hd failed Reason: badarg 1> Testing tests.kernel_test: *** FAILED test case 499 of 750 *** inet_SUITE:do_getif failed on line 952 Reason: {test_case_failed,no_HWAs} 1> Testing tests.kernel_test: *** FAILED test case 502 of 750 *** inet_SUITE:getifaddrs failed on line 999 Reason: {test_case_failed,{should_have_hwaddr,{unix,aix}}} 1> Testing tests.kernel_test: *** FAILED test case 590 of 750 *** prim_file_SUITE:expect failed on line 157 Reason: function_clause 1> Testing tests.kernel_test: *** FAILED test case 591 of 750 *** prim_file_SUITE:expect failed on line 157 Reason: function_clause Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : bgustavsson@REDACTED [bgustavsson@REDACTED] de la part de Bj?rn Gustavsson [bjorn@REDACTED] Envoy? : jeudi 2 mars 2017 13:37 ? : REIX, Tony Cc : erlang-questions@REDACTED; GIRARDET, JEAN Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results Did you run Erlang from the git repository/source tree or did you do "make release" and run it from there? A few tests cases will fail if you have not done "make release". (One example is xref_SUITE:md/1 in tools.) I think that the best way for us to help you is if you can share the names of the failed test cases in the following test suites: emulator_test stdlib_test kernel_test We should be able to tell you which ones that are most likely real bugs, and which ones that fail because of envioronmental issues. /Bj?rn On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang tests > results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all > tests !! > - how few failures I have on AIX, comparable to the amount of failures on > FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so I'm > first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult for me > to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony.reix@REDACTED Thu Mar 2 16:49:51 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 15:49:51 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: , , Message-ID: Resending without the attachments since it is too big. I'll sent the files to Hans directly. Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________ De : REIX, Tony Envoy? : jeudi 2 mars 2017 16:46 ? : Hans Nilsson R; erlang-questions@REDACTED Objet : RE:[erlang-questions] Desperately looking for official Erlang tests results Hi Hans, On AIX, I have only 1 test failing: OK FAIL SKIP TOTAL crypto_test: 111 1 3 115 However, on FC25/x86_64, I have 4 more: OK FAIL SKIP TOTAL crypto_test: 107 5 3 115 1) AIX: 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** crypto:strong_rand_mpint failed on line 1417 Reason: low_entropy # cd otp_src_19.2 # ./bin/erl /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec Erlang/OTP 19 [erts-8.2] [source] Eshell V8.2 (abort with ^G) 1> crypto:info_lib(). [{<<"OpenSSL">>,268443823, <<"OpenSSL 1.0.2j 26 Sep 2016">>}] 2> crypto:supports(). [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, aes_gcm]}, {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}] 3> 2) FC25/x86_64: 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} *** 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 *** crypto:ecdh_compute_key_nif failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 *** crypto:ec_key_generate failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** Common with AIX crypto:strong_rand_mpint failed on line 1417 Reason: low_entropy 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 *** crypto:ec_key_generate failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 *** Error detected: {'EXIT',low_entropy} # pwd /root/rpmbuild/BUILD/otp-OTP-19.2.3 #./bin/erl Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.2.2 (abort with ^G) 1> crypto:info_lib(). [{<<"OpenSSL">>,268443839, <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}] 2> crypto:supports(). [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, aes_gcm]}, {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}] Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : Hans Nilsson R [hans.r.nilsson@REDACTED] Envoy? : jeudi 2 mars 2017 15:39 ? : erlang-questions@REDACTED; REIX, Tony Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results Hi, I looked at the [crypto, public_key, ssh, ssl] group of applications. It does look very bad indeed, except for public_key. So let's start from the bottom (i.e. crypto). I need the following: 1) start your tested Erlang and call 1> crypto:info_lib(). -- result 1 --- 2> crypto:supports(). -- result 2 --- 3> What are the "result *" you get? 2) Mail me the crypto test results directory ..../ct_run.test_server@REDACTED/tests.crypto_test.logs /Hans On 03/02/2017 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang > tests results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of > all tests !! > - how few failures I have on AIX, comparable to the amount of failures > on FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so > I'm first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult > for me to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Thu Mar 2 17:05:32 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 2 Mar 2017 17:05:32 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: On Thu, Mar 2, 2017 at 4:26 PM, REIX, Tony wrote: > Resending this email WITHOUT the attachment since it is too big to be sent > without some approval. > It seems to have been approved now. I have received the other email. I have taken a quick look at the log. The major problem is that many test cases that call init_per_SUITE fail when they attempt to compile C code during the running of the test suite, especially when running emulator_tests. You should be able to find more information from the generated html files. Starting with index.html, you could click on test.emulator_test. On that page, you could click on the links for the failed test cases. On the page for a failed init_per_SUITE test case, you should be able to see why the compilation failed. (Those failed test cases cause all test in the same module to be skipped.) On other thing so far: erl_boot_server_SUITE is not a good test suite. It has some hard-coded assumptions about our testing environment. It will fail outside our lab. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From raghav.karol@REDACTED Thu Mar 2 10:39:31 2017 From: raghav.karol@REDACTED (Raghav Karol) Date: Thu, 2 Mar 2017 10:39:31 +0100 Subject: [erlang-questions] Understanding dialyzer errors better Message-ID: I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: ``` src/bitcask.erl 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 520: The created fun has no local return 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'> 1870: Function wrap_write_file/1 has no local return 1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() src/bitcask_fileops.erl 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() 240: Function close_hintfile/1 has no local return 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() 314: The pattern 'undefined' can never match the type port() src/bitcask_merge_delete.erl 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() ===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings ===> Warnings occured running dialyzer: 17 ``` Particularly confusing, are errors like the first one ``` 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') ``` I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack. Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer. Best, Raghav -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony.reix@REDACTED Thu Mar 2 16:46:58 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 15:46:58 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: , Message-ID: Hi Hans, On AIX, I have only 1 test failing: OK FAIL SKIP TOTAL crypto_test: 111 1 3 115 However, on FC25/x86_64, I have 4 more: OK FAIL SKIP TOTAL crypto_test: 107 5 3 115 1) AIX: 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** crypto:strong_rand_mpint failed on line 1417 Reason: low_entropy # cd otp_src_19.2 # ./bin/erl /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec Erlang/OTP 19 [erts-8.2] [source] Eshell V8.2 (abort with ^G) 1> crypto:info_lib(). [{<<"OpenSSL">>,268443823, <<"OpenSSL 1.0.2j 26 Sep 2016">>}] 2> crypto:supports(). [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, aes_gcm]}, {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}] 3> 2) FC25/x86_64: 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} *** 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 *** crypto:ecdh_compute_key_nif failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 *** crypto:ec_key_generate failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** Common with AIX crypto:strong_rand_mpint failed on line 1417 Reason: low_entropy 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 *** crypto:ec_key_generate failed Reason: badarg 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 *** Error detected: {'EXIT',low_entropy} # pwd /root/rpmbuild/BUILD/otp-OTP-19.2.3 #./bin/erl Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.2.2 (abort with ^G) 1> crypto:info_lib(). [{<<"OpenSSL">>,268443839, <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}] 2> crypto:supports(). [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, aes_gcm]}, {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}] Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : Hans Nilsson R [hans.r.nilsson@REDACTED] Envoy? : jeudi 2 mars 2017 15:39 ? : erlang-questions@REDACTED; REIX, Tony Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results Hi, I looked at the [crypto, public_key, ssh, ssl] group of applications. It does look very bad indeed, except for public_key. So let's start from the bottom (i.e. crypto). I need the following: 1) start your tested Erlang and call 1> crypto:info_lib(). -- result 1 --- 2> crypto:supports(). -- result 2 --- 3> What are the "result *" you get? 2) Mail me the crypto test results directory ..../ct_run.test_server@REDACTED/tests.crypto_test.logs /Hans On 03/02/2017 12:23 PM, REIX, Tony wrote: > Hi, > > In order to understand which Erlang tests and features do not work fine > within our port of version 19.2 on AIX, we need some "official" Erlang > tests results for current version (19.2.3). > However, searching on Erlang web-site, I was unable to find such tests > results. > > So, I have built and tested Erlang on 3 environments: AIX 6.1, > Fedora25/PPC64, and Fedora25/x86_64. > I hope I've correctly ran the tests, but I'm not sure. > > In short, using nearly the same procedure (use Fedora .spec file), but > without HiPE-SMP-Threads for AIX, I've got: > > OK FAIL SKIP TOTAL > ------------------------------------------ > AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) > FC25/PPC64 12,376 193 3,608 16,177 > FC25/x86_64 12,407 288 3,475 16,170 > > I am really surprised by: > - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of > all tests !! > - how few failures I have on AIX, comparable to the amount of failures > on FC25/x86_64. > (However: - when trying with SMP and/or HiPE, I had hangs, so > I'm first building/testing without them > - we applied several AIX-specific patches) > - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . > (However: there are some more SKIPped tests: 133) > > Since result files contains about 600,000 lines, it is very difficult > for me to understand where/what are the issues. > I think we have understood that, in many cases, some environment is > missing/failing and thus the test is skipped. > > I'd like to get some help about: > - checking that I'm doing the Erlang configuration, build, and tests > correctly, > - finding some "official" Erlang tests results on Fedora / x86_64 > - how to analyze these results and find the real issues. > > I have attached to this email the index.html file produced on > Fedora25/x86_64 : > > /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html > > Thanks > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Erlang.crypto_tests.FC25.x86_64.tar.xz Type: octet/stream Size: 110504 bytes Desc: Erlang.crypto_tests.FC25.x86_64.tar.xz URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Erlang.crypto_tests.AIX.PPC64.tar.xz Type: octet/stream Size: 109200 bytes Desc: Erlang.crypto_tests.AIX.PPC64.tar.xz URL: From alex0player@REDACTED Thu Mar 2 17:11:46 2017 From: alex0player@REDACTED (Alex S.) Date: Thu, 2 Mar 2017 19:11:46 +0300 Subject: [erlang-questions] Understanding dialyzer errors better In-Reply-To: References: Message-ID: <3C384EFD-DCE2-4332-AD47-A479A86F36B9@gmail.com> The general approach I use in these cases is to slap specs on EVERYTHING and see which stuff Dialyzer complains about. > 2 ????? 2017 ?., ? 12:39, Raghav Karol ???????(?): > > I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. > > Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: > > ``` > src/bitcask.erl > 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 520: The created fun has no local return > 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'> > 1870: Function wrap_write_file/1 has no local return > 1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > > src/bitcask_fileops.erl > 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > 240: Function close_hintfile/1 has no local return > 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} > 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() > 314: The pattern 'undefined' can never match the type port() > > src/bitcask_merge_delete.erl > 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > ===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings > ===> Warnings occured running dialyzer: 17 > ``` > > Particularly confusing, are errors like the first one > > ``` > 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > ``` > > I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack. > > Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer. > > Best, > Raghav > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu Mar 2 17:16:33 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 2 Mar 2017 18:16:33 +0200 Subject: [erlang-questions] queue process with capacity In-Reply-To: References: Message-ID: <3FF51FED-6C9E-4D25-9E64-72E9F12AF0B6@gmail.com> Hello, There are multiple Erlang project that provides necessary queuing features. You can google them with ?erlang simple queue? The gen_server callback supplies you a From entity. This is a ?channel? to acknowledge request processing. You queue this entity internally and uses gen_server:reply(?) to acknowledge consumer/producer when it?s request is completed. ``` Module:handle_call(Request, From, State) -> Result http://erldocs.com/19.0/stdlib/gen_server.html?i=13&search=gen_server#reply/2 http://erldocs.com/19.0/stdlib/gen_server.html?i=4&search=gen_server#call/3 ``` Here is a sketch of the code ``` handle_call(dequeue, From, #state{q = Q} = State) -> {noreply, State#state{q = queue:in(From, Q)}}; handle_call({enqueue, X}, _, #state{q = Q} = State) -> gen_server:reply(queue:head(Q), X), ... ``` I hope you got an implementation idea. Please let me know if more info is needed Best Regards, Dmitry Kolesnikov | Digital Cossack dmkolesnikov@REDACTED > On Mar 2, 2017, at 4:44 PM, semmit mondo wrote: > > Hi, > > I need to implement a queue service that has a capacity. This means that the `in` actions has to be able to block the producer when the queue is full, and also the `out` action has to be able to block the consumer when the queue is empty. > > Sending in an item and then receiving with "after infinity" in the producer seems to be a working approach. But my concern is that this way the producer process is blocked and so it is unable to respond to any system messages. If I do this in a supervised process, do I break the contract between OTP and that process? Aren't all the processes supposed to answer certain low level messages that is handled by gen_server without me to even know about? Shouldn't supervised processes be blocked only in the gen_server's main receive loop? > > I feel that receiving indefinitely is something I should avoid. But how can I implement the queue in an OTP friendly way? > > S. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony.reix@REDACTED Thu Mar 2 17:33:45 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 16:33:45 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: , Message-ID: Hi Bj?rn Thanks for studying. I had already looked at these error messages. Here is below what I got about test.emulator_test . This is due to : ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1, I have this issue both on AIX and on FC25/x86_64. I guess that this issue should NOT appear on FC25/x86_64. However, I have it 48 times. On AIX, there are probably MANY reasons why this ts_install_cth failed. However, on FC25./x86_64, I am VERY surprised that this fails. However, the guy who ported Erlang on FC25/x86_64 said to me that he did not run the Erlang tests... Since I found that their RPM .spec file was missing a Request: instruction for the C++ compiler (leading to errors at build), I guess that their .spec file may contain other issues. However, I have NO idea about how I can find the root cause of this ts_install_cth failulre. I have already tried. Tony === Config function: ct_framework:init_per_suite/1 (click for source code) === Group properties: [{suite,a_SUITE}] === Config value: [{tc_logfile,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@REDACTED/tests.emulator_test.logs/run.2017-03-01_16.58.23/ct_framework.init_per_suite.html"}, {tc_group_properties,[{suite,a_SUITE}]}, {tc_group_path,[]}, {data_dir,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/emulator_test/a_SUITE_data/"}, {priv_dir,"/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@REDACTED/tests.emulator_test.logs/run.2017-03-01_16.58.23/log_private/"}] === Current directory is "/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/ct_run.test_server@REDACTED" === Started at 2017-03-01 16:58:31 Making "/root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/emulator_test/a_SUITE_data/Makefile" gcc -c -fPIC -D_THREAD_SAFE -D_REENTRANT -g -O2 -D_THREAD_SAFE -D_REENTRANT -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/sys/unix -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/include -I/root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/include/x86_64-unknown-linux-gnu -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_LIBM=1 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_LINUX_TCP_H=1 -DHAVE_SANE_LINUX_TCP_H=1 -DHAVE_POLL_H=1 -DHAVE_STRERROR=1 -DHAVE_VSNPRINTF=1 -DHAVE_USLEEP=1 -DHAVE_LIBRESOLV=1 -DHAVE_RES_GETHOSTBYNAME=1 -DHAVE_FINITE=1 timer_driver.c In file included from /root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam/erl_driver.h:40:0, from timer_driver.c:6: /root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam/erl_drv_nif.h:76:34: erreur fatale : erl_int_sizes_config.h : No such file or directory #include "erl_int_sizes_config.h" ^ compilation termin?e. Makefile:13 : la recette pour la cible ? timer_driver.so ? a ?chou?e make: *** [timer_driver.so] Erreur 1 *** User 2017-03-01 16:58:31.290 *** ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1, [{file,"ts_make.erl"}, {line,34}]}, {ts_lib,make_non_erlang_do, 2, [{file,"ts_lib.erl"}, {line,357}]}, {ts_install_cth, pre_init_per_suite,3, [{file,"ts_install_cth.erl"}, {line,105}]}, {ct_hooks,catch_apply,4, [{file,"ct_hooks.erl"}, {line,388}]}, {ct_hooks,call_generic,3, [{file,"ct_hooks.erl"}, {line,191}]}, {ct_hooks,call,4, [{file,"ct_hooks.erl"}, {line,240}]}, {ct_hooks,call,4, [{file,"ct_hooks.erl"}, {line,243}]}, {ct_hooks,call,4, [{file,"ct_hooks.erl"}, {line,227}]}]} *** CT Error Notification 2017-03-01 16:58:31.291 *** Error detected: ts_install_cth Full error description and stacktrace === Ended at 2017-03-01 16:58:31 === Location: {ct_framework,init_per_suite} === Reason: {ts_install_cth, {exit, {make_failed,make}, [{ts_make,make,1,[{file,"ts_make.erl"},{line,34}]}, {ts_lib,make_non_erlang_do,2, [{file,"ts_lib.erl"},{line,357}]}, {ts_install_cth,pre_init_per_suite,3, [{file,"ts_install_cth.erl"},{line,105}]}, {ct_hooks,catch_apply,4, [{file,"ct_hooks.erl"},{line,388}]}, {ct_hooks,call_generic,3, [{file,"ct_hooks.erl"},{line,191}]}, {ct_hooks,call,4,[{file,"ct_hooks.erl"},{line,240}]}, {ct_hooks,call,4,[{file,"ct_hooks.erl"},{line,243}]}, {ct_hooks,call,4, [{file,"ct_hooks.erl"},{line,227}]}]}} === *** init_per_suite failed. Skipping all cases. Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : bgustavsson@REDACTED [bgustavsson@REDACTED] de la part de Bj?rn Gustavsson [bjorn@REDACTED] Envoy? : jeudi 2 mars 2017 17:05 ? : REIX, Tony Cc : erlang-questions@REDACTED; GIRARDET, JEAN Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results On Thu, Mar 2, 2017 at 4:26 PM, REIX, Tony wrote: > Resending this email WITHOUT the attachment since it is too big to be sent > without some approval. > It seems to have been approved now. I have received the other email. I have taken a quick look at the log. The major problem is that many test cases that call init_per_SUITE fail when they attempt to compile C code during the running of the test suite, especially when running emulator_tests. You should be able to find more information from the generated html files. Starting with index.html, you could click on test.emulator_test. On that page, you could click on the links for the failed test cases. On the page for a failed init_per_SUITE test case, you should be able to see why the compilation failed. (Those failed test cases cause all test in the same module to be skipped.) On other thing so far: erl_boot_server_SUITE is not a good test suite. It has some hard-coded assumptions about our testing environment. It will fail outside our lab. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From hans.r.nilsson@REDACTED Thu Mar 2 18:05:12 2017 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Thu, 2 Mar 2017 18:05:12 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: <5312143d-2bc7-73d4-9856-4f3d0acb0562@ericsson.com> Hi, Thanks for the log files. I need another check by you. What does the function call crypto:ec_curves(). return on AIX.PPC64 and on .FC25.x86_64 ? /Hans On 03/02/2017 04:49 PM, REIX, Tony wrote: > Resending without the attachments since it is too big. > I'll sent the files to Hans directly. > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > ------------------------------------------------------------------------ > *De :* REIX, Tony > *Envoy? :* jeudi 2 mars 2017 16:46 > *? :* Hans Nilsson R; erlang-questions@REDACTED > *Objet :* RE:[erlang-questions] Desperately looking for official Erlang > tests results > > Hi Hans, > > On AIX, I have only 1 test failing: > > *OK FAIL SKIP TOTAL* > crypto_test: 111 *1* 3 115 > > > However, on FC25/x86_64, I have 4 more: > > *OK FAIL SKIP TOTAL* > crypto_test: 107 *5* 3 115 > > > > *1) AIX:* > > 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** > > crypto:strong_rand_mpint failed on line 1417 > Reason: low_entropy > > # cd otp_src_19.2 > > # ./bin/erl > /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec > Erlang/OTP 19 [erts-8.2] [source] > > Eshell V8.2 (abort with ^G) > 1> crypto:info_lib(). > [{<<"OpenSSL">>,268443823, > <<"OpenSSL 1.0.2j 26 Sep 2016">>}] > 2> crypto:supports(). > [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, > {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, > aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, > aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, > blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, > aes_gcm]}, > {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}] > 3> > > > *2) FC25/x86_64:* > > 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} *** > > 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 *** > crypto:ecdh_compute_key_nif failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 *** > crypto:ec_key_generate failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 > *** Common with AIX > crypto:strong_rand_mpint failed on line 1417 > Reason: low_entropy > > 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 *** > crypto:ec_key_generate failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 *** > Error detected: {'EXIT',low_entropy} > > # pwd > /root/rpmbuild/BUILD/otp-OTP-19.2.3 > > #./bin/erl > Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V8.2.2 (abort with ^G) > 1> crypto:info_lib(). > [{<<"OpenSSL">>,268443839, > <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}] > 2> crypto:supports(). > [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, > {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, > aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, > aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, > blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, > aes_gcm]}, > {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}] > > > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > ________________________________________ > De : Hans Nilsson R [hans.r.nilsson@REDACTED] > Envoy? : jeudi 2 mars 2017 15:39 > ? : erlang-questions@REDACTED; REIX, Tony > Objet : Re: [erlang-questions] Desperately looking for official Erlang > tests results > > Hi, > > I looked at the [crypto, public_key, ssh, ssl] group of applications. > It does look very bad indeed, except for public_key. > > So let's start from the bottom (i.e. crypto). I need the following: > > 1) start your tested Erlang and call > 1> crypto:info_lib(). > -- result 1 --- > 2> crypto:supports(). > -- result 2 --- > 3> > > What are the "result *" you get? > > 2) Mail me the crypto test results directory > > ..../ct_run.test_server@REDACTED/tests.crypto_test.logs > > /Hans > > On 03/02/2017 12:23 PM, REIX, Tony wrote: >> Hi, >> >> In order to understand which Erlang tests and features do not work fine >> within our port of version 19.2 on AIX, we need some "official" Erlang >> tests results for current version (19.2.3). >> However, searching on Erlang web-site, I was unable to find such tests >> results. >> >> So, I have built and tested Erlang on 3 environments: AIX 6.1, >> Fedora25/PPC64, and Fedora25/x86_64. >> I hope I've correctly ran the tests, but I'm not sure. >> >> In short, using nearly the same procedure (use Fedora .spec file), but >> without HiPE-SMP-Threads for AIX, I've got: >> >> OK FAIL SKIP TOTAL >> ------------------------------------------ >> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) >> FC25/PPC64 12,376 193 3,608 16,177 >> FC25/x86_64 12,407 288 3,475 16,170 >> >> I am really surprised by: >> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of >> all tests !! >> - how few failures I have on AIX, comparable to the amount of failures >> on FC25/x86_64. >> (However: - when trying with SMP and/or HiPE, I had hangs, so >> I'm first building/testing without them >> - we applied several AIX-specific patches) >> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . >> (However: there are some more SKIPped tests: 133) >> >> Since result files contains about 600,000 lines, it is very difficult >> for me to understand where/what are the issues. >> I think we have understood that, in many cases, some environment is >> missing/failing and thus the test is skipped. >> >> I'd like to get some help about: >> - checking that I'm doing the Erlang configuration, build, and tests >> correctly, >> - finding some "official" Erlang tests results on Fedora / x86_64 >> - how to analyze these results and find the real issues. >> >> I have attached to this email the index.html file produced on >> Fedora25/x86_64 : >> >> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html >> >> Thanks >> >> Cordialement, >> >> Tony Reix >> >> Bull - ATOS >> IBM Coop Architect & Technical Leader >> Office : +33 (0) 4 76 29 72 67 >> 1 rue de Provence - 38432 ?chirolles - France >> www.atos.net >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4101 bytes Desc: S/MIME Cryptographic Signature URL: From tony.reix@REDACTED Thu Mar 2 18:30:16 2017 From: tony.reix@REDACTED (REIX, Tony) Date: Thu, 2 Mar 2017 17:30:16 +0000 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: <5312143d-2bc7-73d4-9856-4f3d0acb0562@ericsson.com> References: , <5312143d-2bc7-73d4-9856-4f3d0acb0562@ericsson.com> Message-ID: Hi Here are the outputs. FC25/x86_64 : # cd otp-OTP-19.2.3/ # ./bin/erl Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.2.2 (abort with ^G) 1> crypto:ec_curves(). [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1, secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1, secp256k1,secp256r1,secp384r1,secp521r1,prime192v1, prime192v2,prime192v3,prime239v1,prime239v2,prime239v3, prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1, brainpoolP160t1|...] 2> AIX: # ./bin/erl /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec Erlang/OTP 19 [erts-8.2] [source] Eshell V8.2 (abort with ^G) 1> crypto:ec_curves(). [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1, secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1, secp256k1,secp256r1,secp384r1,secp521r1,prime192v1, prime192v2,prime192v3,prime239v1,prime239v2,prime239v3, prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1, brainpoolP160t1|...] 2> Cordialement, Tony Reix Bull - ATOS IBM Coop Architect & Technical Leader Office : +33 (0) 4 76 29 72 67 1 rue de Provence - 38432 ?chirolles - France www.atos.net ________________________________________ De : Hans Nilsson R [hans.r.nilsson@REDACTED] Envoy? : jeudi 2 mars 2017 18:05 ? : REIX, Tony; erlang-questions@REDACTED Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results Hi, Thanks for the log files. I need another check by you. What does the function call crypto:ec_curves(). return on AIX.PPC64 and on .FC25.x86_64 ? /Hans On 03/02/2017 04:49 PM, REIX, Tony wrote: > Resending without the attachments since it is too big. > I'll sent the files to Hans directly. > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > ------------------------------------------------------------------------ > *De :* REIX, Tony > *Envoy? :* jeudi 2 mars 2017 16:46 > *? :* Hans Nilsson R; erlang-questions@REDACTED > *Objet :* RE:[erlang-questions] Desperately looking for official Erlang > tests results > > Hi Hans, > > On AIX, I have only 1 test failing: > > *OK FAIL SKIP TOTAL* > crypto_test: 111 *1* 3 115 > > > However, on FC25/x86_64, I have 4 more: > > *OK FAIL SKIP TOTAL* > crypto_test: 107 *5* 3 115 > > > > *1) AIX:* > > 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** > > crypto:strong_rand_mpint failed on line 1417 > Reason: low_entropy > > # cd otp_src_19.2 > > # ./bin/erl > /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec > Erlang/OTP 19 [erts-8.2] [source] > > Eshell V8.2 (abort with ^G) > 1> crypto:info_lib(). > [{<<"OpenSSL">>,268443823, > <<"OpenSSL 1.0.2j 26 Sep 2016">>}] > 2> crypto:supports(). > [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, > {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, > aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, > aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, > blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, > aes_gcm]}, > {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}] > 3> > > > *2) FC25/x86_64:* > > 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} *** > > 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 *** > crypto:ecdh_compute_key_nif failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 *** > crypto:ec_key_generate failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 > *** Common with AIX > crypto:strong_rand_mpint failed on line 1417 > Reason: low_entropy > > 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 *** > crypto:ec_key_generate failed > Reason: badarg > > 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 *** > Error detected: {'EXIT',low_entropy} > > # pwd > /root/rpmbuild/BUILD/otp-OTP-19.2.3 > > #./bin/erl > Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V8.2.2 (abort with ^G) > 1> crypto:info_lib(). > [{<<"OpenSSL">>,268443839, > <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}] > 2> crypto:supports(). > [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, > {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, > aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, > aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, > blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, > aes_gcm]}, > {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}] > > > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > ________________________________________ > De : Hans Nilsson R [hans.r.nilsson@REDACTED] > Envoy? : jeudi 2 mars 2017 15:39 > ? : erlang-questions@REDACTED; REIX, Tony > Objet : Re: [erlang-questions] Desperately looking for official Erlang > tests results > > Hi, > > I looked at the [crypto, public_key, ssh, ssl] group of applications. > It does look very bad indeed, except for public_key. > > So let's start from the bottom (i.e. crypto). I need the following: > > 1) start your tested Erlang and call > 1> crypto:info_lib(). > -- result 1 --- > 2> crypto:supports(). > -- result 2 --- > 3> > > What are the "result *" you get? > > 2) Mail me the crypto test results directory > > ..../ct_run.test_server@REDACTED/tests.crypto_test.logs > > /Hans > > On 03/02/2017 12:23 PM, REIX, Tony wrote: >> Hi, >> >> In order to understand which Erlang tests and features do not work fine >> within our port of version 19.2 on AIX, we need some "official" Erlang >> tests results for current version (19.2.3). >> However, searching on Erlang web-site, I was unable to find such tests >> results. >> >> So, I have built and tested Erlang on 3 environments: AIX 6.1, >> Fedora25/PPC64, and Fedora25/x86_64. >> I hope I've correctly ran the tests, but I'm not sure. >> >> In short, using nearly the same procedure (use Fedora .spec file), but >> without HiPE-SMP-Threads for AIX, I've got: >> >> OK FAIL SKIP TOTAL >> ------------------------------------------ >> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) >> FC25/PPC64 12,376 193 3,608 16,177 >> FC25/x86_64 12,407 288 3,475 16,170 >> >> I am really surprised by: >> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of >> all tests !! >> - how few failures I have on AIX, comparable to the amount of failures >> on FC25/x86_64. >> (However: - when trying with SMP and/or HiPE, I had hangs, so >> I'm first building/testing without them >> - we applied several AIX-specific patches) >> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . >> (However: there are some more SKIPped tests: 133) >> >> Since result files contains about 600,000 lines, it is very difficult >> for me to understand where/what are the issues. >> I think we have understood that, in many cases, some environment is >> missing/failing and thus the test is skipped. >> >> I'd like to get some help about: >> - checking that I'm doing the Erlang configuration, build, and tests >> correctly, >> - finding some "official" Erlang tests results on Fedora / x86_64 >> - how to analyze these results and find the real issues. >> >> I have attached to this email the index.html file produced on >> Fedora25/x86_64 : >> >> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html >> >> Thanks >> >> Cordialement, >> >> Tony Reix >> >> Bull - ATOS >> IBM Coop Architect & Technical Leader >> Office : +33 (0) 4 76 29 72 67 >> 1 rue de Provence - 38432 ?chirolles - France >> www.atos.net >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Thu Mar 2 18:39:15 2017 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Thu, 2 Mar 2017 14:39:15 -0300 Subject: [erlang-questions] Understanding dialyzer errors better In-Reply-To: References: Message-ID: <1B1E3528-7318-4990-804A-915E1E3B9E35@inakanetworks.com> Hi Raghav, I have a private gig at my company: I dialyze projects for beer (true story). Let me show you an example of my mental process based on your commit. Let?s tackle like 197? It reads: _ = bitcask_fileops:close_for_writing(WriteFile), And dialyzer is saying that the call to that function will never return since it differs the first (and only) argument can?t be fresh nor undefined. There are 3 possibilities here: 1. bitcask_fileops:close_for_writing/1 actually accepts other kinds of arguments, but the clauses where those types are accepted are also questioned by dialyzer so, in dialyzer?s eyes, they can never happen. 2. WriteFile can actually be fresh or undefined, but dialyzer is fooled by a misplaced spec somewhere. 3. The code is actually broken (i.e. bitcask_fileops:close_for_writing/1 expects fresh or undefined and WriteFile can never be any of those) Let?s discard #3? I bet your app is tested with 100% coverage, so that can?t happen. Let?s aim for #1. We have to check the bitcask_fileops:close_for_writing/1?s code. ?and bingo! bitcask_fileops:close_for_writing/1 does have a third clause that, instead of fresh or undefined can receive a #filestate{} record (with some fixed params). Dialyzer must have produced a warning for that one, too? for sure. We check your list and? we found no warning in these lines (236-238). That?s bad, but check it out, that function is calling close_hintfile/1 and we do have warnings for that one, specifically: 240: Function close_hintfile/1 has no local return 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() Ok, so the function has no local return, that explains all! (o_O) Anyway, let?s check the last 2, shall we? On line 240, header of the first clause of the function, we?re using undefined in a record field that doesn?t accept undefined. The warning can be improved, but it can be roughly translated in this way: - Matching of pattern {?filestate?, ?} tagged with a record name? => The pattern with record filestate that you are using? - violates the declared type of #? => can never match an actual #filestate record because it?s using at least one sub-pattern that doesn?t match the type of the corresponding field. Here is the record definition:? On line 251, where we?re building a filestate record, the warning is much clearer: Dialyzer says we?re using the wrong value for field hintfd which is defined as port(). So, it?s pretty clear that our intention was to allow hintfd to be undefined. My next step here would be to fix that (as you did) and run rebar3 dialyzer again. We started from line 197? but we could?ve started this process from like 219 as well and reached the same conclusion, right? I?m not sure if this is what you were actually looking for, but maybe it helps anyway. Happy dialyzing! :) > On Mar 2, 2017, at 06:39, Raghav Karol wrote: > > I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. > > Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: > > ``` > src/bitcask.erl > 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 520: The created fun has no local return > 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > 1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'> > 1870: Function wrap_write_file/1 has no local return > 1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > > src/bitcask_fileops.erl > 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > 240: Function close_hintfile/1 has no local return > 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} > 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() > 314: The pattern 'undefined' can never match the type port() > > src/bitcask_merge_delete.erl > 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() > ===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings > ===> Warnings occured running dialyzer: 17 > ``` > > Particularly confusing, are errors like the first one > > ``` > 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') > ``` > > I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack. > > Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer. > > Best, > Raghav > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Mar 2 21:17:29 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 2 Mar 2017 20:17:29 +0000 (UTC) Subject: [erlang-questions] Making erlang more realtime? References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> Message-ID: <1194559117.897042.1488485849304@mail.yahoo.com> I currently need to stream 60 frames per second of video and I made a simple test case using erlang. I have 1 gen_server doing a erlang:send_after every 17ms, when the handle_info procs it calls os:perf_counter(1000) then compares the last time to the current, if the difference is > 20, print to console. Iv noticed that about every 2-3 seconds, the send_after arrives around in 20-23ms, this is not respecting the millisecond realtime nature that I thought Erlang can provide. Is there any VM arguments that could be passed to increase the realtime properties or other things that can be done? The testcase is just a naked rebar app with 1 gen_server not doing anything else. From dmytro.lytovchenko@REDACTED Thu Mar 2 21:26:59 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Thu, 2 Mar 2017 21:26:59 +0100 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: <1194559117.897042.1488485849304@mail.yahoo.com> References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: Making it "more realtime" has little sense, as it is not realtime platform and never was, and never intended to be. A few points to consider. 1. Erlang is not a realtime platform, i.e. it does not give you millisecond/microsecond guarantees. It is soft realtime, which guarantees that your code will eventually and periodically get CPU time with some +/- 100 ms precision depending on the current load. 2. If you want such millisecond precision you could write the critical decoding/feeding part in native code with an appropriate tool such as C/C++/Rust/Ocaml/Java/C# etc while Erlang will do the rest of the management and orchestration in your system. 3. One more way would be to let Erlang decode video ahead of time, so that you have few frames ready and cached for such a hiccup moment. Then even if some frame is late, there is always another frame ready to cover for it. Then let video player smooth the hiccups. I think streaming services do this at the cost of significant caching ahead. 2017-03-02 21:17 GMT+01:00 Vans S : > I currently need to stream 60 frames per second of video and I made a > simple test case using erlang. > > I have 1 gen_server doing a erlang:send_after every 17ms, when the > handle_info procs it calls > > os:perf_counter(1000) then compares the last time to the current, if the > difference is > 20, > print to console. > > Iv noticed that about every 2-3 seconds, the send_after arrives around in > 20-23ms, this is not > respecting the millisecond realtime nature that I thought Erlang can > provide. > > Is there any VM arguments that could be passed to increase the realtime > properties or other > things that can be done? > > The testcase is just a naked rebar app with 1 gen_server not doing > anything else. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Mar 2 22:31:54 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 2 Mar 2017 21:31:54 +0000 (UTC) Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: <96234916.939859.1488490314848@mail.yahoo.com> >?Erlang is not a realtime platform, i.e. it does not give you millisecond/microsecond guarantees. It is soft realtime, which guarantees that your code will eventually ?and?periodically get CPU time with some +/- 100 ms precision?> depending on the current load. This was my misunderstanding. I thought Erlang guaranteed each process gets a 1ms (+/- 1) timeslice aka 2000 reductions and everything executes as realtime as possible to 1 millisecond. And your #2 and #3, I already wiped up a test case in C and it seems il have to use Erlang as the auth/user management with more realtime app to handle the actual streaming. ?Even caching the frames, with Erlang It seems I am getting similar realtime performance degradation just doing message passing and netio. On Thursday, March 2, 2017 3:27 PM, Dmytro Lytovchenko wrote: Making it "more realtime" has little sense, as it is not realtime platform and never was, and never intended to be. A few points to consider.1. Erlang is not a realtime platform, i.e. it does not give you millisecond/microsecond guarantees. It is soft realtime, which guarantees that your code will eventually ?and?periodically get CPU time with some +/- 100 ms precision depending on the current load. 2. If you want such millisecond precision you could write the critical decoding/feeding part in native code with an appropriate tool such as C/C++/Rust/Ocaml/Java/C# etc while Erlang will do the rest of the management and orchestration in your system. 3. One more way would be to let Erlang decode video ahead of time, so that you have few frames ready and cached for such a hiccup moment. Then even if some frame is late, there is always another frame ready to cover for it. Then let video player smooth the hiccups. I think streaming services do this at the cost of significant caching ahead. 2017-03-02 21:17 GMT+01:00 Vans S : I currently need to stream 60 frames per second of video and I made a simple test case using erlang. I have 1 gen_server doing a erlang:send_after every 17ms, when the handle_info procs it calls os:perf_counter(1000) then compares the last time to the current, if the difference is > 20, print to console. Iv noticed that about every 2-3 seconds, the send_after arrives around in 20-23ms, this is not respecting the millisecond realtime nature that I thought Erlang can provide. Is there any VM arguments that could be passed to increase the realtime properties or other things that can be done? The testcase is just a naked rebar app with 1 gen_server not doing anything else. ______________________________ _________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/ listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ferenc.holzhauser@REDACTED Thu Mar 2 22:49:58 2017 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Thu, 2 Mar 2017 22:49:58 +0100 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: Probably it does not help an awful lot, but I recall an interesting project from couple of years ago called erlyvideo (later flussonic). I believe they have been busy with a similar challenge (video streaming with Erlang) and put a lot of work into tweaking performance. Perhaps you can still find interesting hints and discussions in the mailing list archive (or elsewhere) when searching for this project. On 2 March 2017 at 21:26, Dmytro Lytovchenko wrote: > Making it "more realtime" has little sense, as it is not realtime platform > and never was, and never intended to be. > > A few points to consider. > 1. Erlang is not a realtime platform, i.e. it does not give you > millisecond/microsecond guarantees. It is soft realtime, which guarantees > that your code will eventually and periodically get CPU time with some +/- > 100 ms precision depending on the current load. > > 2. If you want such millisecond precision you could write the critical > decoding/feeding part in native code with an appropriate tool such as > C/C++/Rust/Ocaml/Java/C# etc while Erlang will do the rest of the > management and orchestration in your system. > > 3. One more way would be to let Erlang decode video ahead of time, so that > you have few frames ready and cached for such a hiccup moment. Then even if > some frame is late, there is always another frame ready to cover for it. > Then let video player smooth the hiccups. I think streaming services do > this at the cost of significant caching ahead. > > 2017-03-02 21:17 GMT+01:00 Vans S : > >> I currently need to stream 60 frames per second of video and I made a >> simple test case using erlang. >> >> I have 1 gen_server doing a erlang:send_after every 17ms, when the >> handle_info procs it calls >> >> os:perf_counter(1000) then compares the last time to the current, if the >> difference is > 20, >> print to console. >> >> Iv noticed that about every 2-3 seconds, the send_after arrives around in >> 20-23ms, this is not >> respecting the millisecond realtime nature that I thought Erlang can >> provide. >> >> Is there any VM arguments that could be passed to increase the realtime >> properties or other >> things that can be done? >> >> The testcase is just a naked rebar app with 1 gen_server not doing >> anything else. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Mar 2 23:04:44 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 02 Mar 2017 22:04:44 +0000 Subject: [erlang-questions] Understanding dialyzer errors better In-Reply-To: <1B1E3528-7318-4990-804A-915E1E3B9E35@inakanetworks.com> References: <1B1E3528-7318-4990-804A-915E1E3B9E35@inakanetworks.com> Message-ID: The dialyzer often reports an error which really occurs somewhere else. Hence, if given an error which you don't know how to solve, it is often good to look for other errors which you do know how to solve. In turn, fixing those errors can clear up the call chain such that other errors become clearer or get resolved as no-error-in-the-first-place. The reason it is often confusing is that it is never wrong. Rather, the dialyzer tends to find a path in your code you never thought of. So you are thrown off by the structure of your own code which you think you know how runs. The trick is to throw away assumptions and go chase call chains. And fix those errors which you know how to fix early on. Once they are done, you can turn your eyes on the other errors. Case in point: I did a fix for a single dialyzer warning today. The error count went from about 43 to 24 on that project. The code looked like: foo(_, _, null) -> ok; foo(_, _, X) -> ...CBody... The dialyzer said that the function wouldn't work because my use differed from foo's success typing: (any(), any(), null). What the dialyzer is saying is "there is a bug in CBody, so the function only works if being passed null". Since foo/3 had many uses all over the code base, they were reported with the same error. On Thu, Mar 2, 2017 at 6:39 PM Brujo Benavides < fernando.benavides@REDACTED> wrote: Hi Raghav, I have a private gig at my company: I dialyze projects for beer (true story). Let me show you an example of my mental process based on your commit. Let?s tackle like 197? It reads: _ = bitcask_fileops:close_for_writing(WriteFile), And dialyzer is saying that the call to that function will never return since *it differs* the first *(and only)* argument can?t be *fresh* nor *undefined*. There are 3 possibilities here: 1. bitcask_fileops:close_for_writing/1 actually accepts other kinds of arguments, but the clauses where those types are accepted are also questioned by dialyzer so, in dialyzer?s eyes, they can never happen. 2. WriteFile can actually be *fresh* or *undefined*, but dialyzer is fooled by a misplaced spec somewhere. 3. The code is actually broken (i.e. bitcask_fileops:close_for_writing/1 expects fresh or undefined and WriteFile can never be any of those) Let?s discard #3? I bet your app is tested with 100% coverage, so that can?t happen. Let?s aim for #1. We have to check the bitcask_fileops:close_for_writing/1?s code. ?and *bingo!* bitcask_fileops:close_for_writing/1 does have a third clause that, instead of *fresh* or *undefined* can receive a *#filestate{}* record (with some fixed params). Dialyzer must have produced a warning for that one, too? for sure. We check your list and? we found no warning in these lines (236-238). That?s bad, but check it out, that function is calling *close_hintfile/1* and we do have warnings for that one, specifically: 240: Function close_hintfile/1 has no local return 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() Ok, so the function has no local return, that explains all! (o_O) Anyway, let?s check the last 2, shall we? On line 240, header of the first clause of the function, we?re using *undefined* in a record field that doesn?t accept undefined. The warning can be improved, but it can be roughly translated in this way: - Matching of pattern {?filestate?, ?} tagged with a record name? => The pattern with record filestate that you are using? - violates the declared type of #? => can never match an actual #filestate record because it?s using at least one sub-pattern that doesn?t match the type of the corresponding field. Here is the record definition:? On line 251, where we?re building a filestate record, the warning is much clearer: Dialyzer says we?re using the wrong value for field *hintfd *which is defined as *port()*. So, it?s pretty clear that our intention was to allow hintfd to be *undefined*. My next step here would be to fix that (as you did) and run rebar3 dialyzer again. We started from line 197? but we could?ve started this process from like 219 as well and reached the same conclusion, right? I?m not sure if this is what you were actually looking for, but maybe it helps anyway. Happy dialyzing! :) On Mar 2, 2017, at 06:39, Raghav Karol wrote: I spent a bit of time yesterday on some dialyzer warnings resulting from the singleton type 'undefined' no longer automatically to record fields types OTP-19. What threw me off were several related but hard to decipher warnings. Using OTP-19, `rebar3 dialyzer` on this commit https://github.com/basho/bitcask/commit/cd74f59bfe47a39878d7a56a55ce0ae723723677 produces: ``` src/bitcask.erl 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 219: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 520: The created fun has no local return 529: The call bitcask_fileops:close(FD::{_,_,_,_,_,_,_,_,_,_,_}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 717: The call bitcask_fileops:close(Outfile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 722: The call bitcask_fileops:close(TFile::#filestate{mode::'read_write',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 774: The call bitcask_fileops:close(F::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 1224: The call bitcask_fileops:close(File::#filestate{mode::'read_write',filename::string(),tstamp::integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::non_neg_integer(),ofs::'undefined' | non_neg_integer(),l_ofs::0,l_hbytes::0,l_hintcrc::0}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') 1719: The pattern <_Key, _Value, State, 0, LastErr> can never match the type <_,_,#bc_state{dirname::string(),write_file::'fresh' | 'undefined' | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::[any()],tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},write_lock::'undefined' | reference(),read_files::'undefined' | [{_,_,_,_,_,_,_,_,_,_,_}],max_file_size::'undefined' | integer(),opts::'undefined' | [any()],key_transform::'undefined' | fun(),keydir::reference(),read_write_p::'undefined' | integer(),tombstone_version::0 | 2},100,'undefined'> 1870: Function wrap_write_file/1 has no local return 1924: Record construction #filestate{filename::[any()],hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() src/bitcask_fileops.erl 166: Record construction #filestate{mode::'read_only',filename::string() | #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()},tstamp::integer(),hintfd::'undefined',hintcrc::0,ofs::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() 240: Function close_hintfile/1 has no local return 240: Matching of pattern {'filestate', _, _, _, _, 'undefined', _, _, _, _, _} tagged with a record name violates the declared type of #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} 251: Record construction #filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::'undefined',hintcrc::0,ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()} violates the declared type of field hintfd::port() 314: The pattern 'undefined' can never match the type port() src/bitcask_merge_delete.erl 151: Record construction #filestate{hintfd::'undefined',hintcrc::0,l_ofs::0,l_hbytes::0,l_hintcrc::0} violates the declared type of field hintfd::port() ===> Warnings written to /Users/raghav/github/riak_kv/deps/bitcask/_build/default/19.2.dialyzer_warnings ===> Warnings occured running dialyzer: 17 ``` Particularly confusing, are errors like the first one ``` 197: The call bitcask_fileops:close_for_writing(WriteFile::#filestate{mode::'read_only' | 'read_write' | 'undefined',filename::string(),tstamp::'undefined' | integer(),fd::'undefined' | port(),hintfd::port(),hintcrc::integer(),ofs::'undefined' | non_neg_integer(),l_ofs::non_neg_integer(),l_hbytes::non_neg_integer(),l_hintcrc::non_neg_integer()}) will never return since it differs in the 1st argument from the success typing arguments: ('fresh' | 'undefined') ``` I _think_ dialyzer is saying that the call `bitcask_fileops:close_for_writing(WriteFile#filestate{}, ...)` is not possible because of type violations when creating `#filestate{}` lower down the call stack. Would appreciate if someone could help understand this better and also suggest how one separates noise from real warnings with dialyzer. Best, Raghav _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Fri Mar 3 07:31:17 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 3 Mar 2017 07:31:17 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: Message-ID: On Thu, Mar 2, 2017 at 5:33 PM, REIX, Tony wrote: > Hi Bj?rn > > Thanks for studying. > > I had already looked at these error messages. Here is below what I got about test.emulator_test . > > This is due to : ts_install_cth failed! exit:{{make_failed,make},[{ts_make,make,1, > > I have this issue both on AIX and on FC25/x86_64. > I guess that this issue should NOT appear on FC25/x86_64. However, I have it 48 times. > > On AIX, there are probably MANY reasons why this ts_install_cth failed. > However, on FC25./x86_64, I am VERY surprised that this fails. > However, the guy who ported Erlang on FC25/x86_64 said to me that he did not run the Erlang tests... > Since I found that their RPM .spec file was missing a Request: instruction for the C++ compiler (leading to errors at build), I guess that their .spec file may contain other issues. Yes, it seems that not all necessary include files are included. Without those files, it is not possible to compile drivers or NIF libraries. To avoid debugging an incorrect RPM .spec and failing test cases at the same time, I suggest that you ignore the RPM for now and use our git repository, at least on FC25. Out wiki pages should contain enough information: https://github.com/erlang/otp/wiki If you would prefer to fix the RPM .spec, you must make sure that the following files are copied to the same directory as erl_nif.h: erl_fixed_size_int_types.h erl_int_sizes_config.h erl_memory_trace_parser.h That is, in your case, the files should be in: /root/rpmbuild/BUILD/otp-OTP-19.2.3/erts/emulator/beam /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From hans.r.nilsson@REDACTED Fri Mar 3 11:59:41 2017 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Fri, 3 Mar 2017 11:59:41 +0100 Subject: [erlang-questions] Desperately looking for official Erlang tests results In-Reply-To: References: <5312143d-2bc7-73d4-9856-4f3d0acb0562@ericsson.com> Message-ID: <2906f0d8-395d-5f03-a9f7-13b48820ec92@ericsson.com> Thanks! This just get weirder and weirder. Your output from crypto:* looks good, but the crypto test results made me expect other outputs... The crypto app has a C-part called via NIF, and the C-part interfaces libcrypto.so which in you case is OpenSSL 1.0.2j. It is *very* important to compile with the same version of .h files as the lib that is linked later. I don't say you have that error, but just as a reminder. Since Bj?rn found some strange things in the NIF/driver/emulator area, I would suggest to fix that first. If there then are any errors on crypto+ssh+ssl+public_key when the more basic emulator parts are error free, we could start investigating again. -Hans On 03/02/2017 06:30 PM, REIX, Tony wrote: > Hi* > > *Here are the outputs.* > > > FC25/x86_64 :* > > > # cd otp-OTP-19.2.3/ > # ./bin/erl > Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V8.2.2 (abort with ^G) > 1> crypto:ec_curves(). > [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1, > secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1, > secp256k1,secp256r1,secp384r1,secp521r1,prime192v1, > prime192v2,prime192v3,prime239v1,prime239v2,prime239v3, > prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1, > brainpoolP160t1|...] > 2> > > > *AIX:* > > # ./bin/erl > /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec > Erlang/OTP 19 [erts-8.2] [source] > > Eshell V8.2 (abort with ^G) > 1> crypto:ec_curves(). > [secp112r1,secp112r2,secp128r1,secp128r2,secp160k1, > secp160r1,secp160r2,secp192r1,secp192k1,secp224k1,secp224r1, > secp256k1,secp256r1,secp384r1,secp521r1,prime192v1, > prime192v2,prime192v3,prime239v1,prime239v2,prime239v3, > prime256v1,wtls6,wtls7,wtls8,wtls9,wtls12,brainpoolP160r1, > brainpoolP160t1|...] > 2> > > > > Cordialement, > > Tony Reix > > Bull - ATOS > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 ?chirolles - France > www.atos.net > > ________________________________________ > De : Hans Nilsson R [hans.r.nilsson@REDACTED] > Envoy? : jeudi 2 mars 2017 18:05 > ? : REIX, Tony; erlang-questions@REDACTED > Objet : Re: [erlang-questions] Desperately looking for official Erlang > tests results > > Hi, > > Thanks for the log files. I need another check by you. What does the > function call > > crypto:ec_curves(). > > return on AIX.PPC64 and on .FC25.x86_64 ? > > /Hans > > > On 03/02/2017 04:49 PM, REIX, Tony wrote: >> Resending without the attachments since it is too big. >> I'll sent the files to Hans directly. >> >> Cordialement, >> >> Tony Reix >> >> Bull - ATOS >> IBM Coop Architect & Technical Leader >> Office : +33 (0) 4 76 29 72 67 >> 1 rue de Provence - 38432 ?chirolles - France >> www.atos.net >> ------------------------------------------------------------------------ >> *De :* REIX, Tony >> *Envoy? :* jeudi 2 mars 2017 16:46 >> *? :* Hans Nilsson R; erlang-questions@REDACTED >> *Objet :* RE:[erlang-questions] Desperately looking for official Erlang >> tests results >> >> Hi Hans, >> >> On AIX, I have only 1 test failing: >> >> *OK FAIL SKIP TOTAL* >> crypto_test: 111 *1* 3 115 >> >> >> However, on FC25/x86_64, I have 4 more: >> >> *OK FAIL SKIP TOTAL* >> crypto_test: 107 *5* 3 115 >> >> >> >> *1) AIX:* >> >> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 *** >> >> crypto:strong_rand_mpint failed on line 1417 >> Reason: low_entropy >> >> # cd otp_src_19.2 >> >> # ./bin/erl >> /opt/freeware/src/packages/BUILD/otp_src_19.2/bin/powerpc-ibm-aix6.1/erlexec >> Erlang/OTP 19 [erts-8.2] [source] >> >> Eshell V8.2 (abort with ^G) >> 1> crypto:info_lib(). >> [{<<"OpenSSL">>,268443823, >> <<"OpenSSL 1.0.2j 26 Sep 2016">>}] >> 2> crypto:supports(). >> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, >> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, >> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, >> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, >> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, >> aes_gcm]}, >> {public_keys,[rsa,dss,dh,ec_gf2m,ecdsa,ecdh,srp]}] >> 3> >> >> >> *2) FC25/x86_64:* >> >> 1> Testing tests.crypto_test: *** FAILED {crypto_SUITE,init_per_group} *** >> >> 1> Testing tests.crypto_test: *** FAILED test case 26 of 115 *** >> crypto:ecdh_compute_key_nif failed >> Reason: badarg >> >> 1> Testing tests.crypto_test: *** FAILED test case 27 of 115 *** >> crypto:ec_key_generate failed >> Reason: badarg >> >> 1> Testing tests.crypto_test: *** FAILED test case 97 of 115 >> *** Common with AIX >> crypto:strong_rand_mpint failed on line 1417 >> Reason: low_entropy >> >> 1> Testing tests.crypto_test: *** FAILED test case 109 of 115 *** >> crypto:ec_key_generate failed >> Reason: badarg >> >> 1> Testing tests.crypto_test: *** FAILED test case 115 of 115 *** >> Error detected: {'EXIT',low_entropy} >> >> # pwd >> /root/rpmbuild/BUILD/otp-OTP-19.2.3 >> >> #./bin/erl >> Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:3:3] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> Eshell V8.2.2 (abort with ^G) >> 1> crypto:info_lib(). >> [{<<"OpenSSL">>,268443839, >> <<"OpenSSL 1.0.2k-fips 26 Jan 2017">>}] >> 2> crypto:supports(). >> [{hashs,[sha,sha224,sha256,sha384,sha512,md4,md5,ripemd160]}, >> {ciphers,[des3_cbc,des_ede3,des3_cbf,des3_cfb,aes_cbc, >> aes_cbc128,aes_cfb8,aes_cfb128,aes_cbc256,aes_ctr,aes_ecb, >> aes_ige256,des_cbc,des_cfb,des_ecb,blowfish_cbc, >> blowfish_cfb64,blowfish_ofb64,blowfish_ecb,rc2_cbc,rc4, >> aes_gcm]}, >> {public_keys,[rsa,dss,dh,ecdsa,ecdh,srp]}] >> >> >> >> Cordialement, >> >> Tony Reix >> >> Bull - ATOS >> IBM Coop Architect & Technical Leader >> Office : +33 (0) 4 76 29 72 67 >> 1 rue de Provence - 38432 ?chirolles - France >> www.atos.net >> >> ________________________________________ >> De : Hans Nilsson R [hans.r.nilsson@REDACTED] >> Envoy? : jeudi 2 mars 2017 15:39 >> ? : erlang-questions@REDACTED; REIX, Tony >> Objet : Re: [erlang-questions] Desperately looking for official Erlang >> tests results >> >> Hi, >> >> I looked at the [crypto, public_key, ssh, ssl] group of applications. >> It does look very bad indeed, except for public_key. >> >> So let's start from the bottom (i.e. crypto). I need the following: >> >> 1) start your tested Erlang and call >> 1> crypto:info_lib(). >> -- result 1 --- >> 2> crypto:supports(). >> -- result 2 --- >> 3> >> >> What are the "result *" you get? >> >> 2) Mail me the crypto test results directory >> >> ..../ct_run.test_server@REDACTED/tests.crypto_test.logs >> >> /Hans >> >> On 03/02/2017 12:23 PM, REIX, Tony wrote: >>> Hi, >>> >>> In order to understand which Erlang tests and features do not work fine >>> within our port of version 19.2 on AIX, we need some "official" Erlang >>> tests results for current version (19.2.3). >>> However, searching on Erlang web-site, I was unable to find such tests >>> results. >>> >>> So, I have built and tested Erlang on 3 environments: AIX 6.1, >>> Fedora25/PPC64, and Fedora25/x86_64. >>> I hope I've correctly ran the tests, but I'm not sure. >>> >>> In short, using nearly the same procedure (use Fedora .spec file), but >>> without HiPE-SMP-Threads for AIX, I've got: >>> >>> OK FAIL SKIP TOTAL >>> ------------------------------------------ >>> AIX 6.1 10,838 204 2,991 14,033 (no HiPE-SMP-Threads !) >>> FC25/PPC64 12,376 193 3,608 16,177 >>> FC25/x86_64 12,407 288 3,475 16,170 >>> >>> I am really surprised by: >>> - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of >>> all tests !! >>> - how few failures I have on AIX, comparable to the amount of failures >>> on FC25/x86_64. >>> (However: - when trying with SMP and/or HiPE, I had hangs, so >>> I'm first building/testing without them >>> - we applied several AIX-specific patches) >>> - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 . >>> (However: there are some more SKIPped tests: 133) >>> >>> Since result files contains about 600,000 lines, it is very difficult >>> for me to understand where/what are the issues. >>> I think we have understood that, in many cases, some environment is >>> missing/failing and thus the test is skipped. >>> >>> I'd like to get some help about: >>> - checking that I'm doing the Erlang configuration, build, and tests >>> correctly, >>> - finding some "official" Erlang tests results on Fedora / x86_64 >>> - how to analyze these results and find the real issues. >>> >>> I have attached to this email the index.html file produced on >>> Fedora25/x86_64 : >>> >>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html >>> >>> Thanks >>> >>> Cordialement, >>> >>> Tony Reix >>> >>> Bull - ATOS >>> IBM Coop Architect & Technical Leader >>> Office : +33 (0) 4 76 29 72 67 >>> 1 rue de Provence - 38432 ?chirolles - France >>> www.atos.net >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4101 bytes Desc: S/MIME Cryptographic Signature URL: From christopher.meiklejohn@REDACTED Fri Mar 3 12:06:31 2017 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Fri, 3 Mar 2017 12:06:31 +0100 Subject: [erlang-questions] [ANN] Call for Papers, PMLDC 2017 In-Reply-To: References: Message-ID: There was an error in the CFP, date correction: June 20th. My apologies for any confusion this caused. On Thu, Mar 2, 2017 at 9:47 AM, Christopher Meiklejohn wrote: > Second Workshop on Programming Models and Languages for Distributed Computing > Co-located with ECOOP 2017, Barcelona, Spain > Date: July 20, 2017 > > Whether you are programming a rich web application in JavaScript that > mutates state in the client?s browser, or you are building a massively > deployed mobile application that will operate with client state at the > device, it?s undeniable that you are building a distributed system! > > Two major challenges of programming distributed systems are > concurrency and partial failure. Concurrency of operations can > introduce accidental nondeterminism: computations may result in > different outcomes with the same inputs given scheduling differences > in the underlying system unless a synchronization mechanism is used to > enforce some order. Synchronization is typically expensive, and > reduces the efficiency of user applications. Partial failure, or the > failure of one or more components in a distributed system at one time, > introduces the challenge of knowing, when an operation fails, which > components of the operation completed successfully. To solve these > problems in practice on an unreliable, asynchronous network, atomic > commit protocols and timeouts as failure detection are typically used. > > Because of these challenges, early approaches to providing programming > abstractions for distributed computing that ignored them were > inherently misguided: the canonical example being the Remote Procedure > Call, still widely deployed in industry. > > The goal of this workshop is to discuss new approaches to distributed > programming that provide efficient execution and the elimination of > accidental nondeterminism resulting from concurrency and partial > failure. It will bring together both practitioners and theoreticians > from many disciplines: database theory, distributed systems, systems > programming, programming languages, data-centric programming, web > application development, and verification, to discuss the > state-of-the-art of distributed programming, advancement of the > state-of-the-art and paths forward to better application of theory in > practice. > > The main objectives of this workshop are the following: > > * To review the state-of-the-art research in languages, models, and > systems for distributed programming; > * To identify areas of critical need where research can advance the > state of the art; > * To create a forum for discussion; > * To present open problems from practitioners with an aim towards > motivating academic research on relevant problems faced by industry. > > In the spirit of both ECOOP and Curry On, this workshop aims at > favoring a multidisciplinary perspective by bringing together > researchers, developers, and practitioners from both academia and > industry. > > Submission Guidelines > > We solicit proposals for contributed talks. We recommend preparing > proposals of 2 pages, in ACM 2 column SIGPLAN style, written in > English and in PDF format. However, we will accept longer proposals or > submissions to other conferences, under the understanding that PC > members are only expected to read the first two pages of such longer > submissions. Authors with accepted papers will have the opportunity to > have their submission published on the ACM Digital Library. > > Workshop > > http://2017.ecoop.org/track/pmldc-2017-papers > > Submission Link > > https://easychair.org/conferences/?conf=pmldc2017 > > Important Dates > > Paper submission: Monday, 1 May 2017 > Workshop date: Thursday, 20 July 2017 > Authors notification: TBA > Final version: TBA > > Organizing Committee > > Christopher Meiklejohn > Universit? catholique de Louvain > > Heather Miller > Ecole Polytechnique F?d?rale de Lausanne > > Program Committee > > Carlos Baquero > Universidade do Minho > > Annette Bieniusa > Technischen Universit?t Kaiserslautern > > Carla Ferreira > Universidade Nova Lisboa > > Alexey Gotsman > IMDEA Software Institute > > Tyler McMullen > Fastly > > Rodrigo Rodrigues > Instituto Superior T?cnico, University of Lisboa & INESC-ID > > Ali Shoker > HASLab/INESC TEC & University of Minho > > KC Sivaramakrishnan > University of Cambridge > > Zach Tellman > Lacuna > > Peter Van Roy > Universit? catholique de Louvain > > Leif Walsh > Two Sigma Investments > > Jordan West > NOVA-LINCS > > Hongseok Yang > University of Oxford From benjamindadams@REDACTED Fri Mar 3 12:22:56 2017 From: benjamindadams@REDACTED (Benjamin Adams) Date: Fri, 3 Mar 2017 06:22:56 -0500 Subject: [erlang-questions] Run cowboy in a docker image In-Reply-To: References: Message-ID: Has anyone had luck running cowboy in docker with mk? I have tried with RUN make CMD ["make", "run"] Also CMD ["make", "run", "&"] Also Erlang execution line with console or start Has anyone had any luck with this? Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Fri Mar 3 14:30:35 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Fri, 3 Mar 2017 08:30:35 -0500 Subject: [erlang-questions] Run cowboy in a docker image In-Reply-To: References: Message-ID: <2BD9BEF7-60CC-4AEC-BF8C-EE4C577B4576@waisbrot.net> You're really asking a Docker question here. > CMD ["make", "run"] The list form of CMD is similar to execl(3). You should give an absolute path to "make". > CMD ["make", "run", "&"] This is passing a literal '&' to the 'make' executable (if you corrected its path). To accomplish what you're trying for, you'd use the string form: CMD make run & that line gets interpreted by the shell, so you don't need the absolute path and you can do other shell-ish things. But you wouldn't want to background the job because then the foreground job would be finished and Docker would terminate immediately, taking down your server process. > Has anyone had any luck with this? Yes; built a number of services with Cowboy and ran them in containers. I built a release inside the container and then ran that in foreground mode. From bjorn-egil.xb.dahlberg@REDACTED Fri Mar 3 14:38:18 2017 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Fri, 3 Mar 2017 14:38:18 +0100 Subject: [erlang-questions] Run cowboy in a docker image In-Reply-To: <2BD9BEF7-60CC-4AEC-BF8C-EE4C577B4576@waisbrot.net> References: <2BD9BEF7-60CC-4AEC-BF8C-EE4C577B4576@waisbrot.net> Message-ID: <632e9ae3-be2b-fa31-c7ee-8746b9169f62@ericsson.com> Short docker-erlang primer. Uses cowboy and rebar3 in the example. https://github.com/psyeugenic/docker-erlang Not saying it's *the* way to do it - just a way to do it. // Bj?rn-Egil On 03/03/2017 02:30 PM, Nathaniel Waisbrot wrote: > You're really asking a Docker question here. > > >> CMD ["make", "run"] > > The list form of CMD is similar to execl(3). You should give an absolute path to "make". > >> CMD ["make", "run", "&"] > > This is passing a literal '&' to the 'make' executable (if you corrected its path). To accomplish what you're trying for, you'd use the string form: > > CMD make run & > > that line gets interpreted by the shell, so you don't need the absolute path and you can do other shell-ish things. But you wouldn't want to background the job because then the foreground job would be finished and Docker would terminate immediately, taking down your server process. > > >> Has anyone had any luck with this? > > Yes; built a number of services with Cowboy and ran them in containers. I built a release inside the container and then ran that in foreground mode. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Mar 3 16:23:25 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 3 Mar 2017 18:23:25 +0300 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: main question here is: what does requires sending frames exactly once per 17 ms? Browser do not require it and erlang precision is enough. C may become mandatory if you want to send about 500-800 mbit of video, mux several streams to single multi-stream and maintain strict constant bitrate via UDP to send all this to dumb IP -> DVB-C transmitter. But usually erlang is enough: we are feeding satellite encoder from flussonic now and it is ok. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tthread@REDACTED Fri Mar 3 18:44:21 2017 From: tthread@REDACTED (Ilya Shcherbak) Date: Sat, 4 Mar 2017 00:44:21 +0700 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: hi, sorry, but the issue is unclear for me. can you describe a bit more specific a high level issue. why do you need to send frames in 17ms interval? it shouldn't be a requirement for a sender by design in my understanding. The constant delay doesn't look like a right solution for any reason: - network spikes won't be resolved due to difference I/P/B frame sizes; (need to have a smart algorithm of a sender queue) - playback sync won't be resolved due to network/receiver fluctuations; (need to have JB on a receiver) You can check an implementation of a sender queue in WebRTC project: modules/pacing/paced_sender.cc for instance. If you are transmitting high res video in a local network and facing an issue with transmitting you can try to tune socket buffer sizes for send/recv. Probably i missed something and you can add more information about an initial task. 2017-03-03 22:23 GMT+07:00 Max Lapshin : > main question here is: what does requires sending frames exactly once per > 17 ms? > > Browser do not require it and erlang precision is enough. > > C may become mandatory if you want to send about 500-800 mbit of video, > mux several streams to single multi-stream and maintain strict constant > bitrate via UDP to send all this to dumb IP -> DVB-C transmitter. > > But usually erlang is enough: we are feeding satellite encoder from > flussonic now and it is ok. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Sat Mar 4 00:28:23 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 3 Mar 2017 23:28:23 +0000 (UTC) Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: <1555640599.2071353.1488583703423@mail.yahoo.com> > Probably it does not help an awful lot, but I recall an interesting project from couple of years ago called erlyvideo (later flussonic) Pretty interesting project I would have to dig into the erlyvideo source sometime. > main question here is: what does requires sending frames exactly once per 17 ms? 17 ms was just a random number I picked. Since ~16.777ms represents 60fps. I wanted to see how much extra delay could potentially happen per frame. Realistically we want to forward frames as soon as we receive them. Erlang is just routing, not actually encoding the frames. I wanted to see if I could measure slowdowns between frames, for example if frame 1 spends ~6ms to be forwarded by erlang, we could arrive too late on the client end and the client might need to skip a frame. > Browser do not require it and Erlang precision is enough. Im thinking so too now, I have also gotten better result (lower latency) setting process priority to max. > - network spikes won't be resolved due to difference I/P/B frame sizes; (need to have a smart algorithm of a sender queue) > You can check an implementation of a sender queue in WebRTC project: modules/pacing/paced_sender.cc for instance. I need to consider this. Im not sure if it is possible to drop x264 frames though if the end-to-end transmission is bottle necked. I think the solution would be to drop to a lower bitrate. > - playback sync won't be resolved due to network/receiver fluctuations; (need to have JB on a receiver)Not sure what JB is. > Probably i missed something and you can add more information about an initial task. I am getting clumps of frames arriving in the client using websockets, which is a NaCL web browser application for Google Chrome (localnet). I am trying to narrow down the cause of it. I think I have arrived that it might be because I am sending large ws frames (130kb+) and websocket in chrome chokes, resulting in 5-8 frames being clumped up and presented to the decoder at the same time. I am going to try using a basic TCP connection to the client to see if that fixes the issue. This thought will unfortunately limit my application to running as an extension only. On Friday, March 3, 2017 12:44 PM, Ilya Shcherbak wrote: hi, sorry, but the issue is unclear for me. can you describe a bit more specific a high level issue. why do you need to send frames in 17ms interval? it shouldn't be a requirement for a sender by design in my understanding. The constant delay doesn't look like a right solution for any reason: - network spikes won't be resolved due to difference I/P/B frame sizes; (need to have a smart algorithm of a sender queue) - playback sync won't be resolved due to network/receiver fluctuations; (need to have JB on a receiver) You can check an implementation of a sender queue in WebRTC project: modules/pacing/paced_sender.cc for instance. If you are transmitting high res video in a local network and facing an issue with transmitting you can try to tune socket buffer sizes for send/recv. Probably i missed something and you can add more information about an initial task. 2017-03-03 22:23 GMT+07:00 Max Lapshin : main question here is: what does requires sending frames exactly once per 17 ms? > > >Browser do not require it and erlang precision is enough. > > >C may become mandatory if you want to send about 500-800 mbit of video, mux several streams to single multi-stream and maintain strict constant bitrate via UDP to send all this to dumb IP -> DVB-C transmitter. > > >But usually erlang is enough: we are feeding satellite encoder from flussonic now and it is ok. >______________________________ _________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/ listinfo/erlang-questions > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From vans_163@REDACTED Sat Mar 4 08:30:30 2017 From: vans_163@REDACTED (Vans S) Date: Sat, 04 Mar 2017 07:30:30 -0000 Subject: [erlang-questions] gen_statem confusion In-Reply-To: <924714402.803002.1484841669047@mail.yahoo.com> References: <1798520640.8723249.1484764836743.ref@mail.yahoo.com> <1798520640.8723249.1484764836743@mail.yahoo.com> <20170118192024.GG18549@fhebert-ltm2.internal.salesforce.com> <20170118204820.GH18549@fhebert-ltm2.internal.salesforce.com> <1858145896.95613.1484777025374@mail.yahoo.com> <20170119002742.GD26705@ferdmbp.local> <634078528.341606.1484794867558@mail.yahoo.com> <5ED7B915-476D-4CA2-80FD-6E9CADD2BE83@gmail.com> <20170119101348.GA36166@erix.ericsson.se> <924714402.803002.1484841669047@mail.yahoo.com> Message-ID: <1716449697.822124.1484842106762@mail.yahoo.com> Actually I jumped the gun here: using a state_timeout as next event which is a valid event_type() as said in the docs produces: ** Reason for termination = error:{bad_action_from_state_function, {next_event,state_timeout,hi}} So state_timeout cant be used here. timeout, cast, info can be used. {call, pid()} also cant be used. On Thursday, January 19, 2017 11:01 AM, Vans S wrote: > So if you in a state callback insert more next_event messages these are > inserted first in the queue, and a state_timeout 0 message is inserted last > in the queue. This clarifies things nicely. Also what if one was to use {next_event, state_timeout .. vs {state_timeout .. Also is there a priority such as external, internal, timeout, state_timeout, etc? Or anything with next_event goes first into the queue (or top of stack if using {next_event, external .. ?) regardless queue or stack the idea is the same, it will be first to dequeue/pop. And this will happen before receive is called. On Thursday, January 19, 2017 5:14 AM, Raimo Niskanen wrote: On Thu, Jan 19, 2017 at 11:59:37AM +0300, Alex S. wrote: > > > 19 ???. 2017 ?., ? 6:01, Vans S ???????(?): > > > > > > > > This wording is really confusing: > > > >> instead the the time-out event is enqueued to ensure > >> that it gets processed before any not yet received > >> external event > > Because when I tried a test case > > > > init() -> > > send(self(), hi), > > {_,_,_,0}. %0 timeout basically > > > > The 0 timeout procced before the 'hi' message. > > Using 1 as the timeout, 'hi' message procs first. > > > > But it says not yet received external event. To me send() seems like > > an external event. Hence confusing. > Well, it was sent, but not yet received by the gen_statem?s main loop. Perfectly correct. While gen_statem processes enqueued events it is guaranteed to not enter a receive statement, so no new messages are received until after ther state_timeout 0 message. Other enqueued messages that are being processed such as next_event messages will be processed before a state_timeout 0 message. So if you in a state callback insert more next_event messages these are inserted first in the queue, and a state_timeout 0 message is inserted last in the queue. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Sat Mar 4 09:02:18 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 4 Mar 2017 11:02:18 +0300 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: <1555640599.2071353.1488583703423@mail.yahoo.com> References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> <1555640599.2071353.1488583703423@mail.yahoo.com> Message-ID: 1) You need to start fixing your decoder that have to handle network delays. 2) if you write a proxy, then you don't need to worry about "realtime": receive frame and forward it. 6ms will be spent inside erlang only if you write at least 100-300K of useless LOC that do nothing. In flussonic we measure microseconds. 3) you need to stop thinking about "latency and realtime" in erlang and start looking at tcpdump and understanding that TCP is a source of latency problem in p2p video streaming. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sat Mar 4 15:03:36 2017 From: tony@REDACTED (Tony Rogvall) Date: Sat, 4 Mar 2017 15:03:36 +0100 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: <1555640599.2071353.1488583703423@mail.yahoo.com> References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> <1555640599.2071353.1488583703423@mail.yahoo.com> Message-ID: <4D1AF301-7205-4784-A0F5-016CF2F1F16F@rogvall.se> FYI > >> Browser do not require it and Erlang precision is enough. > Im thinking so too now, I have also gotten better result (lower latency) setting process priority to max. > From the Erlang documentation: http://erlang.org/doc/man/erlang.html#process_flag_priority "Note Priority level max is reserved for internal use in the Erlang runtime system, and is not to be used by others.? /Tony -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From erlang@REDACTED Sat Mar 4 16:37:51 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 4 Mar 2017 16:37:51 +0100 Subject: [erlang-questions] bitcoin blockchain Message-ID: A few questions. 1) How big is the current bitcoin blockchain ? 2) How can I download the first 1000 blocks in the bitcoin blockchain ? Is there a web server that can give me the N'th block???? (I don't want to download the entire blockchain - and then take the first 1000 blocks) 3) Has anybody written an erlang blockchain checker - I'd want to validate the first 1000 blocks Cheers /Joe From gordeev.vladimir.v@REDACTED Sat Mar 4 17:16:38 2017 From: gordeev.vladimir.v@REDACTED (Vladimir Gordeev) Date: Sat, 4 Mar 2017 19:16:38 +0300 Subject: [erlang-questions] erlang:trace_pattern with {return_trace}: is match against return value possible? Message-ID: Hey, I'm tracing http requests in my app by following pattern: erlang:trace_pattern({lhttpc, request, 9}, [{'_', [], [{return_trace}]}], [local]), It works great and sends me every request and result. The problem is that response might be quite huge (several of megabytes) and as far as I understand, it will copy it constantly to my tracer process. I see that I can use size() guard function in match specifications, and I wamt replace body binary with <<"skipped">> if it's too big. Is there a way to match against call returning value? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sat Mar 4 18:15:02 2017 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 4 Mar 2017 18:15:02 +0100 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: References: Message-ID: Hi Joe, This site might help : https://blockexplorer.com See API : https://blockexplorer.com/api/block-index https://blockexplorer.com/api/block cheers Le 04/03/2017 ? 16:37, Joe Armstrong a ?crit : > A few questions. > > 1) How big is the current bitcoin blockchain ? > > 2) How can I download the first 1000 blocks in the bitcoin blockchain ? > Is there a web server that can give me the N'th block???? > > (I don't want to download the entire blockchain - and then take the > first 1000 blocks) > > 3) Has anybody written an erlang blockchain checker - I'd want > to validate the first 1000 blocks > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Sat Mar 4 22:42:41 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 04 Mar 2017 21:42:41 +0000 Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: <1194559117.897042.1488485849304@mail.yahoo.com> References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: On Thu, Mar 2, 2017 at 9:17 PM Vans S wrote: > I currently need to stream 60 frames per second of video and I made a > simple test case using erlang. > > Erlang is in many ways a "best effort" realtime system. It tries very hard to deliver timers and handle deadlines quickly, but it has no built-in provisioning to guarantee that they will be delivered on time. Some systems, which people call "hard" realtime systems, work by you requesting a certain periodic schedule. For instance: I want to be called every 17ms and I will spend at most 2ms of processing time. The system then either accepts that schedule or rejects it based on current workload. Such a system can still fail: CPUs can go bad, and even if you run multiple CPUs in unison and vote on the correct result, the whole system can go bad due to too many failures. Erlangs best-effort delivery can go bad because of numerous reasons: * You requested 1,000,000 processes to be woken up on the same point in time. Some will have to wait and come later. * You run on virtualized hardware, and some other task got the CPU. * You are on battery and not AC on your laptop and the system runs at different clock rates. * You are running OSX, which recently started saving power by piggybacking several wakeups together. * You enabled another CPU, which has to become online and then filled with the relevant data before it can process. Not all of these are the fault of the Erlang subsystem. In general, the only way to get a timely deadline is to overprovision the system enough that it has lots of free time to do what it wants. That said, I'll say the same as Dmytro, Max, Ferenc and Ilya: try to build your system such that it starts processing as soon as a frame arrives and push it onwards. In a certain sense, what you should be worried about is the frame time, not the frame delay. That is, you should worry about how long it takes to process a frame, because if you start falling behind there, you are in trouble. You may find you don't need utter precision, but can extrapolate a bit around the actual point in time you were woken up. This is where some buffering can be nice. Many computer games toy with this idea: they create the illusion of a 16.7ms frame time, but internally things run on completely different schedules, and it runs with lag: whenever you have a frame in the graphics card buffer, the next one is being painted. This introduces one frame of "lag", but humans are able to learn how to ignore that when they frag people in CS:GO :P In short: you are in principle requesting that Erlang does something it cannot do in its current form. You can set up an SLA and then verify that the Erlang system keeps within that SLA. e.g., "for my system, timer delivery happens in the same millisecond that I requested 99.999% of the time". If it doesn't, you can try tuning the system in order to meet that requirement: do not run in the cloud, virtualized, eliminte other processes on the machine. Force the CPU clock cycle to be the same all the time. Do not sleep the CPU. Turn off the NIC so you don't get interference on the memory bus. Replace the OS scheduler with one that gives guaranteed schedules, and so on. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Sun Mar 5 18:28:36 2017 From: paulperegud@REDACTED (Paul Peregud) Date: Sun, 5 Mar 2017 18:28:36 +0100 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: References: Message-ID: There were an attempt to write a bitcoin full node in Erlang. It's here: https://github.com/r-willis/biten On Sat, Mar 4, 2017 at 6:15 PM, PAILLEAU Eric wrote: > Hi Joe, > > This site might help : https://blockexplorer.com > > See API : https://blockexplorer.com/api/block-index > https://blockexplorer.com/api/block > > cheers > > > > Le 04/03/2017 ? 16:37, Joe Armstrong a ?crit : >> >> A few questions. >> >> 1) How big is the current bitcoin blockchain ? >> >> 2) How can I download the first 1000 blocks in the bitcoin blockchain ? >> Is there a web server that can give me the N'th block???? >> >> (I don't want to download the entire blockchain - and then take the >> first 1000 blocks) >> >> 3) Has anybody written an erlang blockchain checker - I'd want >> to validate the first 1000 blocks >> >> Cheers >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Paul Peregud +48602112091 From sashaafm@REDACTED Sun Mar 5 16:11:10 2017 From: sashaafm@REDACTED (Sasha Fonseca) Date: Sun, 5 Mar 2017 15:11:10 +0000 Subject: [erlang-questions] How do mailboxes relate to reduction counts? Message-ID: I?ve read the following blog post (http://jlouisramblings.blogspot.pt/2013/01/how-erlang-does-scheduling.html ), and it has the following excerpt which got me thinking: Both processes and ports have a "reduction budget" of 2000 reductions. Any operation in the system costs reductions. This includes function calls in loops, calling built-in-functions (BIFs), garbage collecting heaps of that process[n1], storing/reading from ETS, sending messages (The size of the recipients mailbox counts, large mailboxes are more expensive to send to). I?m wondering how does the size of a process?s mailbox relates the reduction count? Is it due to each message having to be matched against several message patterns in a case condition, for example? Also how large would the mailbox have to be to noticed any kind of impact in, let?s say, a server process? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Sun Mar 5 20:07:10 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sun, 5 Mar 2017 20:07:10 +0100 Subject: [erlang-questions] How do mailboxes relate to reduction counts? In-Reply-To: References: Message-ID: Increased cost to send to larger mailboxes is a sort of simple approach to work balancing. The idea is that if mailbox of some process grows, we should (naively) try and skew scheduling in such a way so it gets more processing time to clean up that mailbox. So we increase the cost of sending to it, and this makes senders schedule out faster and wait for their turn longer (as they get shorter time to run). This in theory might make the "busy" process with the large mailbox get more CPU and fix the problem faster. Of course this *naive *approach won't work if the mailbox grows because the receiver ignores new messages. Such system is sick and must be debugged and code must be modified so that extra messages are flushed (dropped). This is *artificial *feature, as sending is a very cheap operation. Increased cost of receive is another thing, and indeed it depends on how many instructions are executed to scan and find the right message in *selective *receive. 2017-03-05 16:11 GMT+01:00 Sasha Fonseca : > I?ve read the following blog post (http://jlouisramblings. > blogspot.pt/2013/01/how-erlang-does-scheduling.html), and it has the > following excerpt which got me thinking: > > *Both processes and ports have a "reduction budget" of 2000 reductions. > Any operation in the system costs reductions. This includes function calls > in loops, calling built-in-functions (BIFs), garbage collecting heaps of > that process[n1], storing/reading from ETS, sending messages (The size of > the recipients mailbox counts, large mailboxes are more expensive to send > to). * > > I?m wondering how does the size of a process?s mailbox relates the > reduction count? Is it due to each message having to be matched against > several message patterns in a case condition, for example? Also how large > would the mailbox have to be to noticed any kind of impact in, let?s say, a > server process? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michel.boaventura@REDACTED Sun Mar 5 20:10:31 2017 From: michel.boaventura@REDACTED (Michel Boaventura) Date: Sun, 5 Mar 2017 16:10:31 -0300 Subject: [erlang-questions] How do mailboxes relate to reduction counts? In-Reply-To: References: Message-ID: On Erlang Programming - A Concurrent Approach to Software Development the authors (Francesco Cesarini and Simon Thompson) explain this: Processes are said to act as bottlenecks when, over time, they are sent > messages at a faster rate than they can handle them, resulting in large > mailbox queues. How do processes with many messages in their inbox behave? > The answer is badly. > First, the process itself, through a selective receive, might match only a > specific type of message. If the message is the last one in the mailbox > queue, the whole mailbox has to be traversed before this message is > successfully matched. This causes a performance penalty that is often > visible through high CPU consumption. > Second, processes sending messages to a process with a long message queue > are penalized by increasing the number of reductions it costs to send the > message. This is an attempt by the runtime system to allow processes with > long message queues to catch up by slowing down the processes sending the > messages in the first place. The latter bottleneck often manifests itself > in a reduction of the overall throughput of the system. > The only way to discover whether there are any bottlenecks is to observe > the throughput and message queue buildup when stress-testing the system. > Simple remedies to message queue problems can be achieved by optimizing the > code and fine-tuning the operating system and VM settings. > Another way to slow down message queue buildups is by suspending the > processes generating the messages until they receive an acknowledgment that > the message they have sent has been received and handled, effectively > creating a synchronous call. Replacing asynchronous calls with synchronous > ones will reduce the maximum throughput of the system when running under > heavy load, but never as much as the penalty paid when message queues start > building up. Where bottlenecks are known to occur, it is safer to reduce > the throughput by introducing synchronous calls, and thus guaranteeing a > constant throughput of requests in the system with no degradation of > service under heavy loads. So I think the idea behind this is to give the overloaded process a change to keep up with the other ones. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Mon Mar 6 01:46:19 2017 From: vans_163@REDACTED (Vans S) Date: Mon, 6 Mar 2017 00:46:19 +0000 (UTC) Subject: [erlang-questions] Making erlang more realtime? In-Reply-To: References: <1194559117.897042.1488485849304.ref@mail.yahoo.com> <1194559117.897042.1488485849304@mail.yahoo.com> Message-ID: <1750937787.2909093.1488761179707@mail.yahoo.com> >?If it doesn't, you can try tuning the system in order to meet that requirement: do not run in the cloud, virtualized, eliminte other processes on the machine. Force the CPU clock cycle to be the same all the time.> Do not sleep the CPU. Turn off the NIC so you don't get interference on the memory bus. Replace the OS scheduler with one that gives guaranteed schedules, and so on. I started doing this to hunt down bottlenecks, I found issues all throughout my stack. I am going to try again sometime soon. ?Also I want to try tuning the kernel params for lower latency, I think now that its not Erlangs fault at all here and Erlang can be quite performant down to ~1ms consistent. The send_after delayed timer resolution issues I was having I think are related to the kernel; proper isolation of erlang runtime from the rest of the OS and scheduler pinning I think will make erlang?much more realtime. On Saturday, March 4, 2017 4:42 PM, Jesper Louis Andersen wrote: On Thu, Mar 2, 2017 at 9:17 PM Vans S wrote: I currently need to stream 60 frames per second of video and I made a simple test case using erlang. Erlang is in many ways a "best effort" realtime system. It tries very hard to deliver timers and handle deadlines quickly, but it has no built-in provisioning to guarantee that they will be delivered on time. Some systems, which people call "hard" realtime systems, work by you requesting a certain periodic schedule. For instance: I want to be called every 17ms and I will spend at most 2ms of processing time. The system then either accepts that schedule or rejects it based on current workload. Such a system can still fail: CPUs can go bad, and even if you run multiple CPUs in unison and vote on the correct result, the whole system can go bad due to too many failures. Erlangs best-effort delivery can go bad because of numerous reasons: * You requested 1,000,000 processes to be woken up on the same point in time. Some will have to wait and come later.* You run on virtualized hardware, and some other task got the CPU.* You are on battery and not AC on your laptop and the system runs at different clock rates.* You are running OSX, which recently started saving power by piggybacking several wakeups together.* You enabled another CPU, which has to become online and then filled with the relevant data before it can process. Not all of these are the fault of the Erlang subsystem. In general, the only way to get a timely deadline is to overprovision the system enough that it has lots of free time to do what it wants. That said, I'll say the same as Dmytro, Max, Ferenc and Ilya: try to build your system such that it starts processing as soon as a frame arrives and push it onwards. In a certain sense, what you should be worried about is the frame time, not the frame delay. That is, you should worry about how long it takes to process a frame, because if you start falling behind there, you are in trouble. You may find you don't need utter precision, but can extrapolate a bit around the actual point in time you were woken up. This is where some buffering can be nice. Many computer games toy with this idea: they create the illusion of a 16.7ms frame time, but internally things run on completely different schedules, and it runs with lag: whenever you have a frame in the graphics card buffer, the next one is being painted. This introduces one frame of "lag", but humans are able to learn how to ignore that when they frag people in CS:GO :P In short: you are in principle requesting that Erlang does something it cannot do in its current form. You can set up an SLA and then verify that the Erlang system keeps within that SLA. e.g., "for my system, timer delivery happens in the same millisecond that I requested 99.999% of the time". If it doesn't, you can try tuning the system in order to meet that requirement: do not run in the cloud, virtualized, eliminte other processes on the machine. Force the CPU clock cycle to be the same all the time. Do not sleep the CPU. Turn off the NIC so you don't get interference on the memory bus. Replace the OS scheduler with one that gives guaranteed schedules, and so on. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raghav.karol@REDACTED Mon Mar 6 11:46:57 2017 From: raghav.karol@REDACTED (Raghav Karol) Date: Mon, 6 Mar 2017 11:46:57 +0100 Subject: [erlang-questions] Dialyzer OTP 19.2 warning for tuple call syntax Message-ID: While trying to run dialyzer on `mochiweb` using OTP 19.2, I observed warnings when a tuple, M = `{?MODULE, Args}` is used to call a function like `M:function()`. The compiler and runtime still allow these calls. Here is a simple module that produces the warning. %% BEGIN INLINE CODE -module(dialyzer_warning). -export([new/0, function/1, function/0, main/0]). -ifdef(TEST). -include_lib("eunit/include/eunit.hrl"). -endif. new() -> {?MODULE, ['this', 'is', 'fun']}. function(S = {?MODULE, _Args}) -> S. function() -> ?MODULE. main() -> M0 = ?MODULE, ?MODULE = M0:function(), M1 = new(), M1 = M1:function(). -ifdef(TEST). module_call_test() -> M = ?MODULE, ?MODULE = M:function(). tuple_call_test() -> M = new(), M = M:function(). -endif. %% END INLINE CODE Running dialyzer like `(. ~/erlang/19.2/activate && rebar3 dialyzer)`, produces: src/dialyzer_warning.erl 21: Function main/0 has no local return 26: The call M1:'function'() requires that M1 is of type atom() not {'dialyzer_bug',['fun' | 'is' | 'this',...]} >From a previous post [1] I understand this feature, call with module-as-a-tuple should still be supported and I'm wondering how does one handle using OTP 19.2 on an existing codebase like mochiweb that makes use of module-as-a-tuple? [1] http://erlang.org/pipermail/erlang-bugs/2015-May/004986.html -- Raghav -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Mon Mar 6 15:45:27 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Mon, 6 Mar 2017 16:45:27 +0200 Subject: [erlang-questions] HIPE and on_load Message-ID: Hello, When I compile using HIPE: ERL_COMPILER_OPTIONS="[native,{hipe, [o3]}]" All dependencies using NIF's are failing to compile: EXITED with reason {'trans_fun/2',on_load} @hipe_beam_to_icode:1148 Is this an expected behavior or anything I can do to bypass this ? Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex0player@REDACTED Mon Mar 6 15:46:41 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 6 Mar 2017 17:46:41 +0300 Subject: [erlang-questions] HIPE and on_load In-Reply-To: References: Message-ID: <10603DCE-C857-42AC-86D7-25285F4BEB6A@gmail.com> This is an expected behaviour, I think, as there?s no trampoline in native-compiled functions to load NIFs into. > 6 ????? 2017 ?., ? 17:45, Caragea Silviu ???????(?): > > Hello, > > When I compile using HIPE: ERL_COMPILER_OPTIONS="[native,{hipe, [o3]}]" > > All dependencies using NIF's are failing to compile: > > EXITED with reason {'trans_fun/2',on_load} @hipe_beam_to_icode:1148 > > Is this an expected behavior or anything I can do to bypass this ? > > Silviu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kostis@REDACTED Mon Mar 6 15:51:01 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 6 Mar 2017 16:51:01 +0200 Subject: [erlang-questions] HIPE and on_load In-Reply-To: References: Message-ID: <7a311245-d0e9-711e-6502-63863e695a50@cs.ntua.gr> On 03/06/17 16:45, Caragea Silviu wrote: > Hello, > > When I compile using HIPE: ERL_COMPILER_OPTIONS="[native,{hipe, [o3]}]" > > All dependencies using NIF's are failing to compile: > > EXITED with reason {'trans_fun/2',on_load} > @hipe_beam_to_icode:1148 > > Is this an expected behavior or anything I can do to bypass this ? Yes, this is expected behaviour. The on_load attribute instructs the BEAM loader to do its magic when loading this module (typically containing NIFs). Files containing NIFs typically only contain very minimal Erlang code so they do not benefit much from native code compilation anyway. So simply do not compile them to native code. Perhaps adding an attribute -compile(no_native). on the source of the file will solve the problem? Kostis From silviu.cpp@REDACTED Mon Mar 6 15:57:34 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Mon, 6 Mar 2017 16:57:34 +0200 Subject: [erlang-questions] HIPE and on_load In-Reply-To: <7a311245-d0e9-711e-6502-63863e695a50@cs.ntua.gr> References: <7a311245-d0e9-711e-6502-63863e695a50@cs.ntua.gr> Message-ID: Thanks, I'll try to see if there is a way in mix file to specifiy what projects should be built without HIPE. It's not so simple to take every single dep and modify :) Silviu On Mon, Mar 6, 2017 at 4:51 PM, Kostis Sagonas wrote: > On 03/06/17 16:45, Caragea Silviu wrote: > >> Hello, >> >> When I compile using HIPE: ERL_COMPILER_OPTIONS="[native,{hipe, [o3]}]" >> >> All dependencies using NIF's are failing to compile: >> >> EXITED with reason {'trans_fun/2',on_load} >> @hipe_beam_to_icode:1148 >> >> Is this an expected behavior or anything I can do to bypass this ? >> > > Yes, this is expected behaviour. > > The on_load attribute instructs the BEAM loader to do its magic when > loading this module (typically containing NIFs). Files containing NIFs > typically only contain very minimal Erlang code so they do not benefit much > from native code compilation anyway. So simply do not compile them to > native code. > > Perhaps adding an attribute > > -compile(no_native). > > on the source of the file will solve the problem? > > Kostis > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Mon Mar 6 16:38:23 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 6 Mar 2017 16:38:23 +0100 Subject: [erlang-questions] Tracking progress of mnesia start Message-ID: Hello! I'd like to provide feedback to the user when mnesia is started in out application (something like "4 out of 16 tables loaded"). My first idea was to mnesia:subscribe for events, but I can subscribe only after mnesia is started, so I might miss some tables. My second idea is to turn on debugging, add an event module (both via setting environment values). It kind of works, I receive events like {mnesia_system_event, {mnesia_info, "Table ~w is loaded on ~w. s=~w, r=~w, lc=~w, f=~w, m=~w~n", [album,'c3@REDACTED',ram_copies,nowhere,false,false,false]}}, but it depends on the format of debugging message and even though this message hasn't changed in the last 7 years, I'm a bit uneasy about it. Do you have any better idea? From k.petrauskas@REDACTED Mon Mar 6 17:06:42 2017 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Mon, 6 Mar 2017 18:06:42 +0200 Subject: [erlang-questions] Tracking progress of mnesia start In-Reply-To: References: Message-ID: Hi, I use mnesia:wait_for_tables/2 in a loop with small timeout for that. This function returns a list of missing tables in the case of timeout. Karolis On Mon, Mar 6, 2017 at 5:38 PM, Attila Rajmund Nohl wrote: > Hello! > > I'd like to provide feedback to the user when mnesia is started in out > application (something like "4 out of 16 tables loaded"). My first > idea was to mnesia:subscribe for events, but I can subscribe only > after mnesia is started, so I might miss some tables. My second idea > is to turn on debugging, add an event module (both via setting > environment values). It kind of works, I receive events like > > {mnesia_system_event, > {mnesia_info, > "Table ~w is loaded on ~w. s=~w, r=~w, lc=~w, f=~w, m=~w~n", > [album,'c3@REDACTED',ram_copies,nowhere,false,false,false]}}, > > but it depends on the format of debugging message and even though this > message hasn't changed in the last 7 years, I'm a bit uneasy about it. > Do you have any better idea? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Mon Mar 6 17:08:57 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 06 Mar 2017 16:08:57 +0000 Subject: [erlang-questions] Tracking progress of mnesia start In-Reply-To: References: Message-ID: await_loading(Tables) -> Parent = self(), _ = [spawn_link(fun() -> mnesia:wait_for_tables(T, ?TIMEOUT), Parent ! {loaded_table, T} end) || T <- Tables], await_loading_join(Tables, 0). await_loading_join(Tables, Count) -> receive {loaded_table, T} -> await_loading(Tables -- [T], Count+1); after ?ARGH -> exit(argh) end. Totally untested, but should easily be adaptable to something robust which does things correctly. You will need some serious error-handling pathing in the above such that you can handle the timeouts properly. Probably by sending back an answer to the Parent with the fact that a certain load timed out. Also, you could do some stuff on top of this by logging how many tables are loaded every 5 seconds via a timer and so on. On Mon, Mar 6, 2017 at 4:38 PM Attila Rajmund Nohl wrote: > Hello! > > I'd like to provide feedback to the user when mnesia is started in out > application (something like "4 out of 16 tables loaded"). My first > idea was to mnesia:subscribe for events, but I can subscribe only > after mnesia is started, so I might miss some tables. My second idea > is to turn on debugging, add an event module (both via setting > environment values). It kind of works, I receive events like > > {mnesia_system_event, > {mnesia_info, > "Table ~w is loaded on ~w. s=~w, r=~w, lc=~w, f=~w, > m=~w~n", > [album,'c3@REDACTED > ',ram_copies,nowhere,false,false,false]}}, > > but it depends on the format of debugging message and even though this > message hasn't changed in the last 7 years, I'm a bit uneasy about it. > Do you have any better idea? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Mar 7 01:17:35 2017 From: rvirding@REDACTED (Robert Virding) Date: Tue, 7 Mar 2017 01:17:35 +0100 Subject: [erlang-questions] Cleaning up the dialyzer interface Message-ID: Is anyone working on cleaning up the dialyzer interface? It does some strange things like for no discernable reason requiring all files to be either .erl files or .beam files and not allowing a combination. Also it it is not in any way extensible. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From raghav.karol@REDACTED Tue Mar 7 10:02:08 2017 From: raghav.karol@REDACTED (Raghav Karol) Date: Tue, 7 Mar 2017 10:02:08 +0100 Subject: [erlang-questions] Common Test last test report In-Reply-To: <73d81ce9-ecbc-e823-36aa-e70d8a7366b3@ninenines.eu> References: <73d81ce9-ecbc-e823-36aa-e70d8a7366b3@ninenines.eu> Message-ID: +1 to this request. I love common_test, but and this is my biggest pain point. It's either a few clicks in the browser to the see the last failure or a `find sort cat` like command to see the latest report. On Sat, Feb 25, 2017 at 12:20 AM, Lo?c Hoguin wrote: > Hello, > > It's still difficult to get to the last test report in Common Test. > Currently the URI looks like this: > > file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@REDACTED > calhost.2017-02-25_00.02.28/ninenines.cowboy.rfc7540_SUITE. > logs/run.2017-02-25_00.02.29/suite.log.html > > Which is great, I can always go back and compare, and sometimes that's > what I want (when I test across Erlang releases, for example). But when I'm > fixing a bug, I tend to run a small set of tests, maybe a single suite, or > sometimes even just a single group or a single test case. When I do that, > things becomes more difficult than they should be because I can't simply > refresh the page to get the new report, I have to go back, refresh on the > all_runs.html page, and follow the links from there. > > Would it be possible for Common Test to maintain a separate directory > always containing the latest report? For example: > > file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@REDACTED > calhost.latest/ninenines.cowboy.rfc7540_SUITE.logs/run.lates > t/suite.log.html > > With a URI like this I can repeat the test run, refresh in my browser and > get the new report instantly. (And keep the history separate should I need > to compare.) > > Alternatively, have an option to disable the timestamps so I can just > refresh when I need it, although the history wouldn't be preserved in this > case. > > Cheers, > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Raghav -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell@REDACTED Tue Mar 7 11:11:42 2017 From: russell@REDACTED (Russell Brown) Date: Tue, 7 Mar 2017 10:11:42 +0000 Subject: [erlang-questions] internal error in core_fold_module Message-ID: <126F8890-162A-4DE0-9AF5-FE0219839834@wombat.me> Hi, I?ve never seen this error before: Function: maybe_new_key_epoch/4 Compiling /Users/russell/dev/e/bet365/riak/deps/riak_kv/src/riak_kv_vnode.erl failed: /Users/russell/dev/e/bet365/riak/deps/riak_kv/src/riak_kv_vnode.erl:none: internal error in core_fold_module; crash reason: {function_clause, [{sys_core_fold,will_match_list, [[], [{c_var, [2621,{file,"src/riak_kv_vnode.erl"}], 'LocalEpoch'}, {c_var, [2621,{file,"src/riak_kv_vnode.erl"}], 'LocalCntr'}], yes], [{file,"sys_core_fold.erl"},{line,1656}]}, {sys_core_fold,will_match_list,3, [{file,"sys_core_fold.erl"},{line,1657}]}, {sys_core_fold,clauses_1,4, [{file,"sys_core_fold.erl"},{line,1598}]}, {sys_core_fold,clauses_1,4, [{file,"sys_core_fold.erl"},{line,1617}]}, {sys_core_fold,clauses,5, ERROR: compile failed while processing /Users/russell/dev/e/bet365/riak/deps/riak_kv: rebar_abort [{file,"sys_core_fold.erl"},{line,1575}]}, {sys_core_fold,expr,3, [{file,"sys_core_fold.erl"},{line,309}]}, {sys_core_fold,opt_simple_let_1,4, [{file,"sys_core_fold.erl"},{line,2380}]}, {sys_core_fold,clause,4, [{file,"sys_core_fold.erl"},{line,1413}]}]} make: *** [compile] Error 1 Googling comes up with http://erlang.org/pipermail/erlang-bugs/2014-March/004241.html I?m on r16b02. Is the above output a result of the above bug report? Case statement in question is {true, {undefined, 0, 0}} -> %% Not present locally %% Never acted on this object before and coordindating: %% new epoch. new_key_epoch(State); {_Coord, {LocalId, LocalEpoch, LocalCntr}} -> %% either not coordinating or coordinating and locally present Many thanks in advance. I guess I could just write a less horrid case statement, but I?m curious now. Cheers Russell From S.J.Thompson@REDACTED Tue Mar 7 20:10:40 2017 From: S.J.Thompson@REDACTED (Simon Thompson) Date: Tue, 7 Mar 2017 19:10:40 +0000 Subject: [erlang-questions] Professor of Cyber Security at Kent References: <58BE94BD.2010509@kent.ac.uk> Message-ID: We?re looking to appoint a professor of Cyber Security at Kent, to head up our existing group, with its Masters course and research activities. The Cyber Security group also sits alongside our Programming Languages and Systems group, and someone with strengths in PL aspects of security would fit very well. The advert for a Professor of Cyber Security is live at https://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=40739,2141233446&key=49639912&c=824734655621&pagestamp=sesmvkldtcboaxhswn and post has been advertised with an associated lectureship in security, so the successful candidate will have a chance to shape that appointment too. If you?re interested, do think about applying, or get in touch with us. I?d be happy to answer emails, as would Richard Jones, head of school: cs-hos AT kent DOT ac DOT uk Simon Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Wed Mar 8 00:45:01 2017 From: rvirding@REDACTED (Robert Virding) Date: Wed, 8 Mar 2017 00:45:01 +0100 Subject: [erlang-questions] Cleaning up the dialyzer interface In-Reply-To: References: Message-ID: Yes, I quite agree, as do most users, but you have to start somewhere. Sort of start at the beginning and work through to the end. Robert On 7 March 2017 at 08:35, Zachary Kessin wrote: > It would also be nice if it gave easier to understand error messages, or > at least broke them across several lines to make them easier to read. > > Zach > ? > > On Tue, Mar 7, 2017 at 2:17 AM, Robert Virding wrote: > >> Is anyone working on cleaning up the dialyzer interface? It does some >> strange things like for no discernable reason requiring all files to be >> either .erl files or .beam files and not allowing a combination. Also it it >> is not in any way extensible. >> >> Robert >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Zach Kessin > Skype: zachkessin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Mar 8 01:51:36 2017 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 7 Mar 2017 18:51:36 -0600 Subject: [erlang-questions] bitcoin blockchain Message-ID: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> Hi Joe, You might find this repo helpful: https://github.com/solarbit/pool/tree/master/lib In particular, downloading and parsing old blocks in this: https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl I have more code related, so if there were something particular missing, let me know and I?d be happy to share. BR, /sd -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Mar 8 02:08:34 2017 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 7 Mar 2017 19:08:34 -0600 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> References: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> Message-ID: <8DC6E0AA-24C4-4DDA-A521-EF6C26EC6AF7@gmail.com> BTW the current chain size is ~100 GB > On Mar 7, 2017, at 6:51 PM, Steve Davis wrote: > > Hi Joe, > > You might find this repo helpful: > > https://github.com/solarbit/pool/tree/master/lib > > In particular, downloading and parsing old blocks in this: > > https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl > > I have more code related, so if there were something particular missing, let me know and I?d be happy to share. > > BR, > /sd -------------- next part -------------- An HTML attachment was scrubbed... URL: From fe.belineb@REDACTED Wed Mar 8 01:49:34 2017 From: fe.belineb@REDACTED (Felipe Beline) Date: Tue, 7 Mar 2017 21:49:34 -0300 Subject: [erlang-questions] Erlang dependencies Message-ID: Hello, A dependency should provide a way to be registered in the application supervisor? Ex: I'm using a framework and almost everything that I have in my application will be supervised by the framework itself. (Framework supervision tree) But then I want to create some crazy stuff that needs to be supervised by my application. (App supervision tree) And then I starting to use a database driver that has his own supervisor as well. (Db Driver supervision tree) The point is: Is ok to have 3 different supervision trees that does not communicate by them-selfs? (As if for example the framework supervisor dies probably something really bad had happened in erlang vm, and everything should die as well) or All dependencies should provide a way to be registered in the application supervisor? (And then everything will be tied by the Application supervision tree, even if does not seems fully necessary) Cheers, Felipe -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Mar 8 06:17:25 2017 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 7 Mar 2017 21:17:25 -0800 Subject: [erlang-questions] Cleaning up the dialyzer interface In-Reply-To: References: Message-ID: <58BF93E5.3030002@gmail.com> On 03/06/2017 04:17 PM, Robert Virding wrote: > Is anyone working on cleaning up the dialyzer interface? It does some strange things like for no discernable reason requiring all files to be either .erl files or .beam files and not allowing a combination. Also it it is not in any way extensible. It might be better to consider removing the dialyzer interface and just focus on making the command-line use of dialyzer as efficient as possible, so it may always be part of per-commit regression testing. The dialyzer UI interface is attempting to promote manual use of dialyzer, which isn't a common use-case. Also, often UI creates development burden that leads to source code becoming more inefficient (like comparing Thunderbird and mutt for email). While it is easy to expect a perfect outcome with a UI implementation, I don't believe it is realistic or helpful for normal (required/critical) use of dialyzer. Best Regards, Michael From erlang@REDACTED Wed Mar 8 08:13:57 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 8 Mar 2017 08:13:57 +0100 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> References: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> Message-ID: Thanks Steve, The code looks great - nicely written :-) Next step - download the first 20 blocks and *completely* understand them. BTW - I have a few really stupid questions: I guess these have been answered before and elsewhere, so I'm just like hints as to where to find the answers. - How did Satoshi Nakamoto (SN) publish/spread the first bitcoins? - Does a bitcoin user have to download the entire bitcoin chain to do anything? - How much data does an owner of a bitcoin need to store locally (just their Private key and the hash of the bitcoin???) - How did SN spend the first bitcoins (did he just give them away) - How much is the residual value of SN's coins - How does bitcoin scale into the future - can we make cutoff points and assume we don't need to go back to the genesis block - ie assume the genesis block is number 1,000,000 and not bother to go back further Cheers /Joe On Wed, Mar 8, 2017 at 1:51 AM, Steve Davis wrote: > Hi Joe, > > You might find this repo helpful: > > https://github.com/solarbit/pool/tree/master/lib > > In particular, downloading and parsing old blocks in this: > > https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl > > I have more code related, so if there were something particular missing, let > me know and I?d be happy to share. > > BR, > /sd From erlang@REDACTED Wed Mar 8 08:34:44 2017 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 8 Mar 2017 08:34:44 +0100 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: References: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> Message-ID: To answer my own question(s) I found this https://bitslog.wordpress.com/ which has the answer to many of these questions - the forensic analysis of the first few thousand bitcoins makes a very interesting read /Joe On Wed, Mar 8, 2017 at 8:13 AM, Joe Armstrong wrote: > Thanks Steve, > > The code looks great - nicely written :-) > > Next step - download the first 20 blocks and *completely* understand them. > > BTW - I have a few really stupid questions: > > I guess these have been answered before and elsewhere, > so I'm just like hints as to where to find the answers. > > - How did Satoshi Nakamoto (SN) publish/spread the first bitcoins? > - Does a bitcoin user have to download the entire bitcoin chain to do anything? > - How much data does an owner of a bitcoin need to store locally > (just their Private key and the hash of the bitcoin???) > - How did SN spend the first bitcoins (did he just give them away) > - How much is the residual value of SN's coins > - How does bitcoin scale into the future - can we make cutoff points > and assume we don't need to go back to the genesis block - ie > assume the genesis block is number 1,000,000 and not bother to go back > further > > Cheers > > /Joe > > > On Wed, Mar 8, 2017 at 1:51 AM, Steve Davis > wrote: >> Hi Joe, >> >> You might find this repo helpful: >> >> https://github.com/solarbit/pool/tree/master/lib >> >> In particular, downloading and parsing old blocks in this: >> >> https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl >> >> I have more code related, so if there were something particular missing, let >> me know and I?d be happy to share. >> >> BR, >> /sd From frank.muller.erl@REDACTED Wed Mar 8 10:25:31 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 08 Mar 2017 09:25:31 +0000 Subject: [erlang-questions] Microsoft SQL Server connector Message-ID: Hi guys, I'm looking for an Erlang library to connect to SQL Server on Windows. Thank you. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From ye02@REDACTED Wed Mar 8 10:55:38 2017 From: ye02@REDACTED (=?GBK?B?0rbJ2bKo?=) Date: Wed, 8 Mar 2017 17:55:38 +0800 (CST) Subject: [erlang-questions] The port "tty_sl -c -e"is out of control Message-ID: <57532ed7.aa92.15aad586dc1.Coremail.ye02@163.com> Hi erlang expets, I met a problem about the erlang port "tty_sl -c- e". The scenario is like this: 1) I start two erlang nodes (Lets say one is Node1, another one is Node2) by run_erl, they are connected together. I can login to these two nodes by to_erl and do anythings on the erlang shell. It run very well. 2) But today I tried to use to_erl to login to the Node1, I can`t type anything on the shell, it seems the shell is hanging; Then I login to the Node2 and run rpc:call(node1, shell, start, []) to start a new shell on Node1, and I check the port info of the port "tty_sl -c -e", it shows below: (node1@REDACTED)7> Port. #Port<0.4832> (node1@REDACTED)8> erlang:port_info(Port). [{name,"tty_sl -c -e"}, {links,[]}, {id,38656}, {connected,<0.1502.0>}, {input,2552}, {output,1751871}, {os_pid,undefined}] you can see here, the "links" is empty and the connected process does`t exist. (node1@REDACTED)3> erlang:process_info(pid(0, 1502, 0)). undefined 3) I tried to close the port by erlang:port_close(Port), the result is as below: (titan1_ps3@REDACTED)4> erlang:port_close(Port). ** exception error: bad argument in function port_close/1 called as port_close(#Port<0.4832>) It seems the port already died. As I know, the port will be closed if the linked process dies. But here we can see the port does`t exit. Do you know what is the reason? Or how can I kill the port? Actually I met this problem several times before, I had to reboot the node the fix. Thanks, BRs/Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Wed Mar 8 12:12:13 2017 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Wed, 8 Mar 2017 12:12:13 +0100 Subject: [erlang-questions] Microsoft SQL Server connector In-Reply-To: References: Message-ID: Hello Frank, ODBC is the way to go. I think I saw an Elixir project with a "native protocol" implementation, but I do not know the state of it. Regards, Boris 2017-03-08 10:25 GMT+01:00 Frank Muller : > Hi guys, > > I'm looking for an Erlang library to connect to SQL Server on Windows. > > Thank you. > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Mar 8 13:22:25 2017 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 8 Mar 2017 13:22:25 +0100 Subject: [erlang-questions] The port "tty_sl -c -e"is out of control In-Reply-To: <57532ed7.aa92.15aad586dc1.Coremail.ye02@163.com> References: <57532ed7.aa92.15aad586dc1.Coremail.ye02@163.com> Message-ID: Hello, The port seems to have gotten stuck while flushing it's output to stdout. When a port is flushing, it is invalid to interact with (as you see through the badarg you get from port_close), but it still shows up in erlang:ports() as it is not completely gone yet. Since you are using run_erl, it seems like for some reason that program is not reading from the dup:ed stdout fd. Does the syslog/run_erl log tell you anything about what may be going on there? Lukas On Wed, Mar 8, 2017 at 10:55 AM, ??? wrote: > Hi erlang expets, > > I met a problem about the erlang port "tty_sl -c- e". > > The scenario is like this: > 1) I start two erlang nodes (Lets say one is Node1, another one is Node2) > by run_erl, they are connected together. > I can login to these two nodes by to_erl and do anythings on the > erlang shell. It run very well. > > 2) But today I tried to use to_erl to login to the Node1, I can`t type > anything on the shell, it seems the shell is hanging; > Then I login to the Node2 and run rpc:call(node1, shell, start, []) to > start a new shell on Node1, and I check the port > info of the port "tty_sl -c -e", it shows below: > (node1@REDACTED)7> Port. > #Port<0.4832> > (node1@REDACTED)8> erlang:port_info(Port). > [{name,"tty_sl -c -e"}, > {links,[]}, > {id,38656}, > {connected,<0.1502.0>}, > {input,2552}, > {output,1751871}, > {os_pid,undefined}] > > you can see here, the "links" is empty and the connected process does`t > exist. > (node1@REDACTED)3> erlang:process_info(pid(0, 1502, 0)). > undefined > > 3) I tried to close the port by erlang:port_close(Port), the result is as > below: > (titan1_ps3@REDACTED)4> erlang:port_close(Port). > ** exception error: bad argument > in function port_close/1 > called as port_close(#Port<0.4832>) > > It seems the port already died. > As I know, the port will be closed if the linked process dies. But here we > can see the port does`t exit. > Do you know what is the reason? Or how can I kill the port? > > Actually I met this problem several times before, I had to reboot the node > the fix. > > Thanks, > BRs/Michael > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Mar 8 15:01:46 2017 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 8 Mar 2017 08:01:46 -0600 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: References: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> Message-ID: <295304ED-0503-4B62-BBBE-A3E56E7B7C8D@gmail.com> I found this article particularly helpful early on: http://www.righto.com/2014/02/bitcoins-hard-way-using-raw-bitcoin.html > On Mar 8, 2017, at 1:34 AM, Joe Armstrong wrote: > > To answer my own question(s) > > I found this https://bitslog.wordpress.com/ > which has the answer to many of these questions - the forensic > analysis of the first few thousand bitcoins makes a very interesting > read > > /Joe > > On Wed, Mar 8, 2017 at 8:13 AM, Joe Armstrong wrote: >> Thanks Steve, >> >> The code looks great - nicely written :-) >> >> Next step - download the first 20 blocks and *completely* understand them. >> >> BTW - I have a few really stupid questions: >> >> I guess these have been answered before and elsewhere, >> so I'm just like hints as to where to find the answers. >> >> - How did Satoshi Nakamoto (SN) publish/spread the first bitcoins? >> - Does a bitcoin user have to download the entire bitcoin chain to do anything? >> - How much data does an owner of a bitcoin need to store locally >> (just their Private key and the hash of the bitcoin???) >> - How did SN spend the first bitcoins (did he just give them away) >> - How much is the residual value of SN's coins >> - How does bitcoin scale into the future - can we make cutoff points >> and assume we don't need to go back to the genesis block - ie >> assume the genesis block is number 1,000,000 and not bother to go back >> further >> >> Cheers >> >> /Joe >> >> >> On Wed, Mar 8, 2017 at 1:51 AM, Steve Davis >> wrote: >>> Hi Joe, >>> >>> You might find this repo helpful: >>> >>> https://github.com/solarbit/pool/tree/master/lib >>> >>> In particular, downloading and parsing old blocks in this: >>> >>> https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl >>> >>> I have more code related, so if there were something particular missing, let >>> me know and I?d be happy to share. >>> >>> BR, >>> /sd -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Wed Mar 8 15:17:41 2017 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 8 Mar 2017 08:17:41 -0600 Subject: [erlang-questions] bitcoin blockchain In-Reply-To: <295304ED-0503-4B62-BBBE-A3E56E7B7C8D@gmail.com> References: <6E4FAFA0-211B-4356-A825-99CCC3E1EA36@gmail.com> <295304ED-0503-4B62-BBBE-A3E56E7B7C8D@gmail.com> Message-ID: <21B5B025-3F88-4F1A-B879-48F96E9ED88B@gmail.com> One more: https://en.bitcoin.it/wiki/Main_Page > On Mar 8, 2017, at 8:01 AM, Steve Davis wrote: > > I found this article particularly helpful early on: > > http://www.righto.com/2014/02/bitcoins-hard-way-using-raw-bitcoin.html > > >> On Mar 8, 2017, at 1:34 AM, Joe Armstrong > wrote: >> >> To answer my own question(s) >> >> I found this https://bitslog.wordpress.com/ >> which has the answer to many of these questions - the forensic >> analysis of the first few thousand bitcoins makes a very interesting >> read >> >> /Joe >> >> On Wed, Mar 8, 2017 at 8:13 AM, Joe Armstrong > wrote: >>> Thanks Steve, >>> >>> The code looks great - nicely written :-) >>> >>> Next step - download the first 20 blocks and *completely* understand them. >>> >>> BTW - I have a few really stupid questions: >>> >>> I guess these have been answered before and elsewhere, >>> so I'm just like hints as to where to find the answers. >>> >>> - How did Satoshi Nakamoto (SN) publish/spread the first bitcoins? >>> - Does a bitcoin user have to download the entire bitcoin chain to do anything? >>> - How much data does an owner of a bitcoin need to store locally >>> (just their Private key and the hash of the bitcoin???) >>> - How did SN spend the first bitcoins (did he just give them away) >>> - How much is the residual value of SN's coins >>> - How does bitcoin scale into the future - can we make cutoff points >>> and assume we don't need to go back to the genesis block - ie >>> assume the genesis block is number 1,000,000 and not bother to go back >>> further >>> >>> Cheers >>> >>> /Joe >>> >>> >>> On Wed, Mar 8, 2017 at 1:51 AM, Steve Davis >>> > wrote: >>>> Hi Joe, >>>> >>>> You might find this repo helpful: >>>> >>>> https://github.com/solarbit/pool/tree/master/lib >>>> >>>> In particular, downloading and parsing old blocks in this: >>>> >>>> https://github.com/solarbit/pool/blob/master/lib/solarbit/src/sbt_btc_srv.erl >>>> >>>> I have more code related, so if there were something particular missing, let >>>> me know and I?d be happy to share. >>>> >>>> BR, >>>> /sd > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Mar 9 00:32:22 2017 From: rvirding@REDACTED (Robert Virding) Date: Thu, 9 Mar 2017 00:32:22 +0100 Subject: [erlang-questions] Cleaning up the dialyzer interface In-Reply-To: <58BF93E5.3030002@gmail.com> References: <58BF93E5.3030002@gmail.com> Message-ID: I completely agree with you but was using the term "interface" in a more general way than you do to include the command line as well. The things I mentioned are irrespective of whether you start the dialyzer through function calls or command-line. Robert On 8 March 2017 at 06:17, Michael Truog wrote: > On 03/06/2017 04:17 PM, Robert Virding wrote: > >> Is anyone working on cleaning up the dialyzer interface? It does some >> strange things like for no discernable reason requiring all files to be >> either .erl files or .beam files and not allowing a combination. Also it it >> is not in any way extensible. >> > > It might be better to consider removing the dialyzer interface and just > focus on making the command-line use of dialyzer as efficient as possible, > so it may always be part of per-commit regression testing. The dialyzer UI > interface is attempting to promote manual use of dialyzer, which isn't a > common use-case. Also, often UI creates development burden that leads to > source code becoming more inefficient (like comparing Thunderbird and mutt > for email). While it is easy to expect a perfect outcome with a UI > implementation, I don't believe it is realistic or helpful for normal > (required/critical) use of dialyzer. > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Mar 9 00:47:22 2017 From: rvirding@REDACTED (Robert Virding) Date: Thu, 9 Mar 2017 00:47:22 +0100 Subject: [erlang-questions] Cleaning up the dialyzer interface In-Reply-To: References: <58BF93E5.3030002@gmail.com> Message-ID: I forgot to add this example: Why can't I write: $ dialyzer foo.erl bar.beam there is no logical or practical reason. On 9 March 2017 at 00:32, Robert Virding wrote: > I completely agree with you but was using the term "interface" in a more > general way than you do to include the command line as well. The things I > mentioned are irrespective of whether you start the dialyzer through > function calls or command-line. > > Robert > > > On 8 March 2017 at 06:17, Michael Truog wrote: > >> On 03/06/2017 04:17 PM, Robert Virding wrote: >> >>> Is anyone working on cleaning up the dialyzer interface? It does some >>> strange things like for no discernable reason requiring all files to be >>> either .erl files or .beam files and not allowing a combination. Also it it >>> is not in any way extensible. >>> >> >> It might be better to consider removing the dialyzer interface and just >> focus on making the command-line use of dialyzer as efficient as possible, >> so it may always be part of per-commit regression testing. The dialyzer UI >> interface is attempting to promote manual use of dialyzer, which isn't a >> common use-case. Also, often UI creates development burden that leads to >> source code becoming more inefficient (like comparing Thunderbird and mutt >> for email). While it is easy to expect a perfect outcome with a UI >> implementation, I don't believe it is realistic or helpful for normal >> (required/critical) use of dialyzer. >> >> Best Regards, >> Michael >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxmywc@REDACTED Thu Mar 9 04:00:39 2017 From: fxmywc@REDACTED (fxmy wang) Date: Thu, 9 Mar 2017 11:00:39 +0800 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled Message-ID: Hi guys, So recently I?m toying with SNMP Development Toolkit and having trouble compiling standard LLDP-MIB. Errors shown below 2> snmpc:compile("LLDP-MIB.mib"). LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry lldpLocManAddrEntry [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpConfigManAddrTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemUnknownTLVTable => The default functions won't work properly [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemOrgDefInfoTable => The default functions won't work properly {error,compilation_failed} 3> The LLDP-MIB file is hosted on ieee802.org . How should I deal with this? Cheers, ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Mar 9 10:01:07 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 9 Mar 2017 10:01:07 +0100 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: References: Message-ID: <20170309090107.GA5874@erix.ericsson.se> There was a fix committed to the maint branch at Jan 12. I do not know if it is this problem it fixes; it allows refinements of enum usertypes. See https://bugs.erlang.org/browse/ERL-325 It will be released in OTP-19.3 shortly. On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: > Hi guys, > So recently I?m toying with SNMP Development Toolkit and having trouble > compiling standard LLDP-MIB. Errors shown below > > 2> snmpc:compile("LLDP-MIB.mib"). > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry > lldpLocManAddrEntry > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable > => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpConfigManAddrTable => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable > => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable > => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => > The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable > => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The > default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable > => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpRemUnknownTLVTable => The default functions won't work properly > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpRemOrgDefInfoTable => The default functions won't work properly > {error,compilation_failed} > 3> > > The LLDP-MIB file is hosted on ieee802.org > . > > How should I deal with this? > > Cheers, > ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From fxmywc@REDACTED Thu Mar 9 14:28:37 2017 From: fxmywc@REDACTED (fxmy wang) Date: Thu, 9 Mar 2017 21:28:37 +0800 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled Message-ID: Good to know that Raimo. Thanks, I'm gonna wait for 19.3 and see what happens. Cheers, 2017-03-09 17:01 GMT+08:00 Raimo Niskanen < raimo+erlang-questions@REDACTED>: > There was a fix committed to the maint branch at Jan 12. I do not know if > it is this problem it fixes; it allows refinements of enum usertypes. > > See https://bugs.erlang.org/browse/ERL-325 > > It will be released in OTP-19.3 shortly. > > > > On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: > > Hi guys, > > So recently I?m toying with SNMP Development Toolkit and having trouble > > compiling standard LLDP-MIB. Errors shown below > > > > 2> snmpc:compile("LLDP-MIB.mib"). > > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry > > lldpLocManAddrEntry > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable > > => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > lldpConfigManAddrTable => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable > > => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable > > => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => > > The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable > > => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The > > default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable > > => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > lldpRemUnknownTLVTable => The default functions won't work properly > > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > lldpRemOrgDefInfoTable => The default functions won't work properly > > {error,compilation_failed} > > 3> > > > > The LLDP-MIB file is hosted on ieee802.org > > . > > > > How should I deal with this? > > > > Cheers, > > ? > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Thu Mar 9 19:54:36 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 9 Mar 2017 19:54:36 +0100 (CET) Subject: [erlang-questions] DTLS: doubts about fragment reassembler Message-ID: <117779617.346042.1489085676727.JavaMail.zimbra@tpip.net> Hi Ingela, I tried to understand the fragment reassembler in dtls_handshake.erl and it seems that it is assuming that retransmissions will always use the same record size. RFC 6347, Sect. 4.3.2 states: > When a DTLS implementation receives a handshake message fragment, it > MUST buffer it until it has the entire handshake message. DTLS > implementations MUST be able to handle overlapping fragment ranges. > This allows senders to retransmit handshake messages with smaller > fragment sizes if the PMTU estimate changes. Last time I looked at OpenSSL's DTLS sending side, it did exactly that. It refragmented the handshake on retransmit to the minimum IP MTU. >From my understanding of the fragment merging logic, it will not merge overlapping fragments (it might even become confused). Am I missing something there? Regards Andreas From aschultz@REDACTED Thu Mar 9 22:39:38 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 9 Mar 2017 22:39:38 +0100 (CET) Subject: [erlang-questions] DTLS: doubts about fragment reassembler In-Reply-To: References: <117779617.346042.1489085676727.JavaMail.zimbra@tpip.net> Message-ID: <1521005351.348456.1489095578070.JavaMail.zimbra@tpip.net> ----- On Mar 9, 2017, at 10:00 PM, Ingela Andin wrote: > Hi Andreas! > On Thu, Mar 9, 2017 at 7:54 PM, Andreas Schultz < [ mailto:aschultz@REDACTED | > aschultz@REDACTED ] > wrote: >> Hi Ingela, >> I tried to understand the fragment reassembler in dtls_handshake.erl >> and it seems that it is assuming that retransmissions will always use >> the same record size. > No it is not assuming that. It just does not care what the size is, it suppose > to just fill in the gaps (in the merge) until it has reasemmbled a whole. > Of course I have not written any specific test for it yet so I am not saying it > may not have bugs. I'm staring at dtls_handshake:merge_fragments/2 and I can't convince myself that it will merge overlapping fragments. Let's assume we have <<0,1,2>> and <4,5,6>>, the new fragment is <<2,3,4>>. This would mean the following fragment records: 1: #handshake_fragment{fragment_offset = 0, fragment_length = 3, fragment = <<0,1,2>>} 2: #handshake_fragment{fragment_offset = 2 , fragment_length = 3, fragment = <<2,3,4>>} 3: #handshake_fragment{fragment_offset = 4 , fragment_length = 3, fragment = <<4,5,6>>} IMHO merge_fragments will not merge fragment #1 and #2 (none of the conditions match), the same goes for fragment #2 and #3. The expected output would be a fragment: #handshake_fragment{fragment_offset = 0, fragment_length = 6, fragment = <<0,1,2,3,4,5,6>>} I think the following (untested) change is needed: diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index fd1f969..245dc6e 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -464,10 +464,28 @@ merge_fragments(#handshake_fragment{ #handshake_fragment{ fragment_offset = CurrentOffSet, fragment_length = CurrentLen, - fragment = CurrentData}) when PreviousOffSet + PreviousLen == CurrentOffSet-> - Previous#handshake_fragment{ - fragment_length = PreviousLen + CurrentLen, - fragment = <>}; + fragment = CurrentData}) + when PreviousOffSet + PreviousLen >= CurrentOffSet andalso + PreviousOffSet + PreviousLen < CurrentOffSet + CurrentLength -> + CurrentStart = PreviousOffSet + PreviousLen - CurrentOffSet, + <<_:CurrentStart/bytes, Data/binary>> = CurrentData, + Previous#handshake_fragment{ + fragment_length = PreviousLen + CurrentLen - CurrentStart, + fragment = <>}; +%% already fully contained fragment +merge_fragments(#handshake_fragment{ + fragment_offset = PreviousOffSet, + fragment_length = PreviousLen, + fragment = PreviousData + } = Previous, + #handshake_fragment{ + fragment_offset = CurrentOffSet, + fragment_length = CurrentLen, + fragment = CurrentData}) + when PreviousOffSet + PreviousLen >= CurrentOffSet andalso + PreviousOffSet + PreviousLen >= CurrentOffSet + CurrentLength -> + Previous; + Andreas > RFC 6347, Sect. 4.3.2 states: > > When a DTLS implementation receives a handshake message fragment, it > > MUST buffer it until it has the entire handshake message. DTLS > > implementations MUST be able to handle overlapping fragment ranges. > > This allows senders to retransmit handshake messages with smaller > > fragment sizes if the PMTU estimate changes. > Last time I looked at OpenSSL's DTLS sending side, it did exactly that. > It refragmented the handshake on retransmit to the minimum IP MTU. > From my understanding of the fragment merging logic, it will not merge > overlapping fragments (it might even become confused). > Am I missing something there? It is suppose to merge overlapping fragments. I am currently trying to get all the basic tests going. I acctually merged some new code this week with a few fixes and a lot more tests activated, however I doubt that the nighlty tests drops a lot of packages.... There are still a lot to do but now I think we are getting there. And I had a lot of use of your code even the parts I did not use. I hope to get some quick check tests going too in the long run. Regards Ingela -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Fri Mar 10 11:22:22 2017 From: thoffmann@REDACTED (Torben Hoffmann) Date: Fri, 10 Mar 2017 11:22:22 +0100 Subject: [erlang-questions] [ANN] chronos 0.5.1 released Message-ID: <58c27e66.cdd8190a.421eb.a034@mx.google.com> Hi, I have updated my chronos timer library to 0.5.1 after receiving some nice PRs from Raghav Karol. https://github.com/lehoff/chronos chronos now works with rebar3 and no longer depends on gproc thanks to Raghav. In addition to that I have changed the tests to use EQC mocking and removed tests, which were more focused on testing if Erlang timers behave correctly than test chronos. chronos is registered on http://www.quickcheck-ci.com/ to validate that it passes its tests. Real life usage have shown that chronos has an adequate feature set and once 0.5.1 has been in the wild for a while, I will release 1.0. The README now contains a bit more information on what chronos does and does not do. Plus a description of how to use EQC mocking with chronos. Please raise issues if you find the code or the documentation to be broken/incomprehensible. Cheers, Torben -- Torben Hoffmann Chief Architect, basho.com M: +45 25 14 05 38 From erlangsiri@REDACTED Fri Mar 10 15:41:20 2017 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 10 Mar 2017 15:41:20 +0100 Subject: [erlang-questions] Common Test logs In-Reply-To: <3ab597eb-be1e-f746-391a-5a6653e4ab89@erlang.org> References: <5b05e1d9-6cf5-8ad5-5321-22f06bdabe5d@ninenines.eu> <3ab597eb-be1e-f746-391a-5a6653e4ab89@erlang.org> Message-ID: Hi, I've started looking at this and I wonder how the limit should best be specified. Should it be * keep N log directories * keep N log directories per 'test name' * keep all log directories with timestamps newer than a given datetime * keep all log directories with timestamps newer than a number of days, hours, minutes, seconds back in time? Any thoughts or wishes? Regards /siri 2017-02-02 7:52 GMT+01:00 Peter Andersson E : > > Hi Lo?c, > > This seems lika quite a useful feature, and a reasonable one to implement. > I'll add a ticket for it! > > Best regards, > Peter > > Ericsson AB, Erlang/OTP > > > On 01/30/2017 06:30 PM, Lo?c Hoguin wrote: > >> Hello, >> >> I am looking for a Common Test option that allows me to restrict the >> number of logs Common Test keeps around. >> >> In practice I don't always care about the test runs I did last month, I >> only want the 10 or so most recents. I would like to configure Common Test >> so that it deletes older runs automatically. Today I eventually "rm -rf >> logs" when it gets too big, but it would be good if I didn't have to think >> about it. >> >> Is there such an option (perhaps undocumented) and if not, can it be >> considered adding it? This way when ct_run starts it can cleanup >> automatically. >> >> Thanks, >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 10 15:55:36 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 10 Mar 2017 15:55:36 +0100 Subject: [erlang-questions] Common Test logs In-Reply-To: References: <5b05e1d9-6cf5-8ad5-5321-22f06bdabe5d@ninenines.eu> <3ab597eb-be1e-f746-391a-5a6653e4ab89@erlang.org> Message-ID: <6c0e8d67-dda2-d316-97fd-0dc5b0e4da26@ninenines.eu> Thanks for looking into this! Personally, keep N log directories would be enough. It would be useful on my development machine where I run tests a lot and rarely need to look back at much more than the past 2 or 3 runs. But for my purposes, the 4th option also works (for example if I could say '-keep_logs_only_from_the_past 2days' or something). For CI I simply keep everything, but organize it differently ( logs/$PROJECT/$RUNID/$OS/ and then Common Test has one test run per supported Erlang version, eg https://builds.ninenines.eu/logs/cowboy/125/archlinux/ ) so I would not use it there. Cheers, On 03/10/2017 03:41 PM, Siri Hansen wrote: > Hi, I've started looking at this and I wonder how the limit should best > be specified. Should it be > > * keep N log directories > * keep N log directories per 'test name' > * keep all log directories with timestamps newer than a given datetime > * keep all log directories with timestamps newer than a number of days, > hours, minutes, seconds back in time? > > Any thoughts or wishes? > > Regards > /siri > > 2017-02-02 7:52 GMT+01:00 Peter Andersson E >: > > > Hi Lo?c, > > This seems lika quite a useful feature, and a reasonable one to > implement. I'll add a ticket for it! > > Best regards, > Peter > > Ericsson AB, Erlang/OTP > > > On 01/30/2017 06:30 PM, Lo?c Hoguin wrote: > > Hello, > > I am looking for a Common Test option that allows me to restrict > the number of logs Common Test keeps around. > > In practice I don't always care about the test runs I did last > month, I only want the 10 or so most recents. I would like to > configure Common Test so that it deletes older runs > automatically. Today I eventually "rm -rf logs" when it gets too > big, but it would be good if I didn't have to think about it. > > Is there such an option (perhaps undocumented) and if not, can > it be considered adding it? This way when ct_run starts it can > cleanup automatically. > > Thanks, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From russell@REDACTED Fri Mar 10 18:31:29 2017 From: russell@REDACTED (Russell Brown) Date: Fri, 10 Mar 2017 17:31:29 +0000 Subject: [erlang-questions] No function form in result of epp:parse_file for declared function Message-ID: <4D5C2FD9-EEFA-4B52-A1DB-2E253DBA624F@wombat.me> Hi, I?m new to epp and associated modules, and messing with AST and all that, so please go gently with me. I?ve just run epp:parse_file on this file https://github.com/basho/riak_kv/blob/develop/src/riak_kv_vnode.erl and the resulting forms don?t contain a {function?} tuple for the function `handle_command/3` This seems odd, since it is declared here https://github.com/basho/riak_kv/blob/develop/src/riak_kv_vnode.erl#L560 There are {call?} tuples in the Forms that show local calls to the function, and the attributes Form shows it as an export, but there is no {function tuple in the Forms. I?m baffled. Can you help? Many thanks Russell From fxmywc@REDACTED Sat Mar 11 10:23:07 2017 From: fxmywc@REDACTED (fxmy wang) Date: Sat, 11 Mar 2017 17:23:07 +0800 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: References: Message-ID: Hi guys, I?ve checked out 7f52af8 and tried again but the compilation still fails. =( I?v updated the mib-compiling log to gist in case anyone interested. Hope this log holds some useful information for the SNMP experts. Cheers, ? 2017-03-11 14:01 GMT+08:00 fxmy wang : > Hi guys, > I've checked out [7f52af8](https://github.com/erlang/otp/tree/ > 7f52af8875a1ab1ba96c6cb2403211bfc876639c) and tried again but the > compilation still fails. =( > Below attaches the log file when compiling LLDP-MIB (also uploaded to > [gits](https://gist.github.com/fxmy/0c84243d387f972a725c68004cc02933)). > Hope this log holds some useful information for the SNMP experts. > Cheers, > > 2017-03-09 21:28 GMT+08:00 fxmy wang : > >> Good to know that Raimo. >> Thanks, I'm gonna wait for 19.3 and see what happens. >> Cheers, >> >> 2017-03-09 17:01 GMT+08:00 Raimo Niskanen > ricsson.se>: >> >>> There was a fix committed to the maint branch at Jan 12. I do not know >>> if >>> it is this problem it fixes; it allows refinements of enum usertypes. >>> >>> See https://bugs.erlang.org/browse/ERL-325 >>> >>> It will be released in OTP-19.3 shortly. >>> >>> >>> >>> On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: >>> > Hi guys, >>> > So recently I?m toying with SNMP Development Toolkit and having trouble >>> > compiling standard LLDP-MIB. Errors shown below >>> > >>> > 2> snmpc:compile("LLDP-MIB.mib"). >>> > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry >>> > lldpLocManAddrEntry >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable >>> > => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table >>> > lldpConfigManAddrTable => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable >>> > => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable >>> > => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => >>> > The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable >>> > => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The >>> > default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable >>> > => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table >>> > lldpRemUnknownTLVTable => The default functions won't work properly >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table >>> > lldpRemOrgDefInfoTable => The default functions won't work properly >>> > {error,compilation_failed} >>> > 3> >>> > >>> > The LLDP-MIB file is hosted on ieee802.org >>> > >> >. >>> > >>> > How should I deal with this? >>> > >>> > Cheers, >>> > ? >>> >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> -- >>> >>> / Raimo Niskanen, Erlang/OTP, Ericsson AB >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Sat Mar 11 14:13:14 2017 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Sat, 11 Mar 2017 13:13:14 +0000 Subject: [erlang-questions] SSH ServerAliveInterval Message-ID: Hi, Is there a way to set ServerAliveInterval for the Erlang SSH client? If not, what is its default value? Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.rascao@REDACTED Sat Mar 11 14:47:41 2017 From: luis.rascao@REDACTED (=?UTF-8?B?THVpcyBSYXNjw6Nv?=) Date: Sat, 11 Mar 2017 13:47:41 +0000 Subject: [erlang-questions] Hot code upgrade In-Reply-To: <588EFBB7.8060903@utl.in> References: <588EFBB7.8060903@utl.in> Message-ID: I know this reply is lagging a bit but still you may find it useful. If so inclined, you can try out this rebar3 plugin [1] i wrote which attempts to ease the relup burden by automating some of the steps away, any feedback/bug report is welcome! best Luis Rasc?o [1] https://github.com/lrascao/rebar3_appup_plugin On Mon, Jan 30, 2017 at 8:39 AM, Arun wrote: > Hi, > > I am trying to upgrade my application using rebar3 hot code upgrade > feature from version 0.0.1 to 0.0.2 and my application consists of around > 20 modules, the difference between current version and the new version is > that one new module got added and one existing module code has been > modified. My doubt is that, in the appup file we have to give list of > instructions to upgrade and downgrade software between different versions, > is it mandatory that all 20 modules has to be mentioned in the instructions > or the modules which are added or modified only need to be mentioned. ? > > Can anybody assist me on this > > regards, > Arun > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sun Mar 12 16:21:11 2017 From: g@REDACTED (Guilherme Andrade) Date: Sun, 12 Mar 2017 15:21:11 +0000 Subject: [erlang-questions] Stateful match specifications Message-ID: Hi list, I've recently wondered whether it would be useful (and feasible) to have support for 'stateful match specifications'; i.e., something along the lines of: ets:fun2ms(fun (Head, StateAcc) when Guards -> {Body, NewStateAcc} end, InitialState) ... as well as a corresponding extension to the raw match specifications format. These could then be optionally and transparently used in ets, tracing, etc. What do you think? Perhaps the 'eeps' mailing list is a more appropriate place for making this proposal? -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From quantumpotato@REDACTED Sun Mar 12 17:57:03 2017 From: quantumpotato@REDACTED (qp) Date: Sun, 12 Mar 2017 12:57:03 -0400 Subject: [erlang-questions] Reading serial data from a USB Gamepad Message-ID: Hi, I want to read USB data from a Gamepad in Erlang to play a game. Programs like JoyToKey and Enjoyable convert Gamepad inputs to keyboard input, but they only send 1 character so using io:read would require an additional "return" press to send the input. Where do I start looking to learn how to read gamepads? I've heard that Erlang ports can be used to interact with the world outside the Erlang shell but I have no idea where to look. I found erlang-serial on Github but had trouble installing it https://github.com/tonyg/erlang-serial/issues/14 Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Sun Mar 12 18:23:19 2017 From: jozsef.berces@REDACTED (=?utf-8?B?SsOzenNlZiBCw6lyY2Vz?=) Date: Sun, 12 Mar 2017 17:23:19 +0000 Subject: [erlang-questions] Reading serial data from a USB Gamepad In-Reply-To: References: Message-ID: You can give a try to this one: http://github.com/tomszilagyi/gen_serial From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of qp Sent: Sunday, March 12, 2017 17:57 To: Erlang/OTP discussions Subject: [erlang-questions] Reading serial data from a USB Gamepad Hi, I want to read USB data from a Gamepad in Erlang to play a game. Programs like JoyToKey and Enjoyable convert Gamepad inputs to keyboard input, but they only send 1 character so using io:read would require an additional "return" press to send the input. Where do I start looking to learn how to read gamepads? I've heard that Erlang ports can be used to interact with the world outside the Erlang shell but I have no idea where to look. I found erlang-serial on Github but had trouble installing it https://github.com/tonyg/erlang-serial/issues/14 Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From quantumpotato@REDACTED Sun Mar 12 19:00:26 2017 From: quantumpotato@REDACTED (qp) Date: Sun, 12 Mar 2017 14:00:26 -0400 Subject: [erlang-questions] Reading serial data from a USB Gamepad In-Reply-To: References: Message-ID: I get make errors with that one /usr/local/bin/erlang/lib/gen_serial:../make.sh Recompile: src/gen_serial_test Recompile: src/gen_serial gcc -Wall -O2 -g -ggdb -c -o ../erlang_serial.o ../erlang_serial.c gcc -Wall -O2 -g -ggdb -c -o posix_main.o posix_main.c posix_main.c:84:30: error: use of undeclared identifier 'B460800' case 460800: tio.c_cflag |= B460800; break; ^ posix_main.c:85:30: error: use of undeclared identifier 'B500000' case 500000: tio.c_cflag |= B500000; break; ^ posix_main.c:86:30: error: use of undeclared identifier 'B576000' case 576000: tio.c_cflag |= B576000; break; ^ posix_main.c:87:30: error: use of undeclared identifier 'B921600' case 921600: tio.c_cflag |= B921600; break; ^ posix_main.c:88:31: error: use of undeclared identifier 'B1000000' case 1000000: tio.c_cflag |= B1000000; break; ^ posix_main.c:89:31: error: use of undeclared identifier 'B1152000' case 1152000: tio.c_cflag |= B1152000; break; ^ posix_main.c:90:31: error: use of undeclared identifier 'B1500000' case 1500000: tio.c_cflag |= B1500000; break; ^ posix_main.c:91:31: error: use of undeclared identifier 'B2000000' case 2000000: tio.c_cflag |= B2000000; break; ^ posix_main.c:92:31: error: use of undeclared identifier 'B2500000' case 2500000: tio.c_cflag |= B2500000; break; ^ posix_main.c:93:31: error: use of undeclared identifier 'B3000000' case 3000000: tio.c_cflag |= B3000000; break; ^ posix_main.c:94:31: error: use of undeclared identifier 'B3500000' case 3500000: tio.c_cflag |= B3500000; break; ^ posix_main.c:95:31: error: use of undeclared identifier 'B4000000' case 4000000: tio.c_cflag |= B4000000; break; ^ 12 errors generated. make: *** [posix_main.o] Error 1 /usr/local/bin/erlang/lib/gen_serial:. --- On Sun, Mar 12, 2017 at 1:23 PM, J?zsef B?rces wrote: > You can give a try to this one: > > http://github.com/tomszilagyi/gen_serial > > > > *From:* erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] *On Behalf Of *qp > *Sent:* Sunday, March 12, 2017 17:57 > *To:* Erlang/OTP discussions > *Subject:* [erlang-questions] Reading serial data from a USB Gamepad > > > > Hi, > > I want to read USB data from a Gamepad in Erlang to play a game. > > Programs like JoyToKey and Enjoyable convert Gamepad inputs to keyboard > input, but they only send 1 character so using io:read would require an > additional "return" press to send the input. > > Where do I start looking to learn how to read gamepads? I've heard that > Erlang ports can be used to interact with the world outside the Erlang > shell but I have no idea where to look. I found erlang-serial on Github but > had trouble installing it https://github.com/tonyg/erlang-serial/issues/14 > > Thank you! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greim@REDACTED Sun Mar 12 19:15:29 2017 From: greim@REDACTED (greim) Date: Sun, 12 Mar 2017 19:15:29 +0100 Subject: [erlang-questions] Reading serial data from a USB Gamepad In-Reply-To: References: Message-ID: Hi, for me Erlang-Serial from Tony Garnock-Jones worked well: https://github.com/tonyg/erlang-serial tested with some serial hardware, like a embedded lcd terminal and minicom on Linux amd64 and Raspberry Pi Markus Am 12.03.2017 um 19:00 schrieb qp: > I get make errors with that one > > /usr/local/bin/erlang/lib/gen_serial:../make.sh > Recompile: src/gen_serial_test > Recompile: src/gen_serial > gcc -Wall -O2 -g -ggdb -c -o ../erlang_serial.o ../erlang_serial.c > gcc -Wall -O2 -g -ggdb -c -o posix_main.o posix_main.c > posix_main.c:84:30: error: use of undeclared identifier 'B460800' > case 460800: tio.c_cflag |= B460800; break; > ^ > posix_main.c:85:30: error: use of undeclared identifier 'B500000' > case 500000: tio.c_cflag |= B500000; break; > ^ > posix_main.c:86:30: error: use of undeclared identifier 'B576000' > case 576000: tio.c_cflag |= B576000; break; > ^ > posix_main.c:87:30: error: use of undeclared identifier 'B921600' > case 921600: tio.c_cflag |= B921600; break; > ^ > posix_main.c:88:31: error: use of undeclared identifier 'B1000000' > case 1000000: tio.c_cflag |= B1000000; break; > ^ > posix_main.c:89:31: error: use of undeclared identifier 'B1152000' > case 1152000: tio.c_cflag |= B1152000; break; > ^ > posix_main.c:90:31: error: use of undeclared identifier 'B1500000' > case 1500000: tio.c_cflag |= B1500000; break; > ^ > posix_main.c:91:31: error: use of undeclared identifier 'B2000000' > case 2000000: tio.c_cflag |= B2000000; break; > ^ > posix_main.c:92:31: error: use of undeclared identifier 'B2500000' > case 2500000: tio.c_cflag |= B2500000; break; > ^ > posix_main.c:93:31: error: use of undeclared identifier 'B3000000' > case 3000000: tio.c_cflag |= B3000000; break; > ^ > posix_main.c:94:31: error: use of undeclared identifier 'B3500000' > case 3500000: tio.c_cflag |= B3500000; break; > ^ > posix_main.c:95:31: error: use of undeclared identifier 'B4000000' > case 4000000: tio.c_cflag |= B4000000; break; > ^ > 12 errors generated. > make: *** [posix_main.o] Error 1 > /usr/local/bin/erlang/lib/gen_serial:. > > --- > > > On Sun, Mar 12, 2017 at 1:23 PM, J?zsef B?rces > > wrote: > > You can give a try to this one:____ > > http://github.com/tomszilagyi/gen_serial > ____ > > __ __ > > *From:*erlang-questions-bounces@REDACTED > > [mailto:erlang-questions-bounces@REDACTED > ] *On Behalf Of *qp > *Sent:* Sunday, March 12, 2017 17:57 > *To:* Erlang/OTP discussions > > *Subject:* [erlang-questions] Reading serial data from a USB Gamepad____ > > __ __ > > Hi,____ > > I want to read USB data from a Gamepad in Erlang to play a game.____ > > Programs like JoyToKey and Enjoyable convert Gamepad inputs to > keyboard input, but they only send 1 character so using io:read > would require an additional "return" press to send the input.____ > > Where do I start looking to learn how to read gamepads? I've heard > that Erlang ports can be used to interact with the world outside the > Erlang shell but I have no idea where to look. I found erlang-serial > on Github but had trouble installing it > https://github.com/tonyg/erlang-serial/issues/14 > ____ > > Thank you!____ > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From quantumpotato@REDACTED Sun Mar 12 19:19:22 2017 From: quantumpotato@REDACTED (qp) Date: Sun, 12 Mar 2017 14:19:22 -0400 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return Message-ID: I'm looking at the `io` module http://erlang.org/doc/man/io.html#read-1 and finding that reading input uses a Prompt and waits for the user to press Enter (Return?). How do read 1 character at a time without requiring the enter press? I tried -------------- next part -------------- An HTML attachment was scrubbed... URL: From quantumpotato@REDACTED Sun Mar 12 19:19:48 2017 From: quantumpotato@REDACTED (qp) Date: Sun, 12 Mar 2017 14:19:48 -0400 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return In-Reply-To: References: Message-ID: Sorry, I meant to say I tried using `/bin/stty raw` read() -> Res = os:cmd("/bin/stty raw"), io:format(Res), stty: stdin isn't a terminal but get stdin isn't a terminal error On Sun, Mar 12, 2017 at 2:19 PM, qp wrote: > I'm looking at the `io` module http://erlang.org/doc/man/io.html#read-1 > > and finding that reading input uses a Prompt and waits for the user to > press Enter (Return?). > > How do read 1 character at a time without requiring the enter press? > > > > > I tried > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sun Mar 12 21:36:00 2017 From: tony@REDACTED (Tony Rogvall) Date: Sun, 12 Mar 2017 21:36:00 +0100 Subject: [erlang-questions] Reading serial data from a USB Gamepad In-Reply-To: References: Message-ID: https://github.com/tonyrog/uart /Tony > On 12 mar 2017, at 17:57, qp wrote: > > Hi, > > I want to read USB data from a Gamepad in Erlang to play a game. > > Programs like JoyToKey and Enjoyable convert Gamepad inputs to keyboard input, but they only send 1 character so using io:read would require an additional "return" press to send the input. > > Where do I start looking to learn how to read gamepads? I've heard that Erlang ports can be used to interact with the world outside the Erlang shell but I have no idea where to look. I found erlang-serial on Github but had trouble installing it https://github.com/tonyg/erlang-serial/issues/14 > > Thank you! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From quantumpotato@REDACTED Sun Mar 12 21:53:24 2017 From: quantumpotato@REDACTED (qp) Date: Sun, 12 Mar 2017 16:53:24 -0400 Subject: [erlang-questions] Reading serial data from a USB Gamepad In-Reply-To: References: Message-ID: This looks promising, thank you On Sun, Mar 12, 2017 at 4:36 PM, Tony Rogvall wrote: > https://github.com/tonyrog/uart > > /Tony > > > On 12 mar 2017, at 17:57, qp wrote: > > > > Hi, > > > > I want to read USB data from a Gamepad in Erlang to play a game. > > > > Programs like JoyToKey and Enjoyable convert Gamepad inputs to keyboard > input, but they only send 1 character so using io:read would require an > additional "return" press to send the input. > > > > Where do I start looking to learn how to read gamepads? I've heard that > Erlang ports can be used to interact with the world outside the Erlang > shell but I have no idea where to look. I found erlang-serial on Github but > had trouble installing it https://github.com/tonyg/erlang-serial/issues/14 > > > > Thank you! > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Mar 13 10:12:27 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 13 Mar 2017 12:12:27 +0300 Subject: [erlang-questions] Common Test logs In-Reply-To: <6c0e8d67-dda2-d316-97fd-0dc5b0e4da26@ninenines.eu> References: <5b05e1d9-6cf5-8ad5-5321-22f06bdabe5d@ninenines.eu> <3ab597eb-be1e-f746-391a-5a6653e4ab89@erlang.org> <6c0e8d67-dda2-d316-97fd-0dc5b0e4da26@ninenines.eu> Message-ID: The most annoying thing in CT logs for me is that when I run: ct_run -logdir logs/ -dir apps/dvr/test -suite dvr_SUITE -case h264_recording it creates new row in index.html, not updating existing row for dvr app. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Mon Mar 13 13:55:21 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 13 Mar 2017 13:55:21 +0100 Subject: [erlang-questions] Proposal: a new Dbgi BEAM chunk Message-ID: Many tools in the Erlang ecosystem expect the Erlang Abstract Code chunk to exist or, if it doesn't exist, it automatically generates one from the source code, if it can find the respective Erlang source. This restricts the use of the existing tooling to only some languages and leads to code duplication in different tools (such as dialyzer, debuger, cover, etc) as each tool includes their own implemention of loading abstract code from beams, fetching it from source as well as converting the abstract code to other formats. To partially solve this issue for languages that compile directly to Core, such as LFE, I have earlier proposed a chunk that stores Core AST. However, even if we add such chunk, I can foresee the following problems: * Storing the Core AST chunk still does not include the ability of retrieving the AST on the fly in case the chunk is not available for whatever reason * Adding a new chunk could potentially make the situation worse because tools in the future may work directly on those new chunks, forcing compilers to add both Erlang Abstract Format and Code AST chunks to the .beam file. Futhermore, it is expected that languages may want to store their own AST as well, which will lead to further increase on the .beam file size Therefore we need a mechanism to store abstract code on .beam such that: * The abstract code is stored once but can be retrieved in different formats, as supported by the initial language (where the initial language is erlang, core, lfe, elixir, alpaca, etc) * If the abstract code is omitted, we should still provide the ability to retrieve it from source if desired, regardless of the initial language I have written a proposal which aims to unify how abstract code, or generally speaking, debug information is stored on `.beam` by introducing a new chunk, called "Dbgi" which aims to replace the current "Absc" chunk. The proposal is backwards compatible and solves the problems outlined above. The full proposal alongside a prototype can be found on GitHub: https://github.com/erlang/otp/pull/1367 Feedback is welcome! *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Mon Mar 13 14:56:11 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 13 Mar 2017 13:56:11 +0000 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return In-Reply-To: References: Message-ID: Are you sure the gamepad is a serial device? I would expect it to be a HID that goes through the OS's game controller driver plumbing. Maybe tinkering with libSDL in C would be a good place to experiment, and then that could be make into a port program for use with Erlang. On Sun, Mar 12, 2017 at 1:19 PM qp wrote: > I'm looking at the `io` module http://erlang.org/doc/man/io.html#read-1 > > and finding that reading input uses a Prompt and waits for the user to > press Enter (Return?). > > How do read 1 character at a time without requiring the enter press? > > > > > I tried > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Mar 13 16:11:30 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 13 Mar 2017 16:11:30 +0100 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: References: Message-ID: <20170313151130.GA86931@erix.ericsson.se> Hello. We have debugged your issue, and the problem seems to be that lldpLocManAddrEntry is used before defined, so if you move lines 640 through 698 down to before line 1296 in LLDP-MIB the compilation succedes. Now we are trying to figure out if that MIB is valid or not. On Sat, Mar 11, 2017 at 05:23:07PM +0800, fxmy wang wrote: > Hi guys, > I?ve checked out 7f52af8 > > and tried again but the compilation still fails. =( > > I?v updated the mib-compiling log to gist > in case > anyone interested. > > Hope this log holds some useful information for the SNMP experts. > Cheers, > ? > > 2017-03-11 14:01 GMT+08:00 fxmy wang : > > > Hi guys, > > I've checked out [7f52af8](https://github.com/erlang/otp/tree/ > > 7f52af8875a1ab1ba96c6cb2403211bfc876639c) and tried again but the > > compilation still fails. =( > > Below attaches the log file when compiling LLDP-MIB (also uploaded to > > [gits](https://gist.github.com/fxmy/0c84243d387f972a725c68004cc02933)). > > Hope this log holds some useful information for the SNMP experts. > > Cheers, > > > > 2017-03-09 21:28 GMT+08:00 fxmy wang : > > > >> Good to know that Raimo. > >> Thanks, I'm gonna wait for 19.3 and see what happens. > >> Cheers, > >> > >> 2017-03-09 17:01 GMT+08:00 Raimo Niskanen >> ricsson.se>: > >> > >>> There was a fix committed to the maint branch at Jan 12. I do not know > >>> if > >>> it is this problem it fixes; it allows refinements of enum usertypes. > >>> > >>> See https://bugs.erlang.org/browse/ERL-325 > >>> > >>> It will be released in OTP-19.3 shortly. > >>> > >>> > >>> > >>> On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: > >>> > Hi guys, > >>> > So recently I?m toying with SNMP Development Toolkit and having trouble > >>> > compiling standard LLDP-MIB. Errors shown below > >>> > > >>> > 2> snmpc:compile("LLDP-MIB.mib"). > >>> > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry > >>> > lldpLocManAddrEntry > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable > >>> > => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > >>> > lldpConfigManAddrTable => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable > >>> > => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable > >>> > => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => > >>> > The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable > >>> > => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The > >>> > default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable > >>> > => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > >>> > lldpRemUnknownTLVTable => The default functions won't work properly > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > >>> > lldpRemOrgDefInfoTable => The default functions won't work properly > >>> > {error,compilation_failed} > >>> > 3> > >>> > > >>> > The LLDP-MIB file is hosted on ieee802.org > >>> > >>> >. > >>> > > >>> > How should I deal with this? > >>> > > >>> > Cheers, > >>> > ? > >>> > >>> > _______________________________________________ > >>> > erlang-questions mailing list > >>> > erlang-questions@REDACTED > >>> > http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> > >>> -- > >>> > >>> / Raimo Niskanen, Erlang/OTP, Ericsson AB > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From frank.muller.erl@REDACTED Mon Mar 13 18:09:46 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 13 Mar 2017 17:09:46 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang Message-ID: Hi guys What?s the most efficient way to run PHP (Python) scripts behind Erlang? I would like to receive HTTP requests in Cowboy and forward them to a PHP/Python backend. Get the response back from the backend and forward it to the client. Thanks in advance. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Mar 13 18:19:07 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 13 Mar 2017 18:19:07 +0100 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: Message-ID: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI There was an early attempt by Anthony at https://github.com/extend/ex_fcgi - not sure how much of it still works, but probably worth salvaging. On 03/13/2017 06:09 PM, Frank Muller wrote: > Hi guys > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > I would like to receive HTTP requests in Cowboy and forward them to a > PHP/Python backend. > > Get the response back from the backend and forward it to the client. > > Thanks in advance. > /Frank > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From frank.muller.erl@REDACTED Mon Mar 13 18:47:03 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 13 Mar 2017 17:47:03 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: Thanks Lo?c, that was the first lib I found. But it seems to be outdated. I'll will dig into the FastCGI spec soon. Anyone else? /Frank > The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI > > There was an early attempt by Anthony at > https://github.com/extend/ex_fcgi - not sure how much of it still works, > but probably worth salvaging. > > On 03/13/2017 06:09 PM, Frank Muller wrote: > > Hi guys > > > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > > > I would like to receive HTTP requests in Cowboy and forward them to a > > PHP/Python backend. > > > > Get the response back from the backend and forward it to the client. > > > > Thanks in advance. > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Mar 13 18:51:00 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 13 Mar 2017 20:51:00 +0300 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: we are running python flask backend behind flussonic (written in erlang). we launch small launcher and then proxy http requests. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Mon Mar 13 18:58:22 2017 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Mon, 13 Mar 2017 13:58:22 -0400 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: It depends on what you mean by "efficient" :-) Some methods that I've worked with (Python only) - Python uses Flask+Gunicorn to serve requests, Erlang makes HTTP calls to it You get to customize the request as much as you want in Erlang, the coupling is loose, it's easy to do black-box tests on the Python service alone - Python uses Flask+Gunicorn to serve requests, Nginx reverse-proxies to Python and to Erlang, P&E don't talk to each other Nginx is good at forwarding connections, the coupling is loose, it's easy to do black-box tests on either sub-service alone - Erlang calls Python through ErlPort (http://erlport.org/ ) Faster than a network connection, tighter coupling, you manage queueing and pooling yourself > On Mar 13, 2017, at 1:47 PM, Frank Muller wrote: > > Thanks Lo?c, that was the first lib I found. But it seems to be outdated. > > I'll will dig into the FastCGI spec soon. > > Anyone else? > > /Frank > > > > The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI > > There was an early attempt by Anthony at > https://github.com/extend/ex_fcgi - not sure how much of it still works, > but probably worth salvaging. > > On 03/13/2017 06:09 PM, Frank Muller wrote: > > Hi guys > > > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > > > I would like to receive HTTP requests in Cowboy and forward them to a > > PHP/Python backend. > > > > Get the response back from the backend and forward it to the client. > > > > Thanks in advance. > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Mar 13 19:15:34 2017 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 13 Mar 2017 11:15:34 -0700 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: <58C6E1C6.2080106@gmail.com> On 03/13/2017 10:47 AM, Frank Muller wrote: > Thanks Lo?c, that was the first lib I found. But it seems to be outdated. > > I'll will dig into the FastCGI spec soon. > > Anyone else? If you are using PHP and/or Python code as services that are always running for the sake of efficiency and you want to maximize throughput (avoiding an Erlang port of cnode being a bottleneck), CloudI can provide service messaging and be an application server for the PHP/Python with Erlang. The cloudi_service_http_cowboy and cloudi_service_http_elli provide integration for making HTTP requests CloudI service requests. There is some justification at http://cloudi.org/faq.html#4_Erlang and examples at http://cloudi.org/tutorials.html#cloudi_examples . > > /Frank > > > > > The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI > > There was an early attempt by Anthony at > https://github.com/extend/ex_fcgi - not sure how much of it still works, > but probably worth salvaging. > > On 03/13/2017 06:09 PM, Frank Muller wrote: > > Hi guys > > > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > > > I would like to receive HTTP requests in Cowboy and forward them to a > > PHP/Python backend. > > > > Get the response back from the backend and forward it to the client. > > > > Thanks in advance. > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Mon Mar 13 19:53:29 2017 From: nayibor@REDACTED (Nuku Ameyibor) Date: Mon, 13 Mar 2017 18:53:29 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: <58c6eaa9.c284df0a.c2a05.92c1@mx.google.com> Yaws web server also has this . http://yaws.hyber.org/cgi.yaws . -----Original Message----- From: "Frank Muller" Sent: ?3/?13/?2017 5:47 PM To: "Erlang-Questions Questions" ; "Lo?c Hoguin" Subject: Re: [erlang-questions] Serve PHP/Python scripts behind Erlang Thanks Lo?c, that was the first lib I found. But it seems to be outdated. I'll will dig into the FastCGI spec soon. Anyone else? /Frank The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI There was an early attempt by Anthony at https://github.com/extend/ex_fcgi - not sure how much of it still works, but probably worth salvaging. On 03/13/2017 06:09 PM, Frank Muller wrote: > Hi guys > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > I would like to receive HTTP requests in Cowboy and forward them to a > PHP/Python backend. > > Get the response back from the backend and forward it to the client. > > Thanks in advance. > /Frank > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Mar 13 20:06:49 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 13 Mar 2017 19:06:49 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: Hi Max, Can you please elaborate a bit more? How do you launch these PHP instances? Which protocole do you use between Erlang and PHP? /Frank a ?crit : > we are running python flask backend behind flussonic (written in erlang). > > we launch small launcher and then proxy http requests. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Mar 13 20:08:55 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 13 Mar 2017 19:08:55 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: <58C6E1C6.2080106@gmail.com> References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> <58C6E1C6.2080106@gmail.com> Message-ID: Hi Michael, Never heard about CloudI. Sounds very interesting. I'll give it try. /Frank a ?crit : > On 03/13/2017 10:47 AM, Frank Muller wrote: > > Thanks Lo?c, that was the first lib I found. But it seems to be outdated. > > I'll will dig into the FastCGI spec soon. > > Anyone else? > > > If you are using PHP and/or Python code as services that are always > running for the sake of efficiency and you want to maximize throughput > (avoiding an Erlang port of cnode being a bottleneck), CloudI can provide > service messaging and be an application server for the PHP/Python with > Erlang. The cloudi_service_http_cowboy and cloudi_service_http_elli > provide integration for making HTTP requests CloudI service requests. > There is some justification at http://cloudi.org/faq.html#4_Erlang and > examples at http://cloudi.org/tutorials.html#cloudi_examples . > > > > /Frank > > > > The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI > > There was an early attempt by Anthony at > https://github.com/extend/ex_fcgi - not sure how much of it still works, > but probably worth salvaging. > > On 03/13/2017 06:09 PM, Frank Muller wrote: > > Hi guys > > > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > > > I would like to receive HTTP requests in Cowboy and forward them to a > > PHP/Python backend. > > > > Get the response back from the backend and forward it to the client. > > > > Thanks in advance. > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Mar 13 20:10:05 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 13 Mar 2017 19:10:05 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: Nathaniel, Can the erlport solution scale? /Frank a ?crit : > It depends on what you mean by "efficient" :-) Some methods that I've > worked with (Python only) > > - Python uses Flask+Gunicorn to serve requests, Erlang makes HTTP calls to > it > You get to customize the request as much as you want in Erlang, the > coupling is loose, it's easy to do black-box tests on the Python service > alone > - Python uses Flask+Gunicorn to serve requests, Nginx reverse-proxies to > Python and to Erlang, P&E don't talk to each other > Nginx is good at forwarding connections, the coupling is loose, it's > easy to do black-box tests on either sub-service alone > - Erlang calls Python through ErlPort (http://erlport.org/) > Faster than a network connection, tighter coupling, you manage queueing > and pooling yourself > > > > On Mar 13, 2017, at 1:47 PM, Frank Muller > wrote: > > Thanks Lo?c, that was the first lib I found. But it seems to be outdated. > > I'll will dig into the FastCGI spec soon. > > Anyone else? > > /Frank > > > > The typical setup uses fastcgi: https://en.wikipedia.org/wiki/FastCGI > > There was an early attempt by Anthony at > https://github.com/extend/ex_fcgi - not sure how much of it still works, > but probably worth salvaging. > > On 03/13/2017 06:09 PM, Frank Muller wrote: > > Hi guys > > > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > > > I would like to receive HTTP requests in Cowboy and forward them to a > > PHP/Python backend. > > > > Get the response back from the backend and forward it to the client. > > > > Thanks in advance. > > /Frank > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Mar 13 20:53:52 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 13 Mar 2017 22:53:52 +0300 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: I launch via erlang:open_port and struggle a lot trying to make children stop when erlang server dies: it is not easy. then child ruby/python starts listening http port and erlang forwards all requests from wide web to launched http server. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Mon Mar 13 21:32:30 2017 From: matthias@REDACTED (Matthias Lang) Date: Mon, 13 Mar 2017 21:32:30 +0100 Subject: [erlang-questions] has anyone else seen Erlang 19 break 'bash' after exit? Message-ID: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> Hi, Erlang 19.1 and 19.2 seems to break something to do with terminal input on some but not all of my Debian x86-64 machines. I took a quick look through the recent mailing list archives and haven't seen any mention of this. Maybe I missed it. Maybe it's just me. Example: | ~ >uname -a | Linux hec 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 (2017-02-27) x86_64 GNU/Linux | ~ >read x | hello world | ~ >erl | Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] | | Eshell V8.1 (abort with ^G) | 1> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded | (v)ersion (k)ill (D)b-tables (d)istribution | a | ~ >read x | bash: read: read error: 0: Resource temporarily unavailable The "read error: 0: Resource temporarily unavailable" part is unexpected. Using 'cat' instead of 'read' produces a similar error: | cat: -: Resource temporarily unavailable 'strace' tells me something is screwy about stdin: | read(0, 0x7f1010dbb000, 131072) = -1 EAGAIN (Resource temporarily unavailable) read(2) tells me EAGAIN suggests that stdin has been left in with O_NONBLOCK. Is that Erlang's problem to clean up, or is that supposed to happen on program exit? I haven't read up on that. Doing the same thing on 17.4 works fine. Debian 3.16.39-1 and erts-8.2.1 produces a related by different strangeness: | 1> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded | (v)ersion (k)ill (D)b-tables (d)istribution | a | $ sh: turning off NDELAY mode Can anyone else see this? Any idea what's up? Matt From trapexit@REDACTED Mon Mar 13 21:48:51 2017 From: trapexit@REDACTED (Antonio SJ Musumeci) Date: Mon, 13 Mar 2017 16:48:51 -0400 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: Max, If you control the child code it shouldn't be too difficult to manage the exiting of children when erlang dies. Even if you don't control it you can create a little shim that does. Few things you can mix and match: 1) On Linux you can use prctl with PR_SET_PDEATHSIG 2) look at parent pid on occasion. If it becomes 1 (or not what it was at start, slight race condition but you can set the value as an envvar) then you know your parent is dead. 3) In a shim use the old pipe trick ( http://skarnet.org/software/s6/s6-fghack.html). If the breaks then the other side is gone. If using the shim approach you can then kill the child when the shim detects the parent's death. On Mon, Mar 13, 2017 at 3:53 PM, Max Lapshin wrote: > I launch via erlang:open_port and struggle a lot trying to make children > stop when erlang server dies: it is not easy. > > then child ruby/python starts listening http port and erlang forwards all > requests from wide web to launched http server. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Mar 14 06:16:49 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 14 Mar 2017 18:16:49 +1300 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return In-Reply-To: References: Message-ID: <32E7B13D-78DE-42F0-97F0-C1A32AF704AE@cs.otago.ac.nz> > On 13/03/2017, at 7:19 AM, qp wrote: > > How do read 1 character at a time without requiring the enter press? The obvious question is "why do you want to do this"? It's actually nowhere as simple an issue as you might think. Seriously, it's not clear what "read 1 character at a time" MEANS. Do you mean - read a keycode? - read a byte? - read a possibly multibyte character? and - if the user pushes a an error or function key, sending an ESC [ .... sequence, do you want just the ESC (and if so what happens to everything after it) or the whole thing - how are you going to tell the difference between ESC [ ... sent from a special key and an ESC sent from the ESC key (I've seen two approaches: don't try to distinguish, and use a timeout) and + do you want the character to be echoed - or not + do you want some interrupt characters to be heeded still - or not and - if there are multiple characters already buffered (from a time when the terminal was in a "cooked" mode, perhaps) and you switch to whatever character-at-a-time mode you want, what is to happen to those characters? and above all, * given that Erlang uses GNU readline or something very like it, even when calling io:get_chars/2, how are you NOT going to mess that up? Having said that, http://erlang.org/pipermail/erlang-questions/2009-November/047298.html tells us that Joe Armstrong wrote an emacs-like editor in Erlang. I wrote a simple emacs editor years ago - it's in the widgets subdirectory of http://www.sics.se/~joe/ex11/download/release-2.5.tgz The only tricky part was not the emacs logic, but the screen display and catching the keystrokes and mouse events. That link is dead, but https://github.com/baryluk/ex11 should work. It's not *much* of an emacs: ^A ^E ^D ^B ^F PgUp PgDn arrow keys and inserting plain characters. Call it proof-of-concept. An alternative would be to use a separate program to do this. From vladdu55@REDACTED Tue Mar 14 09:16:43 2017 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 14 Mar 2017 09:16:43 +0100 Subject: [erlang-questions] Proposal: a new Dbgi BEAM chunk In-Reply-To: References: Message-ID: Hi, On Mon, Mar 13, 2017 at 1:55 PM, Jos? Valim wrote: > Many tools in the Erlang ecosystem expect the Erlang Abstract Code chunk > to exist or, if it doesn't exist, it automatically generates one from the > source code, if it can find the respective Erlang source. > > I have written a proposal which aims to unify how abstract code, or > generally speaking, debug information is stored on `.beam` by introducing a > new chunk, called "Dbgi" which aims to replace the current "Absc" chunk. > The proposal is backwards compatible and solves the problems outlined above. > > The full proposal alongside a prototype can be found on GitHub: > https://github.com/erlang/otp/pull/1367 > I think it's a good and useful idea. +1 best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Mar 14 10:49:24 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 14 Mar 2017 10:49:24 +0100 Subject: [erlang-questions] Proposal: a new Dbgi BEAM chunk In-Reply-To: References: Message-ID: <97cb5096-ada5-e0e8-faf7-ee9bd7210ca4@cs.ntua.gr> On 03/13/2017 01:55 PM, Jos? Valim wrote: > Many tools in the Erlang ecosystem expect the Erlang Abstract Code chunk > to exist or, if it doesn't exist, it automatically generates one from > the source code, if it can find the respective Erlang source. This is the point where I got confused... - What does the mechanism that finds the source code have to do with the new chunk which is stored in the .beam file? These two are totally orthogonal mechanisms, aren't they? - How is finding "the respective Erlang source" related to solving the problems that LFE or other languages (existing and future ones) may be facing? Does the proposal come with some magic mechanism to "find" (I guess "generate" is a more appropriate word here) Erlang source code from e.g. LFE source? Don't misunderstand me, I am not necessarily against the proposal. It's just that I do not see why/how renaming a BEAM chunk is helping us solve problems that are orthogonal to the info that gets stored in this particular chunk. > Therefore we need a mechanism to store abstract code on .beam such that: > > * The abstract code is stored once but can be retrieved in different > formats, as supported by the initial language (where the initial > language is erlang, core, lfe, elixir, alpaca, etc) > > * If the abstract code is omitted, we should still provide the ability > to retrieve it from source if desired, regardless of the initial language Does this mean that it will be impossible to hide the original source code from now on? Does this mean that if I have a .beam file lying around from long ago or I have written a compiler that generates .beam files without a .Dbgi chuck this is not a valid .beam file anymore? How is that "backwards compatible"? (as claimed in the PR) Apologies if I have misunderstood something... Kostis From jose.valim@REDACTED Tue Mar 14 11:24:04 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Tue, 14 Mar 2017 11:24:04 +0100 Subject: [erlang-questions] Proposal: a new Dbgi BEAM chunk In-Reply-To: <97cb5096-ada5-e0e8-faf7-ee9bd7210ca4@cs.ntua.gr> References: <97cb5096-ada5-e0e8-faf7-ee9bd7210ca4@cs.ntua.gr> Message-ID: Hi Kostis, Thanks for the comments. Answers inline. > - What does the mechanism that finds the source code have to do with the > new chunk which is stored in the .beam file? These two are totally > orthogonal mechanisms, aren't they? > The new proposed Dbgi chunk does not follow the same format as the Abst chunk. It is made of three fields: {debug_info_v1, Backend, Metadata | none} The backend field must be a module that knows how to: - How to convert Metadata to different formats. For example, Elixir will likely store Elixir AST in the Metadata field and be able to convert the Metadata field to Elixir AST, Erlang AST and Core AST. - How to retrieve the AST from source if Metadata is none. The process will likely involve: 1. find the source for the beam file in the :compile attributes 2. parse the source file and 3. convert it to desired format. That's exactly how fetching abstract code from source works today on tools like cover and debugger The proposed API for the Backend is outlined in the PR: https://github.com/erlang/otp/pull/1367 - How is finding "the respective Erlang source" related to solving the > problems that LFE or other languages (existing and future ones) may be > facing? Does the proposal come with some magic mechanism to "find" (I > guess "generate" is a more appropriate word here) Erlang source code from > e.g. LFE source? > As per above, the Dbgi chunk contains the backend module and the backend module has the implementation of how to retrieve the AST from source. That's why it is important for functions like beam_lib:strip/1 to not erase the Dbgi chunk but instead set the metadata field to none. > Don't misunderstand me, I am not necessarily against the proposal. It's > just that I do not see why/how renaming a BEAM chunk is helping us solve > problems that are orthogonal to the info that gets stored in this > particular chunk. Hopefully the points above clarify it. We are not only renaming the chunk, we are adding extra information to it as well and changing the shape of the metadata stored (which is why a new chunk is required). > Does this mean that it will be impossible to hide the original source code > from now on? > This behaviour will be the same as today. To fully answer the question, let's outline how tools that need the AST work today: 1. Attempt to load the AST from the beam chunk 2. If the AST is not available, see if there is a source file on disk 3. If the source file is available, parse it and convert to AST In other words, the process of hiding a source from a tool is: 1. You can encrypt debug_info 2. Or you can pass debug_info false and remove the source from disk Today, if you set debug_info to false but the source is still on disk, most tools will end-up building the AST from source. If you don't want that reconstruction then the source must not be available on disk. I aim to keep this behaviour. > Does this mean that if I have a .beam file lying around from long ago or I > have written a compiler that generates .beam files without a .Dbgi chuck > this is not a valid .beam file anymore? How is that "backwards > compatible"? (as claimed in the PR) > The beam_lib:chunk(BinOrPath, [:abstract_code]) will continue to look for the Abst chunk for at least 3 releases for backwards compatibility reasons. It will work like this: * Look for the Dbgi chunk, if it is available, it will ask the backend to convert the metadata to Erlang format * If the Dbgi chunk is not available, it will look at the old Abst chunk and return it This means that beam_lib will be able to handle the differences between old and new beams. The only exception is if you lookup directly for the "Abst" chunk, which now will no longer be available, but that should not cause errors because the chunk has always been optional. Your feedback here is very valuable because you have built many tools that work on core. With the proposal above, I hope such tools will have code like this: case beam_lib:chunks(Beam, [debug_info]) of {ok,{Module,[{debug_info, {debug_info_v1, Backend, Metadata}}]}} -> case Backend:debug_info(core, Module, Metadata, [allow_source_lookup]) of {ok, CoreAST} -> {error, Reason} -> %% handle error end {error, Reason} -> %% handle error end The tool no longer needs to retrieve Erlang AST and translate it to core nor know how to perform source lookups. Furthermore, the tool will work with any language that knows how to emit Core AST from the information stored in the Dbgi chunk. Please let me know if there are more questions or points I should clarify, *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From lawrence.ansell@REDACTED Tue Mar 14 11:34:24 2017 From: lawrence.ansell@REDACTED (Lawrence Ansell) Date: Tue, 14 Mar 2017 10:34:24 +0000 Subject: [erlang-questions] [ANN] Only 9 days left until EEF17 San Francisco, register now! Message-ID: [ANN] Hurry - Only 9 days left until EEF17 San Francisco, the best Erlang/Elixir conference in USA, register now for your conference ticket, training and tutorials: http://www.erlang-factory.com/sfbay2017/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Tue Mar 14 13:35:43 2017 From: igor.clark@REDACTED (Igor Clark) Date: Tue, 14 Mar 2017 12:35:43 +0000 Subject: [erlang-questions] has anyone else seen Erlang 19 break 'bash' after exit? In-Reply-To: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> References: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> Message-ID: Hello Matthias, Not sure if it's related, but I see bash v4.4.5 terminal I/O get really messed up on Mac OSX after the erlang runtime aborts. It happens whether through a NIF segfault during dev, or just by doing ^C and 'abort'. The terminal input becomes unusable, vim buffers don't display roughly half the text; 'reset' makes no difference, I have to close the Terminal.app window and start over. Haven't had time to dig into it at all, just kind of got used to it. I thought it might be R19, but I switched back to R18 and it still happened. Which made me wonder if it's anything to do with tmux, but I tried it in a standard Terminal window without tmux, and it happened then too. So I tried using the default OSX /bin/bash (which is v3.2.57) instead of the homebrew version (4.4.5) - and it didn't happen at all. Which is a long-winded way of saying, maybe this might also be a "recent bash" issue, rather than an erlang issue? Cheers, Igor On 13/03/2017 20:32, Matthias Lang wrote: > Hi, > > Erlang 19.1 and 19.2 seems to break something to do with terminal > input on some but not all of my Debian x86-64 machines. I took a quick > look through the recent mailing list archives and haven't seen any > mention of this. Maybe I missed it. Maybe it's just me. > > Example: > > | ~ >uname -a > | Linux hec 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 (2017-02-27) x86_64 GNU/Linux > | ~ >read x > | hello world > | ~ >erl > | Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] > | > | Eshell V8.1 (abort with ^G) > | 1> > | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > | (v)ersion (k)ill (D)b-tables (d)istribution > | a > | ~ >read x > | bash: read: read error: 0: Resource temporarily unavailable > > The "read error: 0: Resource temporarily unavailable" part is unexpected. > Using 'cat' instead of 'read' produces a similar error: > > | cat: -: Resource temporarily unavailable > > 'strace' tells me something is screwy about stdin: > > | read(0, 0x7f1010dbb000, 131072) = -1 EAGAIN (Resource temporarily unavailable) > > read(2) tells me EAGAIN suggests that stdin has been left in with O_NONBLOCK. > Is that Erlang's problem to clean up, or is that supposed to happen on > program exit? I haven't read up on that. > > Doing the same thing on 17.4 works fine. > > Debian 3.16.39-1 and erts-8.2.1 produces a related by different strangeness: > > | 1> > | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > | (v)ersion (k)ill (D)b-tables (d)istribution > | a > | $ sh: turning off NDELAY mode > > Can anyone else see this? Any idea what's up? > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From raimo+erlang-questions@REDACTED Tue Mar 14 15:53:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 14 Mar 2017 15:53:09 +0100 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: <20170313151130.GA86931@erix.ericsson.se> References: <20170313151130.GA86931@erix.ericsson.se> Message-ID: <20170314145309.GA70006@erix.ericsson.se> I have now tried with the 'smilint' tool and it does not find any errors, so this is probably a legitimate way to write a MIB. If so our MIB compiler needs to be rewritten to do two passes; one handling anything not augmented, and one handling augmentation. Please write a Jira ticket at bugs.erlang.org so we get a case to prioritize, if the workaround below to restructure the LLDP-MIB is not good enough for you... Best regards / Raimo Niskanen, Erlang/OTP On Mon, Mar 13, 2017 at 04:11:30PM +0100, Raimo Niskanen wrote: > Hello. > > We have debugged your issue, and the problem seems to be that > lldpLocManAddrEntry is used before defined, so if you move lines 640 > through 698 down to before line 1296 in LLDP-MIB the compilation succedes. > > Now we are trying to figure out if that MIB is valid or not. > > > > On Sat, Mar 11, 2017 at 05:23:07PM +0800, fxmy wang wrote: > > Hi guys, > > I?ve checked out 7f52af8 > > > > and tried again but the compilation still fails. =( > > > > I?v updated the mib-compiling log to gist > > in case > > anyone interested. > > > > Hope this log holds some useful information for the SNMP experts. > > Cheers, > > ? > > > > 2017-03-11 14:01 GMT+08:00 fxmy wang : > > > > > Hi guys, > > > I've checked out [7f52af8](https://github.com/erlang/otp/tree/ > > > 7f52af8875a1ab1ba96c6cb2403211bfc876639c) and tried again but the > > > compilation still fails. =( > > > Below attaches the log file when compiling LLDP-MIB (also uploaded to > > > [gits](https://gist.github.com/fxmy/0c84243d387f972a725c68004cc02933)). > > > Hope this log holds some useful information for the SNMP experts. > > > Cheers, > > > > > > 2017-03-09 21:28 GMT+08:00 fxmy wang : > > > > > >> Good to know that Raimo. > > >> Thanks, I'm gonna wait for 19.3 and see what happens. > > >> Cheers, > > >> > > >> 2017-03-09 17:01 GMT+08:00 Raimo Niskanen > >> ricsson.se>: > > >> > > >>> There was a fix committed to the maint branch at Jan 12. I do not know > > >>> if > > >>> it is this problem it fixes; it allows refinements of enum usertypes. > > >>> > > >>> See https://bugs.erlang.org/browse/ERL-325 > > >>> > > >>> It will be released in OTP-19.3 shortly. > > >>> > > >>> > > >>> > > >>> On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: > > >>> > Hi guys, > > >>> > So recently I?m toying with SNMP Development Toolkit and having trouble > > >>> > compiling standard LLDP-MIB. Errors shown below > > >>> > > > >>> > 2> snmpc:compile("LLDP-MIB.mib"). > > >>> > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry > > >>> > lldpLocManAddrEntry > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpPortConfigTable > > >>> > => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > >>> > lldpConfigManAddrTable => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsTxPortTable > > >>> > => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpStatsRxPortTable > > >>> > => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocPortTable => > > >>> > The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpLocManAddrTable > > >>> > => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable => The > > >>> > default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemManAddrTable > > >>> > => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > >>> > lldpRemUnknownTLVTable => The default functions won't work properly > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > >>> > lldpRemOrgDefInfoTable => The default functions won't work properly > > >>> > {error,compilation_failed} > > >>> > 3> > > >>> > > > >>> > The LLDP-MIB file is hosted on ieee802.org > > >>> > > >>> >. > > >>> > > > >>> > How should I deal with this? > > >>> > > > >>> > Cheers, > > >>> > ? > > >>> > > >>> > _______________________________________________ > > >>> > erlang-questions mailing list > > >>> > erlang-questions@REDACTED > > >>> > http://erlang.org/mailman/listinfo/erlang-questions > > >>> > > >>> > > >>> -- > > >>> > > >>> / Raimo Niskanen, Erlang/OTP, Ericsson AB > > >>> _______________________________________________ > > >>> erlang-questions mailing list > > >>> erlang-questions@REDACTED > > >>> http://erlang.org/mailman/listinfo/erlang-questions > > >>> > > >> > > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From quantumpotato@REDACTED Tue Mar 14 16:03:12 2017 From: quantumpotato@REDACTED (qp) Date: Tue, 14 Mar 2017 11:03:12 -0400 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return In-Reply-To: <32E7B13D-78DE-42F0-97F0-C1A32AF704AE@cs.otago.ac.nz> References: <32E7B13D-78DE-42F0-97F0-C1A32AF704AE@cs.otago.ac.nz> Message-ID: I see. I was wanting to get gamepad input to control a game. I naively assumed this was serial, Daniel's reply suggested the gamepad might be a HID device which I haven't checked yet. "how are you NOT going to mess that up?" coincides with the response on http://stackoverflow.com/questions/42750491/read-a-character-input-from-erlang-without-requiring-the-return-key-pressed-from/42755503#42755503 "you are entering a world of pain you never knew existed" So for now, I'm running a Cowboy server and using a c program to send characters pressed through cURL ??#include #include int main(void){ system ("clear"); int c; /* use system call to make terminal send all keystrokes directly to stdin */ system ("/bin/stty raw"); while((c=getchar())!= '.') { char command[100]; sprintf(command, "curl localhost:8080/info?%c" ,c); system(command); printf("\r\n"); /* type a period to break out of the loop, since CTRL-D won't work raw */ } /* use system call to set terminal behaviour to more normal behaviour */ system ("/bin/stty cooked"); system ("clear"); system ("echo ok\n"); return 0; } On Tue, Mar 14, 2017 at 1:16 AM, Richard A. O'Keefe wrote: > > > On 13/03/2017, at 7:19 AM, qp wrote: > > > > How do read 1 character at a time without requiring the enter press? > > The obvious question is "why do you want to do this"? > > It's actually nowhere as simple an issue as you might think. > Seriously, it's not clear what "read 1 character at a time" > MEANS. Do you mean > - read a keycode? > - read a byte? > - read a possibly multibyte character? > and > - if the user pushes a an error or function key, > sending an ESC [ .... sequence, do you > want just the ESC (and if so what happens to everything > after it) or the whole thing > - how are you going to tell the difference between ESC [ ... > sent from a special key and an ESC sent from the ESC key > (I've seen two approaches: don't try to distinguish, and > use a timeout) > and > + do you want the character to be echoed > - or not > + do you want some interrupt characters to be > heeded still > - or not > and > - if there are multiple characters already buffered > (from a time when the terminal was in a "cooked" mode, perhaps) > and you switch to whatever character-at-a-time mode you want, > what is to happen to those characters? > and above all, > * given that Erlang uses GNU readline or something very like > it, even when calling io:get_chars/2, how are you NOT going > to mess that up? > > Having said that, > http://erlang.org/pipermail/erlang-questions/2009-November/047298.html > tells us that Joe Armstrong wrote an emacs-like editor in Erlang. > > I wrote a simple emacs editor years ago - > it's in the widgets subdirectory of > http://www.sics.se/~joe/ex11/download/release-2.5.tgz > > The only tricky part was not the emacs logic, > but the screen display > and catching the keystrokes and mouse events. > > > That link is dead, but https://github.com/baryluk/ex11 > should work. > > It's not *much* of an emacs: ^A ^E ^D ^B ^F PgUp PgDn > arrow keys and inserting plain characters. Call it > proof-of-concept. > > An alternative would be to use a separate program to do this. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Mar 14 16:40:21 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 14 Mar 2017 16:40:21 +0100 Subject: [erlang-questions] has anyone else seen Erlang 19 break 'bash' after exit? In-Reply-To: References: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> Message-ID: Hello, I think that it may be a change done in Erlang/OTP 18 that causes this behaviour. Specifically OTP-12239. Using non-blocking I/O on stdin/stdout seems to kind of work, until it doesn't. Different platforms and tools handle it very differently. The only "solution" that I can think of right now is to use dedicated read and/or write threads within the VM to handle stdin/stdout, but I was hoping to avoid having to do that. The problem, for those that are interested, is that the O_NONBLOCK flag is set on the file description, not the file descriptor. Which causes all kinds of problems, I thought I had handled them all, but apparently not enough. Lukas On Tue, Mar 14, 2017 at 1:35 PM, Igor Clark wrote: > Hello Matthias, > > Not sure if it's related, but I see bash v4.4.5 terminal I/O get really > messed up on Mac OSX after the erlang runtime aborts. It happens whether > through a NIF segfault during dev, or just by doing ^C and 'abort'. The > terminal input becomes unusable, vim buffers don't display roughly half the > text; 'reset' makes no difference, I have to close the Terminal.app window > and start over. Haven't had time to dig into it at all, just kind of got > used to it. > > I thought it might be R19, but I switched back to R18 and it still > happened. Which made me wonder if it's anything to do with tmux, but I > tried it in a standard Terminal window without tmux, and it happened then > too. So I tried using the default OSX /bin/bash (which is v3.2.57) instead > of the homebrew version (4.4.5) - and it didn't happen at all. > > Which is a long-winded way of saying, maybe this might also be a "recent > bash" issue, rather than an erlang issue? > > Cheers, > Igor > > > On 13/03/2017 20:32, Matthias Lang wrote: > >> Hi, >> >> Erlang 19.1 and 19.2 seems to break something to do with terminal >> input on some but not all of my Debian x86-64 machines. I took a quick >> look through the recent mailing list archives and haven't seen any >> mention of this. Maybe I missed it. Maybe it's just me. >> >> Example: >> >> | ~ >uname -a >> | Linux hec 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 (2017-02-27) x86_64 >> GNU/Linux >> | ~ >read x >> | hello world >> | ~ >erl >> | Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] >> [async-threads:10] [hipe] [kernel-poll:false] >> | >> | Eshell V8.1 (abort with ^G) >> | 1> >> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded >> | (v)ersion (k)ill (D)b-tables (d)istribution >> | a >> | ~ >read x >> | bash: read: read error: 0: Resource temporarily unavailable >> >> The "read error: 0: Resource temporarily unavailable" part is unexpected. >> Using 'cat' instead of 'read' produces a similar error: >> >> | cat: -: Resource temporarily unavailable >> >> 'strace' tells me something is screwy about stdin: >> >> | read(0, 0x7f1010dbb000, 131072) = -1 EAGAIN (Resource >> temporarily unavailable) >> >> read(2) tells me EAGAIN suggests that stdin has been left in with >> O_NONBLOCK. >> Is that Erlang's problem to clean up, or is that supposed to happen on >> program exit? I haven't read up on that. >> >> Doing the same thing on 17.4 works fine. >> >> Debian 3.16.39-1 and erts-8.2.1 produces a related by different >> strangeness: >> >> | 1> >> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded >> | (v)ersion (k)ill (D)b-tables (d)istribution >> | a >> | $ sh: turning off NDELAY mode >> >> Can anyone else see this? Any idea what's up? >> >> Matt >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Tue Mar 14 21:51:47 2017 From: igor.clark@REDACTED (Igor Clark) Date: Tue, 14 Mar 2017 20:51:47 +0000 Subject: [erlang-questions] has anyone else seen Erlang 19 break 'bash' after exit? In-Reply-To: References: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> Message-ID: <2b532230-ba42-fc2e-8095-e7ef1809cea3@gmail.com> Hi Lukas, I just tried with erlang R17 to check, and it seems to work fine with the newer homebrew version of bash as well as the older default /bin/bash. So maybe this helps confirm your suspicion. best, Igor On 14/03/2017 15:40, Lukas Larsson wrote: > Hello, > > I think that it may be a change done in Erlang/OTP 18 that causes this > behaviour. Specifically OTP-12239. Using non-blocking I/O on > stdin/stdout seems to kind of work, until it doesn't. Different > platforms and tools handle it very differently. > > The only "solution" that I can think of right now is to use dedicated > read and/or write threads within the VM to handle stdin/stdout, but I > was hoping to avoid having to do that. > > The problem, for those that are interested, is that the O_NONBLOCK > flag is set on the file description, not the file descriptor. Which > causes all kinds of problems, I thought I had handled them all, but > apparently not enough. > > Lukas > > On Tue, Mar 14, 2017 at 1:35 PM, Igor Clark > wrote: > > Hello Matthias, > > Not sure if it's related, but I see bash v4.4.5 terminal I/O get > really messed up on Mac OSX after the erlang runtime aborts. It > happens whether through a NIF segfault during dev, or just by > doing ^C and 'abort'. The terminal input becomes unusable, vim > buffers don't display roughly half the text; 'reset' makes no > difference, I have to close the Terminal.app window and start > over. Haven't had time to dig into it at all, just kind of got > used to it. > > I thought it might be R19, but I switched back to R18 and it still > happened. Which made me wonder if it's anything to do with tmux, > but I tried it in a standard Terminal window without tmux, and it > happened then too. So I tried using the default OSX /bin/bash > (which is v3.2.57) instead of the homebrew version (4.4.5) - and > it didn't happen at all. > > Which is a long-winded way of saying, maybe this might also be a > "recent bash" issue, rather than an erlang issue? > > Cheers, > Igor > > > On 13/03/2017 20:32, Matthias Lang wrote: > > Hi, > > Erlang 19.1 and 19.2 seems to break something to do with terminal > input on some but not all of my Debian x86-64 machines. I took > a quick > look through the recent mailing list archives and haven't seen any > mention of this. Maybe I missed it. Maybe it's just me. > > Example: > > | ~ >uname -a > | Linux hec 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 > (2017-02-27) x86_64 GNU/Linux > | ~ >read x > | hello world > | ~ >erl > | Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:false] > | > | Eshell V8.1 (abort with ^G) > | 1> > | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > | (v)ersion (k)ill (D)b-tables (d)istribution > | a > | ~ >read x > | bash: read: read error: 0: Resource temporarily unavailable > > The "read error: 0: Resource temporarily unavailable" part is > unexpected. > Using 'cat' instead of 'read' produces a similar error: > > | cat: -: Resource temporarily unavailable > > 'strace' tells me something is screwy about stdin: > > | read(0, 0x7f1010dbb000, 131072) = -1 EAGAIN > (Resource temporarily unavailable) > > read(2) tells me EAGAIN suggests that stdin has been left in > with O_NONBLOCK. > Is that Erlang's problem to clean up, or is that supposed to > happen on > program exit? I haven't read up on that. > > Doing the same thing on 17.4 works fine. > > Debian 3.16.39-1 and erts-8.2.1 produces a related by > different strangeness: > > | 1> > | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > | (v)ersion (k)ill (D)b-tables (d)istribution > | a > | $ sh: turning off NDELAY mode > > Can anyone else see this? Any idea what's up? > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Tue Mar 14 23:54:08 2017 From: tony@REDACTED (Tony Rogvall) Date: Tue, 14 Mar 2017 23:54:08 +0100 Subject: [erlang-questions] Read only 1 character from STDin without requiring Return In-Reply-To: References: <32E7B13D-78DE-42F0-97F0-C1A32AF704AE@cs.otago.ac.nz> Message-ID: > On 14 mar 2017, at 16:03, qp wrote: > > I see. I was wanting to get gamepad input to control a game. I naively assumed this was serial, Daniel's reply suggested the gamepad might be a HID device which I haven't checked yet. If that is the case then you might use https://github.com/tonyrog/hid :-) /Tony > > "how are you NOT going to mess that up?" coincides with the response on http://stackoverflow.com/questions/42750491/read-a-character-input-from-erlang-without-requiring-the-return-key-pressed-from/42755503#42755503 "you are entering a world of pain you never knew existed" > > So for now, I'm running a Cowboy server and using a c program to send characters pressed through cURL > > ??#include > #include > > int main(void){ > system ("clear"); > int c; > /* use system call to make terminal send all keystrokes directly to stdin */ > system ("/bin/stty raw"); > while((c=getchar())!= '.') { > char command[100]; > sprintf(command, "curl localhost:8080/info?%c" ,c); > system(command); > printf("\r\n"); > > /* type a period to break out of the loop, since CTRL-D won't work raw */ > } > /* use system call to set terminal behaviour to more normal behaviour */ > system ("/bin/stty cooked"); > system ("clear"); > system ("echo ok\n"); > return 0; > } > > > On Tue, Mar 14, 2017 at 1:16 AM, Richard A. O'Keefe wrote: > > > On 13/03/2017, at 7:19 AM, qp wrote: > > > > How do read 1 character at a time without requiring the enter press? > > The obvious question is "why do you want to do this"? > > It's actually nowhere as simple an issue as you might think. > Seriously, it's not clear what "read 1 character at a time" > MEANS. Do you mean > - read a keycode? > - read a byte? > - read a possibly multibyte character? > and > - if the user pushes a an error or function key, > sending an ESC [ .... sequence, do you > want just the ESC (and if so what happens to everything > after it) or the whole thing > - how are you going to tell the difference between ESC [ ... > sent from a special key and an ESC sent from the ESC key > (I've seen two approaches: don't try to distinguish, and > use a timeout) > and > + do you want the character to be echoed > - or not > + do you want some interrupt characters to be > heeded still > - or not > and > - if there are multiple characters already buffered > (from a time when the terminal was in a "cooked" mode, perhaps) > and you switch to whatever character-at-a-time mode you want, > what is to happen to those characters? > and above all, > * given that Erlang uses GNU readline or something very like > it, even when calling io:get_chars/2, how are you NOT going > to mess that up? > > Having said that, > http://erlang.org/pipermail/erlang-questions/2009-November/047298.html > tells us that Joe Armstrong wrote an emacs-like editor in Erlang. > > I wrote a simple emacs editor years ago - > it's in the widgets subdirectory of > http://www.sics.se/~joe/ex11/download/release-2.5.tgz > > The only tricky part was not the emacs logic, > but the screen display > and catching the keystrokes and mouse events. > > > That link is dead, but https://github.com/baryluk/ex11 > should work. > > It's not *much* of an emacs: ^A ^E ^D ^B ^F PgUp PgDn > arrow keys and inserting plain characters. Call it > proof-of-concept. > > An alternative would be to use a separate program to do this. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From fxmywc@REDACTED Wed Mar 15 08:58:20 2017 From: fxmywc@REDACTED (fxmy wang) Date: Wed, 15 Mar 2017 15:58:20 +0800 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: <20170314145309.GA70006@erix.ericsson.se> References: <20170313151130.GA86931@erix.ericsson.se> <20170314145309.GA70006@erix.ericsson.se> Message-ID: Nice to hear that Raimo. I'll fire a bug then. =) Cheers, 2017-03-14 22:53 GMT+08:00 Raimo Niskanen < raimo+erlang-questions@REDACTED>: > I have now tried with the 'smilint' tool and it does not find any errors, > so this is probably a legitimate way to write a MIB. > > If so our MIB compiler needs to be rewritten to do two passes; one handling > anything not augmented, and one handling augmentation. > > Please write a Jira ticket at bugs.erlang.org so we get a case to > prioritize, if the workaround below to restructure the LLDP-MIB is not good > enough for you... > > Best regards > / Raimo Niskanen, Erlang/OTP > > > On Mon, Mar 13, 2017 at 04:11:30PM +0100, Raimo Niskanen wrote: > > Hello. > > > > We have debugged your issue, and the problem seems to be that > > lldpLocManAddrEntry is used before defined, so if you move lines 640 > > through 698 down to before line 1296 in LLDP-MIB the compilation > succedes. > > > > Now we are trying to figure out if that MIB is valid or not. > > > > > > > > On Sat, Mar 11, 2017 at 05:23:07PM +0800, fxmy wang wrote: > > > Hi guys, > > > I?ve checked out 7f52af8 > > > bfc876639c> > > > and tried again but the compilation still fails. =( > > > > > > I?v updated the mib-compiling log to gist > > > in > case > > > anyone interested. > > > > > > Hope this log holds some useful information for the SNMP experts. > > > Cheers, > > > ? > > > > > > 2017-03-11 14:01 GMT+08:00 fxmy wang : > > > > > > > Hi guys, > > > > I've checked out [7f52af8](https://github.com/erlang/otp/tree/ > > > > 7f52af8875a1ab1ba96c6cb2403211bfc876639c) and tried again but the > > > > compilation still fails. =( > > > > Below attaches the log file when compiling LLDP-MIB (also uploaded > to > > > > [gits](https://gist.github.com/fxmy/0c84243d387f972a725c68004cc029 > 33)). > > > > Hope this log holds some useful information for the SNMP experts. > > > > Cheers, > > > > > > > > 2017-03-09 21:28 GMT+08:00 fxmy wang : > > > > > > > >> Good to know that Raimo. > > > >> Thanks, I'm gonna wait for 19.3 and see what happens. > > > >> Cheers, > > > >> > > > >> 2017-03-09 17:01 GMT+08:00 Raimo Niskanen > > > >> ricsson.se>: > > > >> > > > >>> There was a fix committed to the maint branch at Jan 12. I do not > know > > > >>> if > > > >>> it is this problem it fixes; it allows refinements of enum > usertypes. > > > >>> > > > >>> See https://bugs.erlang.org/browse/ERL-325 > > > >>> > > > >>> It will be released in OTP-19.3 shortly. > > > >>> > > > >>> > > > >>> > > > >>> On Thu, Mar 09, 2017 at 11:00:39AM +0800, fxmy wang wrote: > > > >>> > Hi guys, > > > >>> > So recently I?m toying with SNMP Development Toolkit and having > trouble > > > >>> > compiling standard LLDP-MIB. Errors shown below > > > >>> > > > > >>> > 2> snmpc:compile("LLDP-MIB.mib"). > > > >>> > LLDP-MIB.mib: 655: Cannot AUGMENT the non-existing table entry > > > >>> > lldpLocManAddrEntry > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpPortConfigTable > > > >>> > => The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > > >>> > lldpConfigManAddrTable => The default functions won't work > properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpStatsTxPortTable > > > >>> > => The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpStatsRxPortTable > > > >>> > => The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpLocPortTable => > > > >>> > The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpLocManAddrTable > > > >>> > => The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table lldpRemTable > => The > > > >>> > default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > lldpRemManAddrTable > > > >>> > => The default functions won't work properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > > >>> > lldpRemUnknownTLVTable => The default functions won't work > properly > > > >>> > [LLDP-MIB.mib][WAR]: No RowStatus column in table > > > >>> > lldpRemOrgDefInfoTable => The default functions won't work > properly > > > >>> > {error,compilation_failed} > > > >>> > 3> > > > >>> > > > > >>> > The LLDP-MIB file is hosted on ieee802.org > > > >>> > 200505060000Z.txt > > > >>> >. > > > >>> > > > > >>> > How should I deal with this? > > > >>> > > > > >>> > Cheers, > > > >>> > ? > > > >>> > > > >>> > _______________________________________________ > > > >>> > erlang-questions mailing list > > > >>> > erlang-questions@REDACTED > > > >>> > http://erlang.org/mailman/listinfo/erlang-questions > > > >>> > > > >>> > > > >>> -- > > > >>> > > > >>> / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > >>> _______________________________________________ > > > >>> erlang-questions mailing list > > > >>> erlang-questions@REDACTED > > > >>> http://erlang.org/mailman/listinfo/erlang-questions > > > >>> > > > >> > > > >> > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbj@REDACTED Wed Mar 15 11:06:21 2017 From: mbj@REDACTED (Martin Bjorklund) Date: Wed, 15 Mar 2017 11:06:21 +0100 (CET) Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: References: <20170313151130.GA86931@erix.ericsson.se> <20170314145309.GA70006@erix.ericsson.se> Message-ID: <20170315.110621.2242307400943831962.mbj@tail-f.com> Hi, fxmy wang wrote: > 2017-03-14 22:53 GMT+08:00 Raimo Niskanen < > raimo+erlang-questions@REDACTED>: > > > I have now tried with the 'smilint' tool and it does not find any errors, > > so this is probably a legitimate way to write a MIB. > > > > If so our MIB compiler needs to be rewritten to do two passes; one handling > > anything not augmented, and one handling augmentation. We have a fix for this bug in our (forked) snmp implementation. A patch is included if you're interested. The idea is that the definitions are sorted so that any augment ends up after the augment target; then the processing can be left as-is. Unfortunately the code is not 100% correct; specifically it puts the definition after the MODULE-COMPLIANCE so the compiler later complains that the compliance reference is invalid (which is not correct (but it works for us)). In order to be really correct, the code needs to be completely order-independent... /martin -------------- next part -------------- A non-text attachment was scrubbed... Name: snmpc.erl.patch Type: text/x-patch Size: 3790 bytes Desc: not available URL: From fxmywc@REDACTED Wed Mar 15 12:49:11 2017 From: fxmywc@REDACTED (fxmy wang) Date: Wed, 15 Mar 2017 19:49:11 +0800 Subject: [erlang-questions] snmpc: Compiling LLDP-MIB failled In-Reply-To: <20170315.110621.2242307400943831962.mbj@tail-f.com> References: <20170313151130.GA86931@erix.ericsson.se> <20170314145309.GA70006@erix.ericsson.se> <20170315.110621.2242307400943831962.mbj@tail-f.com> Message-ID: That was indeed fast Martin. =? For now I could live with restructuring LLDP-MIB though. And bug reported here: ERL-375 . Cheers, ? 2017-03-15 18:06 GMT+08:00 Martin Bjorklund : > Hi, > > fxmy wang wrote: > > 2017-03-14 22:53 GMT+08:00 Raimo Niskanen < > > raimo+erlang-questions@REDACTED>: > > > > > I have now tried with the 'smilint' tool and it does not find any > errors, > > > so this is probably a legitimate way to write a MIB. > > > > > > If so our MIB compiler needs to be rewritten to do two passes; one > handling > > > anything not augmented, and one handling augmentation. > > We have a fix for this bug in our (forked) snmp implementation. A > patch is included if you're interested. The idea is that the > definitions are sorted so that any augment ends up after the augment > target; then the processing can be left as-is. > > Unfortunately the code is not 100% correct; specifically it puts the > definition after the MODULE-COMPLIANCE so the compiler later complains > that the compliance reference is invalid (which is not correct (but it > works for us)). > > In order to be really correct, the code needs to be completely > order-independent... > > > /martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ranjanabhiishek@REDACTED Wed Mar 15 12:37:52 2017 From: ranjanabhiishek@REDACTED (abhishek ranjan) Date: Wed, 15 Mar 2017 17:07:52 +0530 Subject: [erlang-questions] Selecting server for erlang application In-Reply-To: References: Message-ID: I have written erlang code for a game application but don't know about which server to choose for its deployment and how to write the front end of the game so that I can provide any user with an opportunity to play my game from anywhere in the world. Does anyone have any Ideas on this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Wed Mar 15 16:07:13 2017 From: kenneth@REDACTED (Kenneth Lundin) Date: Wed, 15 Mar 2017 16:07:13 +0100 Subject: [erlang-questions] Erlang/OTP 19.3 has been released Message-ID: Erlang/OTP 19.3 is the third service release for the 19 major release. The service release contains mostly bug fixes and characteristics improvements but also some new features. Some highlights for 19.3 - *crypto, ssh*: The implementation of the key exchange algorithms diffie-hellman-group-exchange-sha* are optimized, up to a factor of 11 for the slowest ( = biggest and safest) group size. - *dialyzer*: The peak memory consumption is reduced. Analyzing modules with binary construction with huge strings is now much faster. - *erts*: A received SIGTERM signal to beam will generate a 'stop' message to the init process and terminate the Erlang VM nicely. This is equivalent to calling init:stop/0. - *kernel*: The functions in the file module that take a list of paths (e.g. file:path_consult/2) will now continue to search in the path if the path contains something that is not a directory. - *kernel*: Two OTP processes that are known to receive many messages are rex (used by rpc) and error_logger. Those processes will now store unprocessed messages outside the process heap, which will potentially decrease the cost of garbage collections. - *public_key*: New function pkix_verify_hostname/2,3 implements certificate hostname checking. See the manual and RFC 6125. - *public_key, ssh*: The ssh host key fingerprint generation now also takes a list of algorithms and returns a list of corresponding fingerprints. See public_key:ssh_hostkey_fingerprint/2 and the option silently_accept_hosts in ssh:connect. - *ssl*: Move PEM cache to a dedicated process, to avoid making the SSL manager process a bottleneck. This improves scalability of TLS connections. - *stdlib*: filename:safe_relative_path/1 to sanitize a relative path has been added. - Thanks to more than 20 different contributors You can find the README and the full listing of changes for this service release at http://www.erlang.org/download/otp_src_19.3.readme The source distribution and binary distributions for Windows can be downloaded from http://www.erlang.org/download/otp_src_19.3.tar.gz http://www.erlang.org/download/otp_win32_19.3.exe http://www.erlang.org/download/otp_win64_19.3.exe Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please consult the README file that is part of the distribution. The Erlang/OTP source can also be found at GitHub on the official Erlang repository, https://github.com/erlang/otp with tag OTP-19.3 The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_19.3.tar.gz http://www.erlang.org/download/otp_doc_man_19.3.tar.gz Please report any new issues via Erlang/OTPs public issue tracker https://bugs.erlang.org We want to thank all of those who sent us patches, suggestions and bug reports! Thank you! The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed Mar 15 22:48:33 2017 From: matthias@REDACTED (Matthias Lang) Date: Wed, 15 Mar 2017 22:48:33 +0100 Subject: [erlang-questions] has anyone else seen Erlang 19 break 'bash' after exit? In-Reply-To: References: <20170313203230.m23ch7vrd3tutmv4@corelatus.se> Message-ID: <20170315214833.w5ebnxdd7zrvvfgx@corelatus.se> Hi, With help from my colleague Thomas, I chased the problem I'm seeing. I am fairly sure that - It's a bug in Bash. Already been reported. - Affects all versions of Bash from 4.4.0 onwards (current tip 4.4.12) - Doesn't affect Bash 4.3.x I bisected Bash to find the offending commit. On 'master', it's a0c0a00fc4, but that commit includes 500 changed files and about 200k insertions + deletions. I could chase some more (there are more branches and tags), but it seems pointless given that there's a bug report: http://lists.gnu.org/archive/html/bug-bash/2017-01/msg00039.html There's a patch and a "don't hold your breath" comment a few posts later. Thanks for the confirmations and comments. Matt ---------------------------------------------------------------------- On 14. March 2017, Lukas Larsson wrote: > Hello, > > I think that it may be a change done in Erlang/OTP 18 that causes this > behaviour. Specifically OTP-12239. Using non-blocking I/O on stdin/stdout > seems to kind of work, until it doesn't. Different platforms and tools > handle it very differently. > > The only "solution" that I can think of right now is to use dedicated read > and/or write threads within the VM to handle stdin/stdout, but I was hoping > to avoid having to do that. > > The problem, for those that are interested, is that the O_NONBLOCK flag is > set on the file description, not the file descriptor. Which causes all > kinds of problems, I thought I had handled them all, but apparently not > enough. > > Lukas > > On Tue, Mar 14, 2017 at 1:35 PM, Igor Clark wrote: > > > Hello Matthias, > > > > Not sure if it's related, but I see bash v4.4.5 terminal I/O get really > > messed up on Mac OSX after the erlang runtime aborts. It happens whether > > through a NIF segfault during dev, or just by doing ^C and 'abort'. The > > terminal input becomes unusable, vim buffers don't display roughly half the > > text; 'reset' makes no difference, I have to close the Terminal.app window > > and start over. Haven't had time to dig into it at all, just kind of got > > used to it. > > > > I thought it might be R19, but I switched back to R18 and it still > > happened. Which made me wonder if it's anything to do with tmux, but I > > tried it in a standard Terminal window without tmux, and it happened then > > too. So I tried using the default OSX /bin/bash (which is v3.2.57) instead > > of the homebrew version (4.4.5) - and it didn't happen at all. > > > > Which is a long-winded way of saying, maybe this might also be a "recent > > bash" issue, rather than an erlang issue? > > > > Cheers, > > Igor > > > > > > On 13/03/2017 20:32, Matthias Lang wrote: > > > >> Hi, > >> > >> Erlang 19.1 and 19.2 seems to break something to do with terminal > >> input on some but not all of my Debian x86-64 machines. I took a quick > >> look through the recent mailing list archives and haven't seen any > >> mention of this. Maybe I missed it. Maybe it's just me. > >> > >> Example: > >> > >> | ~ >uname -a > >> | Linux hec 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 (2017-02-27) x86_64 > >> GNU/Linux > >> | ~ >read x > >> | hello world > >> | ~ >erl > >> | Erlang/OTP 19 [erts-8.1] [source] [64-bit] [smp:4:4] > >> [async-threads:10] [hipe] [kernel-poll:false] > >> | > >> | Eshell V8.1 (abort with ^G) > >> | 1> > >> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > >> | (v)ersion (k)ill (D)b-tables (d)istribution > >> | a > >> | ~ >read x > >> | bash: read: read error: 0: Resource temporarily unavailable > >> > >> The "read error: 0: Resource temporarily unavailable" part is unexpected. > >> Using 'cat' instead of 'read' produces a similar error: > >> > >> | cat: -: Resource temporarily unavailable > >> > >> 'strace' tells me something is screwy about stdin: > >> > >> | read(0, 0x7f1010dbb000, 131072) = -1 EAGAIN (Resource > >> temporarily unavailable) > >> > >> read(2) tells me EAGAIN suggests that stdin has been left in with > >> O_NONBLOCK. > >> Is that Erlang's problem to clean up, or is that supposed to happen on > >> program exit? I haven't read up on that. > >> > >> Doing the same thing on 17.4 works fine. > >> > >> Debian 3.16.39-1 and erts-8.2.1 produces a related by different > >> strangeness: > >> > >> | 1> > >> | BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > >> | (v)ersion (k)ill (D)b-tables (d)istribution > >> | a > >> | $ sh: turning off NDELAY mode > >> > >> Can anyone else see this? Any idea what's up? > >> > >> Matt > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From vances@REDACTED Thu Mar 16 05:43:05 2017 From: vances@REDACTED (Vance Shipley) Date: Wed, 15 Mar 2017 22:43:05 -0600 Subject: [erlang-questions] [ANN] SigScale OCS In-Reply-To: References: Message-ID: The SigScale OCS project is now on GitHub. http://github.com/sigscale/ocs This online charging system (OCS) builds on our RADIUS server adding Extensible Authentication Protocol (EAP) methods PWD and TTLS and accounting with real-time credit management, REST API and material design UI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From botanyzh@REDACTED Thu Mar 16 02:43:25 2017 From: botanyzh@REDACTED (=?gb2312?B?1dQguro=?=) Date: Thu, 16 Mar 2017 01:43:25 +0000 Subject: [erlang-questions] How to use ecdh self-signed cert files in R19 ssl app Message-ID: Hi ! With so many search In vain, I can?t find the actual example through google , and I failed again and again with many procedures Only the ??. -nodes rsa:1024?. ? somelike self-sign set of cert files can be use in otp?s ssl node to node communication ??? Windows 10 ????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Thu Mar 16 16:33:02 2017 From: fxn@REDACTED (Xavier Noria) Date: Thu, 16 Mar 2017 16:33:02 +0100 Subject: [erlang-questions] origin of handle_info/2 Message-ID: A curiosity, Where does the `handle_info` callback name come from? The name does not suggest what the method is invoked for to me. I wonder if there is a historical background behind that name, or perhaps "info" reflects usage in a way that I fail to see. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Mar 16 17:32:49 2017 From: rvirding@REDACTED (Robert Virding) Date: Thu, 16 Mar 2017 17:32:49 +0100 Subject: [erlang-questions] Core erlang definition Message-ID: Does there exist a current definition ofCore Erlang? I know you can look in cerl.erl to see which constructions exist but there is a lot which is unclear on how they are to be used. For example one problem I had was HOW to represent literals. They can either be done by having them literally (haha) inside a #c_lit{} record or as a whole nested tree of Core records. I noticed that even if these are in principle equivalent sometimes later passes of the compile required one or the other to work. I got it working in the LFE compiler but it feels a bit risky not to be certain why it works, and what might happen if I change things. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 16 22:59:34 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Mar 2017 22:59:34 +0100 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: The Core Erlang home page is still up: https://www.it.uu.se/research/group/hipe/cerl/ Caveat: the spec has not been updated with the changes for bitstrings and maps. Would be nice to do that some day... The optimal way to represent literals tends to vary between passes, so there's never a clear-cut choice. If you use the cerl.erl module, you'll find some utility functions like fold_literal/1 and unfold_literal/1, is_literal_term/1, etc., that can take care of the details. For example, cons_hd/1 and cons_tl/1 will give you the head subtree even if the argument is a literal list. /Richard 2017-03-16 17:32 GMT+01:00 Robert Virding : > Does there exist a current definition ofCore Erlang? I know you can look > in cerl.erl to see which constructions exist but there is a lot which is > unclear on how they are to be used. For example one problem I had was HOW > to represent literals. They can either be done by having them literally > (haha) inside a #c_lit{} record or as a whole nested tree of Core records. > I noticed that even if these are in principle equivalent sometimes later > passes of the compile required one or the other to work. > > I got it working in the LFE compiler but it feels a bit risky not to be > certain why it works, and what might happen if I change things. > > Robert > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 16 23:10:59 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 16 Mar 2017 23:10:59 +0100 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: Message-ID: It's "info" in the sense of "any other messages to this process, that are not recognised as special OTP framework messages". When you use a function like gen_server:call(...), the OTP libraries wrap your message in a way that lets the receiving server process see that it is a part of the OTP framework and redirects it to the standard callbacks like handle_call() or handle_cast(). If you just send a message to the gen_server process with the ! operator, it will not have the right wrapper, and will be dispatched to handle_info(). Typical uses of info messages are timeouts and other "note to self" style messages. /Richard 2017-03-16 16:33 GMT+01:00 Xavier Noria : > A curiosity, > > Where does the `handle_info` callback name come from? The name does not > suggest what the method is invoked for to me. > > I wonder if there is a historical background behind that name, or perhaps > "info" reflects usage in a way that I fail to see. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Thu Mar 16 23:18:56 2017 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Thu, 16 Mar 2017 22:18:56 +0000 Subject: [erlang-questions] Client to concurrently receive and sending data on TCP socket Message-ID: Hi I am looking into building an client application which connects to an existing server via an TCP socket. When the application has established the socket it must keep it open, and receive and send requests (concurrently) around 2000 Req/s. What is the best design for such an application? Like: - active or passive socket ? - gen_server ? - how many processes ? - supervison tree ? I have found examples of server and clients, but not an client which concurrently sends and receives data on the socket, does there exist anything similar applications i can look at ? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Thu Mar 16 23:57:22 2017 From: fxn@REDACTED (Xavier Noria) Date: Thu, 16 Mar 2017 22:57:22 +0000 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: Message-ID: On Thu, 16 Mar 2017 at 23:11, Richard Carlsson wrote: It's "info" in the sense of "any other messages to this process, that are > not recognised as special OTP framework messages". When you use a function > like gen_server:call(...), the OTP libraries wrap your message in a way > that lets the receiving server process see that it is a part of the OTP > framework and redirects it to the standard callbacks like handle_call() or > handle_cast(). If you just send a message to the gen_server process with > the ! operator, it will not have the right wrapper, and will be dispatched > to handle_info(). Typical uses of info messages are timeouts and other > "note to self" style messages. > Yes, that is the way it works, but "info" doesn't convey that meaning to me. Does it to you? I wondered if maybe historically it had a smaller contract where "info" was a natural choice and with time the contract was relaxed up to accepting anything but calls and casts. > -- Sent from Gmail Mobile -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Mar 17 10:58:18 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 17 Mar 2017 10:58:18 +0100 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: Message-ID: No, I think it's more to do with the fact that the original authors were not native English speakers, and thought "info" was a good enough shorthand for "any other stuff that someone sends us". /Richard 2017-03-16 23:57 GMT+01:00 Xavier Noria : > On Thu, 16 Mar 2017 at 23:11, Richard Carlsson > wrote: > > It's "info" in the sense of "any other messages to this process, that are >> not recognised as special OTP framework messages". When you use a function >> like gen_server:call(...), the OTP libraries wrap your message in a way >> that lets the receiving server process see that it is a part of the OTP >> framework and redirects it to the standard callbacks like handle_call() or >> handle_cast(). If you just send a message to the gen_server process with >> the ! operator, it will not have the right wrapper, and will be dispatched >> to handle_info(). Typical uses of info messages are timeouts and other >> "note to self" style messages. >> > > > Yes, that is the way it works, but "info" doesn't convey that meaning to > me. Does it to you? > > I wondered if maybe historically it had a smaller contract where "info" > was a natural choice and with time the contract was relaxed up to accepting > anything but calls and casts. > >> -- > Sent from Gmail Mobile > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Fri Mar 17 11:24:44 2017 From: fxn@REDACTED (Xavier Noria) Date: Fri, 17 Mar 2017 11:24:44 +0100 Subject: [erlang-questions] when is data copied? Message-ID: I would like to have a mental model for data copying in Erlang. By that I mean immutability is in my view a property of an API, but I suppose the implementation, compiler optimizations, etc., can precisely leverage that property to reuse stuff behind the scenes for performance. Would you recommend any particular article, talk, book, ..., that would help me get an idea of how that works in Erlang? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Fri Mar 17 11:31:56 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Fri, 17 Mar 2017 11:31:56 +0100 Subject: [erlang-questions] when is data copied? In-Reply-To: References: Message-ID: Data is stored in process heap most of the time. As long as data is moved INSIDE the process, it is NOT COPIED, only a pointer is passed. If the data ever leaves the process, a message for example, it will be COPIED to its new owner. If the data is moved to or from ETS, it behaves similarly, it is COPIED (ETS has own heap separate from any process). If the data is moved between a process and a port, it is COPIED (ports behave similar to processes, and own resources in a similar way) Large binaries >64 bytes are NEVER copied, instead they are reference counted and reference is shared between processes. This is why you want to GC all processes which touched a large binary periodically, otherwise it may take a long time before the binary is freed. 2017-03-17 11:24 GMT+01:00 Xavier Noria : > I would like to have a mental model for data copying in Erlang. > > By that I mean immutability is in my view a property of an API, but I > suppose the implementation, compiler optimizations, etc., can precisely > leverage that property to reuse stuff behind the scenes for performance. > > Would you recommend any particular article, talk, book, ..., that would > help me get an idea of how that works in Erlang? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Fri Mar 17 11:45:09 2017 From: fxn@REDACTED (Xavier Noria) Date: Fri, 17 Mar 2017 11:45:09 +0100 Subject: [erlang-questions] when is data copied? In-Reply-To: References: Message-ID: Awesome, thanks! What about pattern matching? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Fri Mar 17 11:55:14 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Fri, 17 Mar 2017 11:55:14 +0100 Subject: [erlang-questions] when is data copied? In-Reply-To: References: Message-ID: Pattern matching is broken into a decision tree during compile time, and this tree is evaluated while doing jumps to corresponding clauses. It is normal Erlang code under the hood, which manipulates Erlang values normally withing current process, so they are handled efficiently. There's nothing you can optimize in regards to the data copying. To see how bad is the code you can do benchmarks yourself using timer:tc, thats the simplest. You can compile to intermediate code such as Kernel erlang (shell command c(mymodule, [to_kernel]). ) and read the produced code, or disassemble the module (shell command erts_debug:df(module). ) 2017-03-17 11:45 GMT+01:00 Xavier Noria : > Awesome, thanks! > > What about pattern matching? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Fri Mar 17 12:01:48 2017 From: fxn@REDACTED (Xavier Noria) Date: Fri, 17 Mar 2017 12:01:48 +0100 Subject: [erlang-questions] when is data copied? In-Reply-To: References: Message-ID: Gotcha, those guidelines are going to be helpful (and they are simple, which is nice!). Thanks Dmytro. -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Fri Mar 17 12:36:20 2017 From: per@REDACTED (Per Hedeland) Date: Fri, 17 Mar 2017 12:36:20 +0100 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: Message-ID: <58CBCA34.1060409@hedeland.org> On 2017-03-17 10:58, Richard Carlsson wrote: > No, I think it's more to do with the fact that the original authors were not native English speakers, and thought "info" was a good enough shorthand for "any other stuff that someone sends us". Well, some of the original authors were (are) native English speakers:-) - but in any case, what's wrong with "info"? Some process, or the VM, sent us a message - seems like a reasonable assumption that there is some relevant information in it, but that's pretty much the only assumption that can be made. Maybe the OP has a term in mind, that would be better at conveying the meaning "any other stuff that someone sends us"? --Per > /Richard > > 2017-03-16 23:57 GMT+01:00 Xavier Noria >: > > On Thu, 16 Mar 2017 at 23:11, Richard Carlsson > wrote: > > It's "info" in the sense of "any other messages to this process, that are not recognised as special OTP framework messages". When you use a function like gen_server:call(...), the OTP > libraries wrap your message in a way that lets the receiving server process see that it is a part of the OTP framework and redirects it to the standard callbacks like handle_call() or > handle_cast(). If you just send a message to the gen_server process with the ! operator, it will not have the right wrapper, and will be dispatched to handle_info(). Typical uses of info > messages are timeouts and other "note to self" style messages. > > > > Yes, that is the way it works, but "info" doesn't convey that meaning to me. Does it to you? > > I wondered if maybe historically it had a smaller contract where "info" was a natural choice and with time the contract was relaxed up to accepting anything but calls and casts. > > -- > Sent from Gmail Mobile > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From michal@REDACTED Fri Mar 17 12:44:23 2017 From: michal@REDACTED (=?utf-8?Q?Micha=C5=82_Muska=C5=82a?=) Date: Fri, 17 Mar 2017 12:44:23 +0100 Subject: [erlang-questions] when is data copied? In-Reply-To: References: Message-ID: <419ba1f4-b67c-4e2f-a39b-c0253f1e0ca0@Spark> I think this article might also be helpful regarding memory layout and data handling:?https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html Micha?. On 17 Mar 2017, 12:02 +0100, Xavier Noria , wrote: > Gotcha, those guidelines are going to be helpful (and they are simple, which is nice!). Thanks?Dmytro. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Mar 17 12:55:30 2017 From: tony@REDACTED (Tony Rogvall) Date: Fri, 17 Mar 2017 12:55:30 +0100 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: <58CBCA34.1060409@hedeland.org> References: <58CBCA34.1060409@hedeland.org> Message-ID: <072B8006-2604-4CEF-9841-C7FC7A6CBD33@rogvall.se> I vote for handle_random_stuff :-) /Tony > On 17 mar 2017, at 12:36, Per Hedeland wrote: > > On 2017-03-17 10:58, Richard Carlsson wrote: >> No, I think it's more to do with the fact that the original authors were not native English speakers, and thought "info" was a good enough shorthand for "any other stuff that someone sends us". > > Well, some of the original authors were (are) native English speakers:-) > - but in any case, what's wrong with "info"? Some process, or the VM, > sent us a message - seems like a reasonable assumption that there is > some relevant information in it, but that's pretty much the only > assumption that can be made. > > Maybe the OP has a term in mind, that would be better at conveying the > meaning "any other stuff that someone sends us"? > > --Per > >> /Richard >> >> 2017-03-16 23:57 GMT+01:00 Xavier Noria >: >> >> On Thu, 16 Mar 2017 at 23:11, Richard Carlsson > wrote: >> >> It's "info" in the sense of "any other messages to this process, that are not recognised as special OTP framework messages". When you use a function like gen_server:call(...), the OTP >> libraries wrap your message in a way that lets the receiving server process see that it is a part of the OTP framework and redirects it to the standard callbacks like handle_call() or >> handle_cast(). If you just send a message to the gen_server process with the ! operator, it will not have the right wrapper, and will be dispatched to handle_info(). Typical uses of info >> messages are timeouts and other "note to self" style messages. >> >> >> >> Yes, that is the way it works, but "info" doesn't convey that meaning to me. Does it to you? >> >> I wondered if maybe historically it had a smaller contract where "info" was a natural choice and with time the contract was relaxed up to accepting anything but calls and casts. >> >> -- >> Sent from Gmail Mobile >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From alex0player@REDACTED Fri Mar 17 12:57:07 2017 From: alex0player@REDACTED (Alex S.) Date: Fri, 17 Mar 2017 14:57:07 +0300 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: <58CBCA34.1060409@hedeland.org> References: <58CBCA34.1060409@hedeland.org> Message-ID: <3FB23106-9017-468E-AA36-B261EEB94A52@gmail.com> > 17 ????? 2017 ?., ? 14:36, Per Hedeland ???????(?): > > On 2017-03-17 10:58, Richard Carlsson wrote: >> No, I think it's more to do with the fact that the original authors were not native English speakers, and thought "info" was a good enough shorthand for "any other stuff that someone sends us". > > Well, some of the original authors were (are) native English speakers:-) > - but in any case, what's wrong with "info"? Some process, or the VM, > sent us a message - seems like a reasonable assumption that there is > some relevant information in it, but that's pretty much the only > assumption that can be made. > > Maybe the OP has a term in mind, that would be better at conveying the > meaning "any other stuff that someone sends us"? > > --Per > Well, it *is* most often used for timeouts and DOWN?s which are kind of informational. From fxn@REDACTED Fri Mar 17 12:59:26 2017 From: fxn@REDACTED (Xavier Noria) Date: Fri, 17 Mar 2017 12:59:26 +0100 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: <58CBCA34.1060409@hedeland.org> References: <58CBCA34.1060409@hedeland.org> Message-ID: On Fri, Mar 17, 2017 at 12:36 PM, Per Hedeland wrote: Well, some of the original authors were (are) native English speakers:-) > - but in any case, what's wrong with "info"? Some process, or the VM, > sent us a message - seems like a reasonable assumption that there is > some relevant information in it, but that's pretty much the only > assumption that can be made. > But that is what _any_ function does, right? You get a message, that contains some data. You could have called the callback "do_stuff" for that matter! Maybe the OP has a term in mind, that would be better at conveying the > meaning "any other stuff that someone sends us"? Yes, I would expect a name more in the line of: handle_rest handle_internal handle_timeout (split interface for the timeout use-case) ... Something that resonates closer to "this is called for any other message than a call or cast". Hey, not bikeshedding the callback name at all uh? but was wondering if it could have had a historic path from a narrower use-case that then widened with time. Or maybe some rationale for the name that I could be missing. If there is no such historic background... that is fine, curiosity solved :). -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Mar 17 13:20:32 2017 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 17 Mar 2017 12:20:32 +0000 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: <58CBCA34.1060409@hedeland.org> References: <58CBCA34.1060409@hedeland.org> Message-ID: On 17 March 2017 at 11:36, Per Hedeland wrote: > Maybe the OP has a term in mind, that would be better at conveying the > meaning "any other stuff that someone sends us"? As an also-native English speaker, I vote for "handle_gubbins" :) From bjorn@REDACTED Fri Mar 17 15:52:15 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 17 Mar 2017 15:52:15 +0100 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: On Thu, Mar 16, 2017 at 10:59 PM, Richard Carlsson wrote: > The optimal way to represent literals tends to vary between passes, so > there's never a clear-cut choice. If you use the cerl.erl module, you'll > find some utility functions like fold_literal/1 and unfold_literal/1, > is_literal_term/1, etc., that can take care of the details. For example, > cons_hd/1 and cons_tl/1 will give you the head subtree even if the argument > is a literal list. It is true that literals may be differently represented within different passes, but some passes (sys_core_fold, v3_kernel) make implicit assumptions that a literal is represented as #c_literal{}. Breaking those assumptions can lead to sub-optimal code or possibly even compiler crashes. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From donpedrothird@REDACTED Fri Mar 17 17:58:52 2017 From: donpedrothird@REDACTED (John Doe) Date: Fri, 17 Mar 2017 19:58:52 +0300 Subject: [erlang-questions] Client to concurrently receive and sending data on TCP socket In-Reply-To: References: Message-ID: This https://github.com/brigadier/tara/blob/master/src/tara_worker.erl client concurrently sends and receives data. The protocol is supposed to be standard request -> reply, but replies are send asynchronously (though with little to no delay) with the IDs of the request. It uses gen_server with async handle_call which returns {noreply, Reason} If I misunderstood what you mean by 'concurrently', then might be you just need a pool of such clients. 2017-03-17 1:18 GMT+03:00 Thomas Elsgaard : > Hi > > I am looking into building an client application which connects to an > existing server via an TCP socket. When the application has established the > socket it must keep it open, and receive and send requests (concurrently) > around 2000 Req/s. > > What is the best design for such an application? Like: > > - active or passive socket ? > - gen_server ? > - how many processes ? > - supervison tree ? > > I have found examples of server and clients, but not an client which > concurrently sends and receives data on the socket, does there exist > anything similar applications i can look at ? > > Thomas > > > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From powers_brady@REDACTED Fri Mar 17 18:42:12 2017 From: powers_brady@REDACTED (Brady Powers) Date: Fri, 17 Mar 2017 17:42:12 +0000 (UTC) Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: <58CBCA34.1060409@hedeland.org> Message-ID: <919070595.2356478.1489772532466@mail.yahoo.com> "handle_other" "handle_all" "handle_else" The above, in my opinion as a native english speaker, portray the intent of "handle_info" better than the actual name. Now I'm not very good at naming, so my suggestions could be equally confusing to others, but I find it hard to support "handle_info" as a good name.? On Friday, March 17, 2017 8:20 AM, Roger Lipscombe wrote: On 17 March 2017 at 11:36, Per Hedeland wrote: > Maybe the OP has a term in mind, that would be better at conveying the > meaning "any other stuff that someone sends us"? As an also-native English speaker, I vote for "handle_gubbins" :) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri Mar 17 20:31:53 2017 From: rvirding@REDACTED (Robert Virding) Date: Fri, 17 Mar 2017 20:31:53 +0100 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: So this means that when you are representing data structures/patterns they should be "as literal as possible"? And only use explicit structures, for example #c_cons{}/#c_tuple{}, when they have elements which aren't literal values? Robert On 17 March 2017 at 15:52, Bj?rn Gustavsson wrote: > On Thu, Mar 16, 2017 at 10:59 PM, Richard Carlsson > wrote: > > > The optimal way to represent literals tends to vary between passes, so > > there's never a clear-cut choice. If you use the cerl.erl module, you'll > > find some utility functions like fold_literal/1 and unfold_literal/1, > > is_literal_term/1, etc., that can take care of the details. For example, > > cons_hd/1 and cons_tl/1 will give you the head subtree even if the > argument > > is a literal list. > > It is true that literals may be differently represented > within different passes, but some passes (sys_core_fold, > v3_kernel) make implicit assumptions that a literal > is represented as #c_literal{}. Breaking those assumptions > can lead to sub-optimal code or possibly even compiler > crashes. > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Sat Mar 18 07:03:27 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 18 Mar 2017 07:03:27 +0100 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: On Fri, Mar 17, 2017 at 8:31 PM, Robert Virding wrote: > So this means that when you are representing data structures/patterns they > should be "as literal as possible"? And only use explicit structures, for > example #c_cons{}/#c_tuple{}, when they have elements which aren't literal > values? Yes, that's how v3_core represents literals when translating to Core Erlang. That said, I just come to think about some changes we have made in sys_core_fold in the master branch (to be OTP 20). Starting from OTP 20, sys_core_fold will do a fix point iteration on each function, that is, perform all transformations again and again until there is no further change (or until the limit for the max number of iterations is reached). That probably means that it does not matter as much how literals are represented. If the literals are not "as literal as possible", the first round through sys_core_fold will make them "as literal as possible". Subsequent rounds can then do the appropriate optimizations. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From dkuhlman@REDACTED Fri Mar 17 22:21:58 2017 From: dkuhlman@REDACTED (Dave Kuhlman) Date: Fri, 17 Mar 2017 14:21:58 -0700 Subject: [erlang-questions] Client to concurrently receive and sending data on TCP socket In-Reply-To: References: Message-ID: <20170317212158.GA29918@box280.bluehost.com> It sounds like chapter 3 of the book "Erlang and OTP in action", by Martin Logan et al, would be helpful to you. The book is very worthwhile, and you can also find the source code for the solutions to the problems discuss in the book (including "TCP-based RPC server" from chapter 3) at github: https://github.com/erlware/Erlang-and-OTP-in-Action-Source Here is a bit of summary of the chapter: "What you're creating -- The RPC server will allow you to listen on a TCP socket and accept a single connection from an outside TCP client. After it's connected, it will let a client run functions via a simple ASCII text protocol over TCP. ..." Dave On Thu, Mar 16, 2017 at 10:18:56PM +0000, Thomas Elsgaard wrote: > Hi > I am looking into building an client application which connects to an > existing server via an TCP socket. When the application has established > the socket it must keep it open, and ??receive and send requests > (concurrently) around 2000 Req/s.?? > What is the best design for such an application? Like: > - active or passive socket ? > - gen_server ? > - how many processes ? > - supervison tree ? > I have found examples of server and clients, but not an client which > concurrently sends and receives data on the socket, does there exist > anything similar applications i can look at ? > Thomas > ?? > ?? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Dave Kuhlman http://www.davekuhlman.org From eric.pailleau@REDACTED Sat Mar 18 14:07:59 2017 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 18 Mar 2017 14:07:59 +0100 Subject: [erlang-questions] [ANN] geas 2.0.11 Message-ID: <8f94f6ab-4fd3-b1e0-38c3-90d2d2d1cc1c@wanadoo.fr> Hi, Geas 2.0.11 has been released. This is an update for Erlang 19.3 database and a bug fix. Geas is a tool that will detect the runnable official Erlang release window for your project, including dependancies, either from source code or beam files. Geas will tell you what are the offending functions in the beam/source files that reduce the available window. Geas will tell you if some beam files are compiled native. Geas is available as a module, erlang.mk and rebar 2/3 plugins. https://github.com/crownedgrouse/geas Cheers ! Eric From dch@REDACTED Sat Mar 18 16:09:09 2017 From: dch@REDACTED (Dave Cottlehuber) Date: Sat, 18 Mar 2017 16:09:09 +0100 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: <903a4531-1b43-a1bb-7cc1-a19969a0d900@ninenines.eu> Message-ID: <1489849749.577647.915537472.14EA9C28@webmail.messagingengine.com> > On Mon, Mar 13, 2017 at 3:53 PM, Max Lapshin > wrote: > > > I launch via erlang:open_port and struggle a lot trying to make children > > stop when erlang server dies: it is not easy. https://github.com/saleyn/erlexec A+ Dave From raghav.karol@REDACTED Mon Mar 20 14:26:11 2017 From: raghav.karol@REDACTED (Raghav Karol) Date: Mon, 20 Mar 2017 14:26:11 +0100 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: Message-ID: Hi Frank, It might be worthwhile to look into Cowboy + RabbitMQ RPC. I have seen this pattern used in production, (<10 ms latency in AWS). Cowboy handles web requests and makes AMQP RPC calls to a PHP (Python) service. You get all the benefits (and drawbacks) of having a queue of work, e.g., being able to start multiple workers dynamically to scale up and down. -- Raghav On Mon, Mar 13, 2017 at 6:09 PM, Frank Muller wrote: > Hi guys > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > I would like to receive HTTP requests in Cowboy and forward them to a > PHP/Python backend. > > Get the response back from the backend and forward it to the client. > > Thanks in advance. > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Raghav -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Mon Mar 20 16:52:30 2017 From: dmercer@REDACTED (David Mercer) Date: Mon, 20 Mar 2017 10:52:30 -0500 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: <58CBCA34.1060409@hedeland.org> Message-ID: Native English speaker here, and while I think `handle_info` is just fine, I think `handle_message_not_handled_by_handle_call_or_handle_cast` would be more descriptive. On Fri, Mar 17, 2017 at 7:20 AM, Roger Lipscombe wrote: > On 17 March 2017 at 11:36, Per Hedeland wrote: > > Maybe the OP has a term in mind, that would be better at conveying the > > meaning "any other stuff that someone sends us"? > > As an also-native English speaker, I vote for "handle_gubbins" :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Mar 20 16:54:17 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 20 Mar 2017 15:54:17 +0000 Subject: [erlang-questions] Serve PHP/Python scripts behind Erlang In-Reply-To: References: Message-ID: Thank you guys for all these links. Still studying them. /Frank wrote : > Hi Frank, > > It might be worthwhile to look into Cowboy + RabbitMQ RPC. I have seen > this pattern used in production, (<10 ms latency in AWS). > > Cowboy handles web requests and makes AMQP RPC calls to a PHP (Python) > service. You get all the benefits (and drawbacks) of having a queue of > work, e.g., being able to start multiple workers dynamically to scale up > and down. > > -- > Raghav > > On Mon, Mar 13, 2017 at 6:09 PM, Frank Muller > wrote: > > Hi guys > > What?s the most efficient way to run PHP (Python) scripts behind Erlang? > > I would like to receive HTTP requests in Cowboy and forward them to a > PHP/Python backend. > > Get the response back from the backend and forward it to the client. > > Thanks in advance. > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Raghav > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex0player@REDACTED Mon Mar 20 16:54:50 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 20 Mar 2017 18:54:50 +0300 Subject: [erlang-questions] origin of handle_info/2 In-Reply-To: References: <58CBCA34.1060409@hedeland.org> Message-ID: <915F8B9B-130D-4413-8A5A-FC02E5BAAF1B@gmail.com> Hi, non-native English speaker here, and `handle_plain` | `handle_other` wouldve been more sensible I guess?.. Way too late to break compatibility tho. > 20 ????? 2017 ?., ? 18:52, David Mercer ???????(?): > > Native English speaker here, and while I think `handle_info` is just fine, I think `handle_message_not_handled_by_handle_call_or_handle_cast` would be more descriptive. > > On Fri, Mar 17, 2017 at 7:20 AM, Roger Lipscombe > wrote: > On 17 March 2017 at 11:36, Per Hedeland > wrote: > > Maybe the OP has a term in mind, that would be better at conveying the > > meaning "any other stuff that someone sends us"? > > As an also-native English speaker, I vote for "handle_gubbins" :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Tue Mar 21 05:10:43 2017 From: vans_163@REDACTED (Vans S) Date: Tue, 21 Mar 2017 04:10:43 +0000 (UTC) Subject: [erlang-questions] enif_send, how to call off a smp thread? References: <144675736.45252.1490069443146.ref@mail.yahoo.com> Message-ID: <144675736.45252.1490069443146@mail.yahoo.com> I am having a weird issue with enif_send. What is happening is I am interoping with a 3rd party C library which has its own event loop and procs callbacks from its own thread. When the callback procs (in the C NIF) I want to call enif_send to get the message back to an erlang pid. When I call enif_send the first term I pass it works, then every subsequent message gets dropped unless its exactly the same as the first time passed. For example static ErlNifPid my_temp_pid; static void on_callback(void* user_data) { ErlNifEnv* msg_env = enif_alloc_env(); enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); enif_free_env(msg_env);} *on_callback proced 6> receive X-> X after 1 -> none end. okk 7> receive X-> X after 1 -> none end.none *on_callback proced 8> receive X-> X after 1 -> none end. okk 9> receive X-> X after 1 -> none end. none *on_callback proced *on_callback proced 10> receive X-> X after 1 -> none end. okk 11> receive X-> X after 1 -> none end. okk 12> receive X-> X after 1 -> none end. none What is the correct way to do this? Thanks. From sverker.eriksson@REDACTED Tue Mar 21 11:51:37 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 21 Mar 2017 11:51:37 +0100 Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: <144675736.45252.1490069443146@mail.yahoo.com> References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> Message-ID: enif_send docs says: "The message environment msg_env with all its terms (including msg) is invalidated by a successful call to enif_send. The environment is to either be freed with enif_free_env of cleared for reuse with enif_clear_env." which means you have to call enif_clear_env before reusing msg_env in the second call to enif_send. Or if you're running OTP 19 you can pass msg_env as NULL, in which case the message term is copied and its environment left intact by enif_send. /Sverker, Erlang/OTP On 03/21/2017 05:10 AM, Vans S wrote: > I am having a weird issue with enif_send. > > What is happening is I am interoping with a 3rd party C library which has > its own event loop and procs callbacks from its own thread. > > When the callback procs (in the C NIF) I want to call enif_send to get > the message back to an erlang pid. > > When I call enif_send the first term I pass it works, then every subsequent message > gets dropped unless its exactly the same as the first time passed. > > For example > > > static ErlNifPid my_temp_pid; > static void on_callback(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env);} > > *on_callback proced > 6> receive X-> X after 1 -> none end. > okk > > 7> receive X-> X after 1 -> none end.none > *on_callback proced > 8> receive X-> X after 1 -> none end. > okk > 9> receive X-> X after 1 -> none end. > none > *on_callback proced > *on_callback proced > 10> receive X-> X after 1 -> none end. > okk > 11> receive X-> X after 1 -> none end. > okk > 12> receive X-> X after 1 -> none end. > none > What is the correct way to do this? Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vans_163@REDACTED Tue Mar 21 14:57:04 2017 From: vans_163@REDACTED (Vans S) Date: Tue, 21 Mar 2017 13:57:04 +0000 (UTC) Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> Message-ID: <647032232.584180.1490104624452@mail.yahoo.com> Iv tried so many combinations of passing the default env in the user_data of the callback/setting a global or setting that to NULL and using the msg_env, same result. Even if I do it like this or that: static ErlNifPid my_temp_pid; static void on_callback(void* user_data) { ErlNifEnv* msg_env = enif_alloc_env(); enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); enif_free_env(msg_env); msg_env = enif_alloc_env();enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); enif_free_env(msg_env); } static void on_callback2(void* user_data) { ErlNifEnv* msg_env = enif_alloc_env(); enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); enif_clear_env(msg_env); enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); enif_free_env(msg_env); } Only the first msg comes, its like it caches/remembers it and ignores the rest. I wonder if its due to not being able to call enif_send from a random non SMP thread and if I need to make a queue that I poll later? By only the first msg comes, if I later do a enif_send with "okk2" atom as the first, it wont send unless the atom is exactly "okk". The same thing if you substitute the atom with an integer or binary. On Tuesday, March 21, 2017 6:52 AM, Sverker Eriksson wrote: enif_send docs says: "The message environment msg_env with all its terms (including msg) is invalidated by a successful call to enif_send. The environment is to either be freed with enif_free_env of cleared for reuse with enif_clear_env." which means you have to call enif_clear_env before reusing msg_env in the second call to enif_send. Or if you're running OTP 19 you can pass msg_env as NULL, in which case the message term is copied and its environment left intact by enif_send. /Sverker, Erlang/OTP On 03/21/2017 05:10 AM, Vans S wrote: > I am having a weird issue with enif_send. > > What is happening is I am interoping with a 3rd party C library which has > its own event loop and procs callbacks from its own thread. > > When the callback procs (in the C NIF) I want to call enif_send to get > the message back to an erlang pid. > > When I call enif_send the first term I pass it works, then every subsequent message > gets dropped unless its exactly the same as the first time passed. > > For example > > > static ErlNifPid my_temp_pid; > static void on_callback(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env);} > > *on_callback proced > 6> receive X-> X after 1 -> none end. > okk > > 7> receive X-> X after 1 -> none end.none > *on_callback proced > 8> receive X-> X after 1 -> none end. > okk > 9> receive X-> X after 1 -> none end. > none > *on_callback proced > *on_callback proced > 10> receive X-> X after 1 -> none end. > okk > 11> receive X-> X after 1 -> none end. > okk > 12> receive X-> X after 1 -> none end. > none > What is the correct way to do this? Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Tue Mar 21 15:28:15 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 21 Mar 2017 15:28:15 +0100 Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: <647032232.584180.1490104624452@mail.yahoo.com> References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> <647032232.584180.1490104624452@mail.yahoo.com> Message-ID: enif_send can be called from a non scheduler thread in an SMP enabled VM (beam.smp) but not in a non-SMP VM (beam). The outcome being dependent on the message term sounds very strange. Run with debug compiled VM to get more runtime sanity checks. $> (cd $ERL_TOP/erts/emulator && make TYPE=debug smp) $> $ERL_TOP/bin/cerl -debug /Sverker On 03/21/2017 02:57 PM, Vans S wrote: > Iv tried so many combinations of passing the default env in the user_data of the callback/setting a global or setting that to NULL and using the msg_env, same result. Even if I do it like this or that: > > static ErlNifPid my_temp_pid; > static void on_callback(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_free_env(msg_env); > > msg_env = enif_alloc_env();enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env); > } > > > static void on_callback2(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_clear_env(msg_env); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env); > } > > > Only the first msg comes, its like it caches/remembers it and ignores the rest. I wonder if its due to not being able to > call enif_send from a random non SMP thread and if I need to make a queue that I poll later? > > By only the first msg comes, if I later do a enif_send with "okk2" atom as the first, it wont send unless the atom is exactly "okk". > > The same thing if you substitute the atom with an integer or binary. > > On Tuesday, March 21, 2017 6:52 AM, Sverker Eriksson wrote: > > > > enif_send docs says: > > "The message environment msg_env with all its terms (including msg) > is invalidated by a successful call to enif_send. The environment is to > either be freed with enif_free_env of cleared for reuse with > enif_clear_env." > > which means you have to call enif_clear_env before reusing msg_env > in the second call to enif_send. > > Or if you're running OTP 19 you can pass msg_env as NULL, > in which case the message term is copied and its environment left intact > by enif_send. > > > /Sverker, Erlang/OTP > > > > On 03/21/2017 05:10 AM, Vans S wrote: >> I am having a weird issue with enif_send. >> >> What is happening is I am interoping with a 3rd party C library which has >> its own event loop and procs callbacks from its own thread. >> >> When the callback procs (in the C NIF) I want to call enif_send to get >> the message back to an erlang pid. >> >> When I call enif_send the first term I pass it works, then every subsequent message >> gets dropped unless its exactly the same as the first time passed. >> >> For example >> >> >> static ErlNifPid my_temp_pid; >> static void on_callback(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env);} >> >> *on_callback proced >> 6> receive X-> X after 1 -> none end. >> okk >> >> 7> receive X-> X after 1 -> none end.none >> *on_callback proced >> 8> receive X-> X after 1 -> none end. >> okk >> 9> receive X-> X after 1 -> none end. >> none >> *on_callback proced >> *on_callback proced >> 10> receive X-> X after 1 -> none end. >> okk >> 11> receive X-> X after 1 -> none end. >> okk >> 12> receive X-> X after 1 -> none end. >> none >> What is the correct way to do this? Thanks. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vans_163@REDACTED Tue Mar 21 15:59:15 2017 From: vans_163@REDACTED (Vans S) Date: Tue, 21 Mar 2017 14:59:15 +0000 (UTC) Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> <647032232.584180.1490104624452@mail.yahoo.com> Message-ID: <1752286703.6143797.1490108355186@mail.yahoo.com> Im going to try that, for now I made a test case that fails for me https://github.com/vans163/nif_test. The 3rd receive we get ok instead of 56. I am on Erlang 19.1.2 though, maybe I should try the R19.3 or master HEAD? On Tuesday, March 21, 2017 10:28 AM, Sverker Eriksson wrote: enif_send can be called from a non scheduler thread in an SMP enabled VM (beam.smp) but not in a non-SMP VM (beam). The outcome being dependent on the message term sounds very strange. Run with debug compiled VM to get more runtime sanity checks. $> (cd $ERL_TOP/erts/emulator && make TYPE=debug smp) $> $ERL_TOP/bin/cerl -debug /Sverker On 03/21/2017 02:57 PM, Vans S wrote: > Iv tried so many combinations of passing the default env in the user_data of the callback/setting a global or setting that to NULL and using the msg_env, same result. Even if I do it like this or that: > > static ErlNifPid my_temp_pid; > static void on_callback(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_free_env(msg_env); > > msg_env = enif_alloc_env();enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env); > } > > > static void on_callback2(void* user_data) > { > ErlNifEnv* msg_env = enif_alloc_env(); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); > enif_clear_env(msg_env); > enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); > enif_free_env(msg_env); > } > > > Only the first msg comes, its like it caches/remembers it and ignores the rest. I wonder if its due to not being able to > call enif_send from a random non SMP thread and if I need to make a queue that I poll later? > > By only the first msg comes, if I later do a enif_send with "okk2" atom as the first, it wont send unless the atom is exactly "okk". > > The same thing if you substitute the atom with an integer or binary. > > On Tuesday, March 21, 2017 6:52 AM, Sverker Eriksson wrote: > > > > enif_send docs says: > > "The message environment msg_env with all its terms (including msg) > is invalidated by a successful call to enif_send. The environment is to > either be freed with enif_free_env of cleared for reuse with > enif_clear_env." > > which means you have to call enif_clear_env before reusing msg_env > in the second call to enif_send. > > Or if you're running OTP 19 you can pass msg_env as NULL, > in which case the message term is copied and its environment left intact > by enif_send. > > > /Sverker, Erlang/OTP > > > > On 03/21/2017 05:10 AM, Vans S wrote: >> I am having a weird issue with enif_send. >> >> What is happening is I am interoping with a 3rd party C library which has >> its own event loop and procs callbacks from its own thread. >> >> When the callback procs (in the C NIF) I want to call enif_send to get >> the message back to an erlang pid. >> >> When I call enif_send the first term I pass it works, then every subsequent message >> gets dropped unless its exactly the same as the first time passed. >> >> For example >> >> >> static ErlNifPid my_temp_pid; >> static void on_callback(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env);} >> >> *on_callback proced >> 6> receive X-> X after 1 -> none end. >> okk >> >> 7> receive X-> X after 1 -> none end.none >> *on_callback proced >> 8> receive X-> X after 1 -> none end. >> okk >> 9> receive X-> X after 1 -> none end. >> none >> *on_callback proced >> *on_callback proced >> 10> receive X-> X after 1 -> none end. >> okk >> 11> receive X-> X after 1 -> none end. >> okk >> 12> receive X-> X after 1 -> none end. >> none >> What is the correct way to do this? Thanks. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Tue Mar 21 16:47:18 2017 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 21 Mar 2017 16:47:18 +0100 Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: <1752286703.6143797.1490108355186@mail.yahoo.com> References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> <647032232.584180.1490104624452@mail.yahoo.com> <1752286703.6143797.1490108355186@mail.yahoo.com> Message-ID: I think I know the answer: single assignment. Try flush() in the shell and get all messages, even those that don't match X :-) /Sverker On 03/21/2017 03:59 PM, Vans S wrote: > Im going to try that, for now I made a test case that fails for me > https://github.com/vans163/nif_test. > > The 3rd receive we get ok instead of 56. > > I am on Erlang 19.1.2 though, maybe I should try the R19.3 or master HEAD? > > > > On Tuesday, March 21, 2017 10:28 AM, Sverker Eriksson wrote: > > > > enif_send can be called from a non scheduler thread in an SMP enabled VM > (beam.smp) > but not in a non-SMP VM (beam). > > The outcome being dependent on the message term sounds very strange. > > > Run with debug compiled VM to get more runtime sanity checks. > > > $> (cd $ERL_TOP/erts/emulator && make TYPE=debug smp) > > $> $ERL_TOP/bin/cerl -debug > > > /Sverker > > > > On 03/21/2017 02:57 PM, Vans S wrote: >> Iv tried so many combinations of passing the default env in the user_data of the callback/setting a global or setting that to NULL and using the msg_env, same result. Even if I do it like this or that: >> >> static ErlNifPid my_temp_pid; >> static void on_callback(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_free_env(msg_env); >> >> msg_env = enif_alloc_env();enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env); >> } >> >> >> static void on_callback2(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_clear_env(msg_env); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env); >> } >> >> >> Only the first msg comes, its like it caches/remembers it and ignores the rest. I wonder if its due to not being able to >> call enif_send from a random non SMP thread and if I need to make a queue that I poll later? >> >> By only the first msg comes, if I later do a enif_send with "okk2" atom as the first, it wont send unless the atom is exactly "okk". >> >> The same thing if you substitute the atom with an integer or binary. >> >> On Tuesday, March 21, 2017 6:52 AM, Sverker Eriksson wrote: >> >> >> >> enif_send docs says: >> >> "The message environment msg_env with all its terms (including msg) >> is invalidated by a successful call to enif_send. The environment is to >> either be freed with enif_free_env of cleared for reuse with >> enif_clear_env." >> >> which means you have to call enif_clear_env before reusing msg_env >> in the second call to enif_send. >> >> Or if you're running OTP 19 you can pass msg_env as NULL, >> in which case the message term is copied and its environment left intact >> by enif_send. >> >> >> /Sverker, Erlang/OTP >> >> >> >> On 03/21/2017 05:10 AM, Vans S wrote: >>> I am having a weird issue with enif_send. >>> >>> What is happening is I am interoping with a 3rd party C library which has >>> its own event loop and procs callbacks from its own thread. >>> >>> When the callback procs (in the C NIF) I want to call enif_send to get >>> the message back to an erlang pid. >>> >>> When I call enif_send the first term I pass it works, then every subsequent message >>> gets dropped unless its exactly the same as the first time passed. >>> >>> For example >>> >>> >>> static ErlNifPid my_temp_pid; >>> static void on_callback(void* user_data) >>> { >>> ErlNifEnv* msg_env = enif_alloc_env(); >>> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >>> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >>> enif_free_env(msg_env);} >>> >>> *on_callback proced >>> 6> receive X-> X after 1 -> none end. >>> okk >>> >>> 7> receive X-> X after 1 -> none end.none >>> *on_callback proced >>> 8> receive X-> X after 1 -> none end. >>> okk >>> 9> receive X-> X after 1 -> none end. >>> none >>> *on_callback proced >>> *on_callback proced >>> 10> receive X-> X after 1 -> none end. >>> okk >>> 11> receive X-> X after 1 -> none end. >>> okk >>> 12> receive X-> X after 1 -> none end. >>> none >>> What is the correct way to do this? Thanks. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vans_163@REDACTED Tue Mar 21 17:40:36 2017 From: vans_163@REDACTED (Vans S) Date: Tue, 21 Mar 2017 16:40:36 +0000 (UTC) Subject: [erlang-questions] enif_send, how to call off a smp thread? In-Reply-To: References: <144675736.45252.1490069443146.ref@mail.yahoo.com> <144675736.45252.1490069443146@mail.yahoo.com> <647032232.584180.1490104624452@mail.yahoo.com> <1752286703.6143797.1490108355186@mail.yahoo.com> Message-ID: <1241306838.6244299.1490114436069@mail.yahoo.com> > I think I know the answer: single assignment. > Try flush() in the shell and get all messages, > even those that don't match X :-) x.x I think I can cancel going to Indonesia, seen enough dragons for today :) On Tuesday, March 21, 2017 11:47 AM, Sverker Eriksson wrote: I think I know the answer: single assignment. Try flush() in the shell and get all messages, even those that don't match X :-) /Sverker On 03/21/2017 03:59 PM, Vans S wrote: > Im going to try that, for now I made a test case that fails for me > https://github.com/vans163/nif_test. > > The 3rd receive we get ok instead of 56. > > I am on Erlang 19.1.2 though, maybe I should try the R19.3 or master HEAD? > > > > On Tuesday, March 21, 2017 10:28 AM, Sverker Eriksson wrote: > > > > enif_send can be called from a non scheduler thread in an SMP enabled VM > (beam.smp) > but not in a non-SMP VM (beam). > > The outcome being dependent on the message term sounds very strange. > > > Run with debug compiled VM to get more runtime sanity checks. > > > $> (cd $ERL_TOP/erts/emulator && make TYPE=debug smp) > > $> $ERL_TOP/bin/cerl -debug > > > /Sverker > > > > On 03/21/2017 02:57 PM, Vans S wrote: >> Iv tried so many combinations of passing the default env in the user_data of the callback/setting a global or setting that to NULL and using the msg_env, same result. Even if I do it like this or that: >> >> static ErlNifPid my_temp_pid; >> static void on_callback(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_free_env(msg_env); >> >> msg_env = enif_alloc_env();enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env); >> } >> >> >> static void on_callback2(void* user_data) >> { >> ErlNifEnv* msg_env = enif_alloc_env(); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >> enif_clear_env(msg_env); >> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >> enif_free_env(msg_env); >> } >> >> >> Only the first msg comes, its like it caches/remembers it and ignores the rest. I wonder if its due to not being able to >> call enif_send from a random non SMP thread and if I need to make a queue that I poll later? >> >> By only the first msg comes, if I later do a enif_send with "okk2" atom as the first, it wont send unless the atom is exactly "okk". >> >> The same thing if you substitute the atom with an integer or binary. >> >> On Tuesday, March 21, 2017 6:52 AM, Sverker Eriksson wrote: >> >> >> >> enif_send docs says: >> >> "The message environment msg_env with all its terms (including msg) >> is invalidated by a successful call to enif_send. The environment is to >> either be freed with enif_free_env of cleared for reuse with >> enif_clear_env." >> >> which means you have to call enif_clear_env before reusing msg_env >> in the second call to enif_send. >> >> Or if you're running OTP 19 you can pass msg_env as NULL, >> in which case the message term is copied and its environment left intact >> by enif_send. >> >> >> /Sverker, Erlang/OTP >> >> >> >> On 03/21/2017 05:10 AM, Vans S wrote: >>> I am having a weird issue with enif_send. >>> >>> What is happening is I am interoping with a 3rd party C library which has >>> its own event loop and procs callbacks from its own thread. >>> >>> When the callback procs (in the C NIF) I want to call enif_send to get >>> the message back to an erlang pid. >>> >>> When I call enif_send the first term I pass it works, then every subsequent message >>> gets dropped unless its exactly the same as the first time passed. >>> >>> For example >>> >>> >>> static ErlNifPid my_temp_pid; >>> static void on_callback(void* user_data) >>> { >>> ErlNifEnv* msg_env = enif_alloc_env(); >>> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "okk")); >>> enif_send(NULL, &my_temp_pid, msg_env, mk_atom(msg_env, "aaokk")); >>> enif_free_env(msg_env);} >>> >>> *on_callback proced >>> 6> receive X-> X after 1 -> none end. >>> okk >>> >>> 7> receive X-> X after 1 -> none end.none >>> *on_callback proced >>> 8> receive X-> X after 1 -> none end. >>> okk >>> 9> receive X-> X after 1 -> none end. >>> none >>> *on_callback proced >>> *on_callback proced >>> 10> receive X-> X after 1 -> none end. >>> okk >>> 11> receive X-> X after 1 -> none end. >>> okk >>> 12> receive X-> X after 1 -> none end. >>> none >>> What is the correct way to do this? Thanks. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From frank.muller.erl@REDACTED Tue Mar 21 19:22:15 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 21 Mar 2017 18:22:15 +0000 Subject: [erlang-questions] Cowboy handler: wait for slow backend Message-ID: Hi Everyone, I?m facing a little strange issue with Cowboy (latest from GitHub). My backend (a DB) takes couple of seconds before returning a result. The problem is that my Cowboy? handler seems to close the connexion with the client before my backend returns. Or wait forever!!! Is there a way to tell Cowboy to wait let say 30sec before closing the connexion with the client ??? Thanks in advance. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Mar 21 19:26:32 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 21 Mar 2017 19:26:32 +0100 Subject: [erlang-questions] Cowboy handler: wait for slow backend In-Reply-To: References: Message-ID: <26dddb7b-1ecf-98a2-1db1-3c7fd24baa2f@ninenines.eu> It's a known issue: https://github.com/ninenines/cowboy/issues/1032#issuecomment-280765513 A temporary fix is to set a larger request_timeout. Check the commit by Benoit linked to the issue for an example. It will be fixed sometimes before 2.0 is released. On 03/21/2017 07:22 PM, Frank Muller wrote: > Hi Everyone, > > I?m facing a little strange issue with Cowboy (latest from GitHub). > > My backend (a DB) takes couple of seconds before returning a result. > > The problem is that my Cowboy? handler seems to close the connexion with > the client before my backend returns. Or wait forever!!! > > Is there a way to tell Cowboy to wait let say 30sec before closing the > connexion with the client ??? > > Thanks in advance. > > /Frank > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From frank.muller.erl@REDACTED Tue Mar 21 20:22:05 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 21 Mar 2017 19:22:05 +0000 Subject: [erlang-questions] Cowboy handler: wait for slow backend In-Reply-To: <3D25018F-2559-49B2-A841-8ED52246FBC2@pobox.com> References: <3D25018F-2559-49B2-A841-8ED52246FBC2@pobox.com> Message-ID: Thanks a lot guys. Works perfectly now :-) /Frank Hi, > > You can set a request timeout in the cowboy options. The default is 5 > seconds. > > See also the change here (at the bottom) > > https://github.com/zotonic/zotonic/commit/ad0ec7576ee1cbbb589f61ff4528fbeb29b11d36 > > Best, Marc > > Sent from my iPhone > > On 21 Mar 2017, at 19:22, Frank Muller wrote: > > Hi Everyone, > > I?m facing a little strange issue with Cowboy (latest from GitHub). > > My backend (a DB) takes couple of seconds before returning a result. > > The problem is that my Cowboy? handler seems to close the connexion with > the client before my backend returns. Or wait forever!!! > > Is there a way to tell Cowboy to wait let say 30sec before closing the > connexion with the client ??? > > Thanks in advance. > > /Frank > > _______________________________________________ > > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.sedelnikov@REDACTED Wed Mar 22 11:07:23 2017 From: n.sedelnikov@REDACTED (Nick Sedelnikov) Date: Wed, 22 Mar 2017 10:07:23 +0000 Subject: [erlang-questions] Erlang jobs Message-ID: Hello all, Sorry for asking, but maybe someone could help me. Is there any chance for Erlang newcomers to find a job? I was looking for an erlang job for a while and there are not so many positions available. I'm experienced python developer, but some time ago I've met Erlang in one of my projects and fell in love with it. But unfortunately I have no opportunity to work with it at my current position. So I think that the best way for me is to find a new job. Is there any special job sites for erlang? In what companies should I look for open positions? Thanks in advance. Kind regards, Nikolay -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Wed Mar 22 12:52:51 2017 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Wed, 22 Mar 2017 11:52:51 +0000 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: Hi Nikolay I good place to start would be: http://erlangcentral.org/jobs Thomas ons. 22. mar. 2017 kl. 11.41 skrev Nick Sedelnikov : > Hello all, > > Sorry for asking, but maybe someone could help me. > Is there any chance for Erlang newcomers to find a job? I was looking for > an erlang job for a while and there are not so many positions available. > I'm experienced python developer, but some time ago I've met Erlang in one > of my projects and fell in love with it. But unfortunately I have no > opportunity to work with it at my current position. > So I think that the best way for me is to find a new job. > > Is there any special job sites for erlang? In what companies should I look > for open positions? > > Thanks in advance. > > > Kind regards, > Nikolay > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.sedelnikov@REDACTED Thu Mar 23 11:42:15 2017 From: n.sedelnikov@REDACTED (Nick Sedelnikov) Date: Thu, 23 Mar 2017 10:42:15 +0000 Subject: [erlang-questions] Erlang jobs In-Reply-To: References: Message-ID: Hi Thomas, Thank you, I'll check it out! Nikolay ??, 22 ????? 2017 ?., 12:53 Thomas Elsgaard : > Hi Nikolay > > I good place to start would be: > > http://erlangcentral.org/jobs > > Thomas > ons. 22. mar. 2017 kl. 11.41 skrev Nick Sedelnikov >: > > Hello all, > > Sorry for asking, but maybe someone could help me. > Is there any chance for Erlang newcomers to find a job? I was looking for > an erlang job for a while and there are not so many positions available. > I'm experienced python developer, but some time ago I've met Erlang in one > of my projects and fell in love with it. But unfortunately I have no > opportunity to work with it at my current position. > So I think that the best way for me is to find a new job. > > Is there any special job sites for erlang? In what companies should I look > for open positions? > > Thanks in advance. > > > Kind regards, > Nikolay > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stamarit@REDACTED Thu Mar 23 20:33:35 2017 From: stamarit@REDACTED (Salvador Tamarit) Date: Thu, 23 Mar 2017 20:33:35 +0100 Subject: [erlang-questions] CfP: WAO 2017 (2nd Workshop on Actors and Active Objects); Satellite event of iFM17 Message-ID: [Apologies if you receive multiple copies of this announcement.] CALL FOR PAPERS WAO 2017 2nd Workshop on Actors and Active Objects A Satellite workshop of iFM 2017 Torino, Italy 18 September 2017 http://costa.ls.fi.upm.es/wao17 Description of the workshop =========================== Distribution and concurrency are currently mainstream. The Internet and the broad availability of multi-processors radically influence software. This brings renewed interest in developing both new concurrency models and associated programming languages techniques that help in understanding, analyzing, and verifying the behavior of concurrent and distributed programs. The actors or active objects concurrency model has emerged as an alternative to the usual thread-based concurrency model, providing programmers with high-level concurrency constructs that help in producing concurrent applications more modularly and in a less error-prone way. It is gaining a lot of popularity mainly because of the success of languages like Scala and Erlang. Also, most mainstream languages are nowadays providing actors or active-objects libraries. The objective of the workshop is to discuss about current evolution of actors and active objects and related languages, technology and tools. The goal is to make the workshop an active discussion forum for all work related to actor languages and active-objects in order to have a better view on the current and future trends in this field and perhaps build longer term collaborations. The workshop will be a mixture of invited presentations and short tutorials on state-of-the-art languages/techniques/tools by experts in the field, presentations of recently published high-quality papers, prototype demonstrations, and presentations of proofs-of-concept and promising ideas. Invited speaker =============== Lars Ake Fredlund, Technical University of Madrid Important dates =============== Abstract submission: May 25, 2017 Paper submission: June 1, 2017 Notification: July 10, 2017 Submission information ====================== We solicit papers in the following categories: (1) Regular research papers of at most 12 pages (excluding references) describing original scientific research results or its relevance to real applications. Case studies and tool demonstrations are also welcome, in these cases with a maximum length of 8 pages. The paper should explain why the technique/case-study/tool is relevant for the community, and, in particular, for practitioners. Submissions will be judged on the basis of significance, relevance, correctness, originality, and clarity. (2) Position papers of at most 4 pages describing exciting but not fully polished research (proofs-of-concept, promising ideas, etc.). (3) High-quality already published papers. Authors will send an extended abstract of at most 2 pages describing the work and a link to the publication. Revised versions of accepted papers of category (1), taking into account the feedback received at the workshop, will be published in a volume of the Electronic Proceedings in Theoretical Computer (EPTCS) series after the workshop. For submissions in categories (2) and (3) there will be a light-weight reviewing process where submissions will be judged on their interest to the workshop audience. Papers will be submitted (in PDF format) through Easychair at https://easychair.org/conferences/?conf=wao2017 Organisers ========== Miguel Gomez-Zamalloa, Complutense University of Madrid, Spain Guillermo Rom?n-D?ez, Technical University of Madrid, Spain All questions about the workshop can be addressed to them via mail: mzamalloa@REDACTED groman@REDACTED Program Committee ================== Gul Agha University of Illinois at Urbana-Champaign Nikolaos Bezirgiannis Centrum Wiskunde & Informatica (CWI) Richard Bubel Technische Universitat Darmstadt Stijn De Gouw Centrum Wiskunde & Informatica (CWI) Enrique Martin-Martin Complutense University of Madrid Ludovic Henrio CNRS, Sophia Antipolis, France Ka I Pun University of Oslo Rudolf Schlatte University of Oslo Kostis Sagonas Uppsala University Salvador Tamarit Technical University of Valencia -------------- next part -------------- An HTML attachment was scrubbed... URL: From ducondez@REDACTED Thu Mar 23 22:39:05 2017 From: ducondez@REDACTED (Danniel Condez) Date: Thu, 23 Mar 2017 14:39:05 -0700 Subject: [erlang-questions] Storing Map into DB Message-ID: Hi, Any recommendation for persisting a Map into an unstructured db/filesystem. The goal is to store and fetch it as is, so that both erlang backend and FrontEnd (GraphQL) can easily fetch it. Thanks! Danniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hachreak@REDACTED Fri Mar 24 09:32:26 2017 From: hachreak@REDACTED (Leo) Date: Fri, 24 Mar 2017 09:32:26 +0100 Subject: [erlang-questions] Storing Map into DB In-Reply-To: References: Message-ID: 2017-03-23 22:39 GMT+01:00 Danniel Condez : > Hi, > > Any recommendation for persisting a Map into an unstructured > db/filesystem. The goal is to store and fetch it as is, so that both erlang > backend and FrontEnd (GraphQL) can easily fetch it. > I would say a document-oriented nosql DB: https://en.wikipedia.org/wiki/Document-oriented_database cheers, Leo > > Thanks! > Danniel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Fri Mar 24 15:12:41 2017 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 24 Mar 2017 09:12:41 -0500 Subject: [erlang-questions] Storing Map into DB Message-ID: <28641727-5E14-4A48-8404-FE9DFC05E89E@gmail.com> Simplest (and most portable) would be: for the put/write: term_to_binary(Map), for the get/read: binary_to_term(Blob) From vans_163@REDACTED Fri Mar 24 15:45:06 2017 From: vans_163@REDACTED (Vans S) Date: Fri, 24 Mar 2017 14:45:06 +0000 (UTC) Subject: [erlang-questions] Storing Map into DB In-Reply-To: <28641727-5E14-4A48-8404-FE9DFC05E89E@gmail.com> References: <28641727-5E14-4A48-8404-FE9DFC05E89E@gmail.com> Message-ID: <1989216628.9337753.1490366706707@mail.yahoo.com> A mnesia table with the key being the uuid of the object. {db_mytable, uuid4, #{}} You have full fold/match/select available to you now.? [Valid] = :mnesia.match_object({:db_mytable, _, #{auth=>#{token=> "353442323232"}}) On Friday, March 24, 2017 10:13 AM, Steve Davis wrote: Simplest (and most portable) would be: for the put/write: term_to_binary(Map), for the get/read: binary_to_term(Blob) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Mar 24 16:01:39 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 24 Mar 2017 15:01:39 +0000 Subject: [erlang-questions] Storing Map into DB In-Reply-To: References: Message-ID: On Thu, Mar 23, 2017 at 10:51 PM Danniel Condez wrote: > Hi, > > Any recommendation for persisting a Map into an unstructured > db/filesystem. The goal is to store and fetch it as is, so that both erlang > backend and FrontEnd (GraphQL) can easily fetch it. > > Our GraphQL system uses Postgres as a database and utilizes jsonb columns to store map data inside the database. The tradeoff is that your map keys are binary(), but the advantage is that you have the ability to add indexes over your jsonb columsn for quicker access to report-style queries in your data set. We essentially took Facebook's TAO-paper (Types, Associations & Objects) and implemented it on top of Postgres rather than MySQL. We can then use the TAO-API in order to satisfy queries in the GraphQL system. Preliminary analysis and benchmarking shows that this works exceptionally well. Most queries are satisfied by index lookups from memory and is in the sub 3ms range, even for larger blobs of data. Our current primary gotcha is that you can't define -spec m() :: #{ <<"foo">> := integer(), <<"bar">> := integer() }. for the dialyzer. So we can't make the dialyzer efficient at debugging right now. However, the keys are static keys, so we could probably convert them into atom()'s without too much hassle. They are limited in what they can be. This requires some future rework of our GraphQL layer as well though. (For those interested in the GraphQL stuff: talk and release is currently slated for EUC 2017). -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Mar 24 22:14:02 2017 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 24 Mar 2017 22:14:02 +0100 Subject: [erlang-questions] Storing Map into DB In-Reply-To: References: Message-ID: Hi, Sharing a nice article on postgresql and jsonb... http://obartunov.livejournal.com/193870.html "Envoy? depuis mon mobile " Eric ---- Jesper Louis Andersen a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dima_kakurin@REDACTED Fri Mar 24 19:42:28 2017 From: dima_kakurin@REDACTED (Dmitry Kakurin) Date: Fri, 24 Mar 2017 18:42:28 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options Message-ID: Hi all, I've started using Dialyzer and tried to enable as many warnings as possible. Two of them that seem especially useful are underspecs and overspecs. But after struggling with warnings produced by them for a while I've realized that what I really need is a subset of each. In the example below (sorry, I'm using Elixir syntax but hope it's still obvious) the 2 warnings I want to enable are under_in and over_out. They are the ones that will break callers relying on the spec. While over_in and under_out warnings are more of a hints to the function author about current implementation and are much less serious. The current Dialyzer options allow me to enable either both over_* warnings or both under_* warnings. But that's not useful. Is there a way to configure Dialyzer the way I want? Here is the demo code: # suboptimal implementation, fine for caller @spec over_in(x :: integer) :: nil def over_in(x) when is_number(x) do nil end # broken caller contract, could return unexpected value @spec over_out(x :: number) :: integer def over_out(x) when is_number(x) do x end # broken caller contract, rejects input allowed by the spec @spec under_in(x :: number) :: nil def under_in(x) when is_integer(x) do nil end # current implementation detail, does not concern caller @spec under_out(x :: integer) :: number def under_out(x) when is_integer(x) do x end and here are the warnings: demo.ex:4: Type specification 'Elixir.Virt':over_in(x::integer()) -> 'nil' is a subtype of the success typing: 'Elixir.Virt':over_in(number()) -> 'nil' demo.ex:9: Type specification 'Elixir.Virt':over_out(x::number()) -> integer() is a subtype of the success typing: 'Elixir.Virt':over_out(number()) -> number() demo.ex:14: Type specification 'Elixir.Virt':under_in(x::number()) -> 'nil' is a supertype of the success typing: 'Elixir.Virt':under_in(integer()) -> 'nil' demo.ex:19: The specification for 'Elixir.Virt':under_out/1 states that the function might also return float() but the inferred return is integer() Thanks, Dmitry. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ducondez@REDACTED Sat Mar 25 03:43:36 2017 From: ducondez@REDACTED (Danniel Condez) Date: Fri, 24 Mar 2017 19:43:36 -0700 Subject: [erlang-questions] Storing Map into DB In-Reply-To: References: Message-ID: Thanks Jesper! The TAO-paper is very helpful. One of the data structures I'm looking to work with are RDF triples. Right now, the end goal is rapid prototyping with no overhead on db administration and normalization. So as much as possible, I would want to keep almost a 1-1 mapping from Erlang maps to GraphQL data in the frontend. And it sounds like you built your own graphQL layer as well. What did you think about https://github.com/graphql-erlang/graphql ? Thanks! Danniel On Fri, Mar 24, 2017 at 8:01 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Thu, Mar 23, 2017 at 10:51 PM Danniel Condez > wrote: > >> Hi, >> >> Any recommendation for persisting a Map into an unstructured >> db/filesystem. The goal is to store and fetch it as is, so that both erlang >> backend and FrontEnd (GraphQL) can easily fetch it. >> >> > Our GraphQL system uses Postgres as a database and utilizes jsonb columns > to store map data inside the database. The tradeoff is that your map keys > are binary(), but the advantage is that you have the ability to add indexes > over your jsonb columsn for quicker access to report-style queries in your > data set. > > We essentially took Facebook's TAO-paper (Types, Associations & Objects) > and implemented it on top of Postgres rather than MySQL. We can then use > the TAO-API in order to satisfy queries in the GraphQL system. Preliminary > analysis and benchmarking shows that this works exceptionally well. Most > queries are satisfied by index lookups from memory and is in the sub 3ms > range, even for larger blobs of data. > > Our current primary gotcha is that you can't define > > -spec m() :: #{ <<"foo">> := integer(), <<"bar">> := integer() }. > > for the dialyzer. So we can't make the dialyzer efficient at debugging > right now. However, the keys are static keys, so we could probably convert > them into atom()'s without too much hassle. They are limited in what they > can be. This requires some future rework of our GraphQL layer as well > though. > > (For those interested in the GraphQL stuff: talk and release is currently > slated for EUC 2017). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ducondez@REDACTED Sat Mar 25 03:49:57 2017 From: ducondez@REDACTED (Danniel Condez) Date: Fri, 24 Mar 2017 19:49:57 -0700 Subject: [erlang-questions] Storing Map into DB In-Reply-To: <1218707733.3788494.1490376302365@mail.yahoo.com> References: <28641727-5E14-4A48-8404-FE9DFC05E89E@gmail.com> <1989216628.9337753.1490366706707@mail.yahoo.com> <1218707733.3788494.1490376302365@mail.yahoo.com> Message-ID: Vans, I think the second approach you mentioned is more in-lined to what I'm looking for cause I'm trying to stay away from the data being relational. Instead I would like to work more with RDF triples. Thank you so much for your help! -Danniel On Fri, Mar 24, 2017 at 10:25 AM, Vans S wrote: > > I try to stay away from normalized data so have more flexibility in > terms of storing and retrieving data. > > In an ideal world I would like mnesia to support maps. Then you can drop > the record and mnesia becomes a document db, you give it the > primary/secondary indexes as you create the table, > it will find those in the maps that are the content. > > > > However, making relationships or pointers to other records with > different types is still kinda vague. I'm prett ynew to erlang but I think > I can start with this and learn further. > This is tough for me too. If anyone has good ideas I'm all ears. > > The simplest approach is writing a module with the same name as the tables > record name and putting the logic into there. The associations would be > %{address_ref=> uuid4, wallet_ref=> uuid4}, then to find the wallet > associated with a particular user you would just do > :mnesia.read({:db_wallet, wallet_ref}). > > > Another approach is a seperate sql style reference table, > db_r_user_wallet to map the uuid references. This has the advantage of > keeping the data cleaner as you don't have refs in there. > > Personally I use the first approach, and only considered the second. > > Mnesia's transactions either succeed or fail, so you can do complex things > in a single transaction like touching multiple tables, and if something > goes wrong you wont have a half committed transaction. > > One of my recent projects was a compute cloud and attaching disks to > virtual machines looks like this: > > -module(db_disk). > > > attach(UuidOwner, UuidVm, UuidDisk) -> > ?ENSURE_OWNER(OwnerUuid, UuidVm, UuidDisk), > ?ENSURE_ATTACHABLE(UuidVm, UuidDisk), > > ?T( > ok = mnesia:write({db_disk, UuidDisk, %{attached_to=> %{UuidVm=> > UuidVm}}}) > ). > > > -module(db_vm). > > disks(OwnerUuid, UuidVm) -> > ?ENSURE_OWNER(OwnerUuid, UuidVm), > mnesia:activity(transcation, fun()-> > mnesia:match_object({db_disk, UuidVm, %{attached_to=> > %{UuidVm=>_}}}) > end). > > > The reason 'attached_to' == %{} is because of a hack for a missing data > type / maps functionality in erlang. > Maps are new and selects/matches are not able to work on lists, unless > someone can tell me how. > > Ideally %{UuidVm=> UuidVm} could be simplified to %{UuidVm} if erlang were > to allow sets like usage with > map syntax. And %{UuidVm=> _} would also become %{UuidVm}. > > Im not aware of a way in a match/select to do a lists:member/2. > > Each disk can be attached to multiple VMs in the case of network shared > block storage. > > > > On Friday, March 24, 2017 12:28 PM, Danniel Condez > wrote: > > > > Hi Vans, > > Looks like this is a pretty simple and flexible solution. I'm looking > deeper into mnesia and found `Object based programming in mnesia`( > http://erlang.org/doc/apps/mnesia/Mnesia_chap5.html#id82246) to be close > to what I'm trying to achieve. I try to stay away from normalized data so > have more flexibility in terms of storing and retrieving data. However, > making relationships or pointers to other records with different types is > still kinda vague. I'm prett ynew to erlang but I think I can start with > this and learn further. > > > On Fri, Mar 24, 2017 at 7:45 AM, Vans S wrote: > > A mnesia table with the key being the uuid of the object. > > > >{db_mytable, uuid4, #{}} > > > >You have full fold/match/select available to you now. > > > >[Valid] = :mnesia.match_object({:db_ mytable, _, #{auth=>#{token=> > "353442323232"}}) > > > > > > > >On Friday, March 24, 2017 10:13 AM, Steve Davis > wrote: > > > > > > > >Simplest (and most portable) would be: > > > > > > > >for the put/write: term_to_binary(Map), for the get/read: > binary_to_term(Blob) > > > >______________________________ _________________ > > > >erlang-questions mailing list > > > >erlang-questions@REDACTED > > > >http://erlang.org/mailman/ listinfo/erlang-questions > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Mar 25 18:38:57 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 25 Mar 2017 17:38:57 +0000 Subject: [erlang-questions] Storing Map into DB In-Reply-To: References: Message-ID: On Sat, Mar 25, 2017 at 3:43 AM Danniel Condez wrote: > And it sounds like you built your own graphQL layer as well. What did you > think about https://github.com/graphql-erlang/graphql ? > > Looks nice. Our implementation is a bit ahead of that though: * We can lint and type check schemas as well as queries * We have validations in place * We have directives in place * We parse the graphql.schema and bind it functions of the form execute(Context, CurObj, Field, Args) -> {ok, Res} | {error, Reason} | no_return() Our implementation is also closer to a typical compiler toolchain: compile queries into query plans. Execute those. I've started a bit on the documentation, but I'll make sure to open up our repository before that is totally done. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Sun Mar 26 14:45:50 2017 From: marco.molteni@REDACTED (Marco Molteni) Date: Sun, 26 Mar 2017 14:45:50 +0200 Subject: [erlang-questions] simple virtual file system in Erlang? Message-ID: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Hello colleagues, I would like to start from the presentation "Build an FTP Server with Ranch in 30 Minutes" and really build an FTP server, so at a certain point I need to hit the filesystem. Since the idea is still as a presentation (as opposed to building a production FTP server) I don't want to touch the real filesystem of the host, I want to use a as simple as possible virtual filesystem. It could be backed by DETS or Mnesia for example, or it could stay only in memory. On the other hand, it must still behave as a filesystem, that is, it must be hierarchical, so a direct mapping to a key/value store would not be enough. In the spirit of simplicity, I don't need any concept of read/write permission, I simply need a sort of graph with two types: inner nodes are directories, leaf nodes are files or empty directories. I am thinking to use DETS and somehow introduce a very simple intermediate layer that would offer the impression to be in a graph (each non-leaf node a directory) and map it to the DETS key/value API. Any suggestions? thanks marco [1] https://ninenines.eu/articles/ranch-ftp/ From jesper.louis.andersen@REDACTED Sun Mar 26 16:57:43 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 26 Mar 2017 14:57:43 +0000 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: The simplest way is probably to map a path to its data, where the path is a list of components. That is, the UNIX file ls in /usr/bin would be represented as ["usr", "bin", "ls"]. But you could also go the way of Plan9's venti. Map something like ["usr", "bin", "ls"] into a sha3 checksum of the data currently residing in the file and keep the data inside a storage suitable for storing data by its content addressing. venti also extends this by storing a tree of 64 kilobyte blocks which can then be regarded as the data when taken together. Thus, you have two components: the data store, and the path mapping service, mapping a path to the underlying data. This idea has the advantage of being practically useful in many situations beside a toy example :) On Sun, Mar 26, 2017 at 2:46 PM Marco Molteni wrote: > Hello colleagues, > > I would like to start from the presentation "Build an FTP Server with > Ranch in 30 Minutes" and really build an FTP server, so at a certain point > I need to hit the filesystem. > > Since the idea is still as a presentation (as opposed to building a > production FTP server) I don't want to touch the real filesystem of the > host, I want to use a as simple as possible virtual filesystem. It could be > backed by DETS or Mnesia for example, or it could stay only in memory. > > On the other hand, it must still behave as a filesystem, that is, it must > be hierarchical, so a direct mapping to a key/value store would not be > enough. In the spirit of simplicity, I don't need any concept of read/write > permission, I simply need a sort of graph with two types: inner nodes are > directories, leaf nodes are files or empty directories. > > I am thinking to use DETS and somehow introduce a very simple intermediate > layer that would offer the impression to be in a graph (each non-leaf node > a directory) and map it to the DETS key/value API. > > Any suggestions? > > thanks > marco > > [1] https://ninenines.eu/articles/ranch-ftp/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Mar 26 17:31:44 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 26 Mar 2017 08:31:44 -0700 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: Yes - mapping the path to a content addressable store address would be great. It would also be great in a wider context. A fun extension would be to add a an http interface. Something like POST To store , then later GET /blob/sha256/ to recover the blob. is the SHA256 checksum of the data (the path contains the type of the checksum - so you might say GETblob/md5/ then add a DHT - then run on every webserver on the planet then store all data forever. The nice thing about this is that it's self-securing - a person-in-the-middle cannot change the data without it being detected (since you can check the SHA of the data when you get it back) Such a system has three relatively simple layers - transport (say HTTP over TCP) - DHT - storage If you ask Jesper nicely he'll tell you all about DHT's :-) Have fun with the storage layer for this (and no it's NOT a toy project) /Joe On Sun, Mar 26, 2017 at 7:57 AM, Jesper Louis Andersen wrote: > The simplest way is probably to map a path to its data, where the path is a > list of components. That is, the UNIX file ls in /usr/bin would be > represented as ["usr", "bin", "ls"]. > > But you could also go the way of Plan9's venti. Map something like ["usr", > "bin", "ls"] into a sha3 checksum of the data currently residing in the file > and keep the data inside a storage suitable for storing data by its content > addressing. venti also extends this by storing a tree of 64 kilobyte blocks > which can then be regarded as the data when taken together. Thus, you have > two components: the data store, and the path mapping service, mapping a path > to the underlying data. > > This idea has the advantage of being practically useful in many situations > beside a toy example :) > > On Sun, Mar 26, 2017 at 2:46 PM Marco Molteni > wrote: >> >> Hello colleagues, >> >> I would like to start from the presentation "Build an FTP Server with >> Ranch in 30 Minutes" and really build an FTP server, so at a certain point I >> need to hit the filesystem. >> >> Since the idea is still as a presentation (as opposed to building a >> production FTP server) I don't want to touch the real filesystem of the >> host, I want to use a as simple as possible virtual filesystem. It could be >> backed by DETS or Mnesia for example, or it could stay only in memory. >> >> On the other hand, it must still behave as a filesystem, that is, it must >> be hierarchical, so a direct mapping to a key/value store would not be >> enough. In the spirit of simplicity, I don't need any concept of read/write >> permission, I simply need a sort of graph with two types: inner nodes are >> directories, leaf nodes are files or empty directories. >> >> I am thinking to use DETS and somehow introduce a very simple intermediate >> layer that would offer the impression to be in a graph (each non-leaf node a >> directory) and map it to the DETS key/value API. >> >> Any suggestions? >> >> thanks >> marco >> >> [1] https://ninenines.eu/articles/ranch-ftp/ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Sun Mar 26 19:31:02 2017 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 26 Mar 2017 10:31:02 -0700 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: Somebody tweeted me me a link to upspin.io https://upspin.io/doc/overview.md This has a very good way of naming files - well worth reading /Joe On Sun, Mar 26, 2017 at 8:31 AM, Joe Armstrong wrote: > Yes - mapping the path to a content addressable store address would be > great. It would also be great in a wider context. > > A fun extension would be to add a an http interface. Something like > > POST > > To store , then later > > GET /blob/sha256/ > > to recover the blob. is the SHA256 checksum of the data (the > path contains the > type of the checksum - so you might say GETblob/md5/ > > then add a DHT - then run on every webserver on the planet > then store all data forever. > > The nice thing about this is that it's self-securing - a person-in-the-middle > cannot change the data without it being detected (since you can check > the SHA of the > data when you get it back) > > > Such a system has three relatively simple layers > > - transport (say HTTP over TCP) > - DHT > - storage > > If you ask Jesper nicely he'll tell you all about DHT's :-) > > Have fun with the storage layer for this (and no it's NOT a toy project) > > /Joe > > On Sun, Mar 26, 2017 at 7:57 AM, Jesper Louis Andersen > wrote: >> The simplest way is probably to map a path to its data, where the path is a >> list of components. That is, the UNIX file ls in /usr/bin would be >> represented as ["usr", "bin", "ls"]. >> >> But you could also go the way of Plan9's venti. Map something like ["usr", >> "bin", "ls"] into a sha3 checksum of the data currently residing in the file >> and keep the data inside a storage suitable for storing data by its content >> addressing. venti also extends this by storing a tree of 64 kilobyte blocks >> which can then be regarded as the data when taken together. Thus, you have >> two components: the data store, and the path mapping service, mapping a path >> to the underlying data. >> >> This idea has the advantage of being practically useful in many situations >> beside a toy example :) >> >> On Sun, Mar 26, 2017 at 2:46 PM Marco Molteni >> wrote: >>> >>> Hello colleagues, >>> >>> I would like to start from the presentation "Build an FTP Server with >>> Ranch in 30 Minutes" and really build an FTP server, so at a certain point I >>> need to hit the filesystem. >>> >>> Since the idea is still as a presentation (as opposed to building a >>> production FTP server) I don't want to touch the real filesystem of the >>> host, I want to use a as simple as possible virtual filesystem. It could be >>> backed by DETS or Mnesia for example, or it could stay only in memory. >>> >>> On the other hand, it must still behave as a filesystem, that is, it must >>> be hierarchical, so a direct mapping to a key/value store would not be >>> enough. In the spirit of simplicity, I don't need any concept of read/write >>> permission, I simply need a sort of graph with two types: inner nodes are >>> directories, leaf nodes are files or empty directories. >>> >>> I am thinking to use DETS and somehow introduce a very simple intermediate >>> layer that would offer the impression to be in a graph (each non-leaf node a >>> directory) and map it to the DETS key/value API. >>> >>> Any suggestions? >>> >>> thanks >>> marco >>> >>> [1] https://ninenines.eu/articles/ranch-ftp/ >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From mjtruog@REDACTED Sun Mar 26 22:56:30 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 26 Mar 2017 13:56:30 -0700 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: <58D82AFE.7070700@gmail.com> Hi Marco, There is an undocumented module in the kernel application called ram_file which could allow you to use file functions while operating on data that is only in-memory. Despite how advantageous this can be, to switch from in-memory file operations to filesystem operations, you shouldn't use that approach until it becomes documented, since it could be removed at any time. So far, it appears the ram_file module will be removed based on the closed bug at https://bugs.erlang.org/browse/ERL-36 Best Regards, Michael On 03/26/2017 05:45 AM, Marco Molteni wrote: > Hello colleagues, > > I would like to start from the presentation "Build an FTP Server with Ranch in 30 Minutes" and really build an FTP server, so at a certain point I need to hit the filesystem. > > Since the idea is still as a presentation (as opposed to building a production FTP server) I don't want to touch the real filesystem of the host, I want to use a as simple as possible virtual filesystem. It could be backed by DETS or Mnesia for example, or it could stay only in memory. > > On the other hand, it must still behave as a filesystem, that is, it must be hierarchical, so a direct mapping to a key/value store would not be enough. In the spirit of simplicity, I don't need any concept of read/write permission, I simply need a sort of graph with two types: inner nodes are directories, leaf nodes are files or empty directories. > > I am thinking to use DETS and somehow introduce a very simple intermediate layer that would offer the impression to be in a graph (each non-leaf node a directory) and map it to the DETS key/value API. > > Any suggestions? > > thanks > marco > > [1] https://ninenines.eu/articles/ranch-ftp/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From alex0player@REDACTED Mon Mar 27 10:09:45 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 27 Mar 2017 11:09:45 +0300 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: Message-ID: What you want is provided as a default behaviour. Overspecs you shouldn?t really enable for production needs, it?s ?debug the Dialyzer? feature. > 24 ????? 2017 ?., ? 21:42, Dmitry Kakurin ???????(?): > > Hi all, > I've started using Dialyzer and tried to enable as many warnings as possible. > Two of them that seem especially useful are underspecs and overspecs. > But after struggling with warnings produced by them for a while I've realized that what I really need is a subset of each. > In the example below (sorry, I'm using Elixir syntax but hope it's still obvious) the 2 warnings I want to enable are under_in and over_out. They are the ones that will break callers relying on the spec. While over_in and under_out warnings are more of a hints to the function author about current implementation and are much less serious. > The current Dialyzer options allow me to enable either both over_* warnings or both under_* warnings. But that's not useful. > Is there a way to configure Dialyzer the way I want? > > Here is the demo code: > > # suboptimal implementation, fine for caller > @spec over_in(x :: integer) :: nil > def over_in(x) when is_number(x) do > nil > end > > # broken caller contract, could return unexpected value > @spec over_out(x :: number) :: integer > def over_out(x) when is_number(x) do > x > end > > # broken caller contract, rejects input allowed by the spec > @spec under_in(x :: number) :: nil > def under_in(x) when is_integer(x) do > nil > end > > # current implementation detail, does not concern caller > @spec under_out(x :: integer) :: number > def under_out(x) when is_integer(x) do > x > end > > and here are the warnings: > > demo.ex:4: Type specification > 'Elixir.Virt':over_in(x::integer()) -> 'nil' is a subtype of the success typing: > 'Elixir.Virt':over_in(number()) -> 'nil' > > demo.ex:9: Type specification > 'Elixir.Virt':over_out(x::number()) -> integer() is a subtype of the success typing: > 'Elixir.Virt':over_out(number()) -> number() > > demo.ex:14: Type specification > 'Elixir.Virt':under_in(x::number()) -> 'nil' is a supertype of the success typing: > 'Elixir.Virt':under_in(integer()) -> 'nil' > > demo.ex:19: The specification for 'Elixir.Virt':under_out/1 states that the function might also return > float() but the inferred return is > integer() > > Thanks, Dmitry. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andras.Bekes@REDACTED Mon Mar 27 18:03:40 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Mon, 27 Mar 2017 16:03:40 +0000 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes Message-ID: Hi All, I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). http://man7.org/linux/man-pages/man7/unix.7.html Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: > Sun_path_length=108. 108 > UDS_path = <<"whatever">>. <<"whatever">> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). 792 > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,...>>} > Port = 0. 0 > gen_tcp:connect(Address,Port,[local],infinity). (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} ** exception exit: badarg in function gen_tcp:connect/4 (gen_tcp.erl, line 149) (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. Seems to me that this is a bug in the Erlang implementation. What do you think? Thank you very much, Andras G. Bekes -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex0player@REDACTED Mon Mar 27 19:59:30 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 27 Mar 2017 20:59:30 +0300 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: Message-ID: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> I have a strong suspicion the binary might be automatically zero-terminated. > 27 ????? 2017 ?., ? 19:03, Bekes, Andras G ???????(?): > > > Hi All, > > I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. > If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). > http://man7.org/linux/man-pages/man7/unix.7.html > > Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: > > > Sun_path_length=108. > 108 > > > UDS_path = <<"whatever">>. > <<"whatever">> > > > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). > 792 > > > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. > {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,...>>} > > > Port = 0. > 0 > > > gen_tcp:connect(Address,Port,[local],infinity). > (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) > (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) > (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) > (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} > ** exception exit: badarg > in function gen_tcp:connect/4 (gen_tcp.erl, line 149) > (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} > (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} > > If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: > connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) > When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. > > Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. > Seems to me that this is a bug in the Erlang implementation. What do you think? > > Thank you very much, > > Andras G. Bekes > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Mon Mar 27 20:07:15 2017 From: marco.molteni@REDACTED (Marco Molteni) Date: Mon, 27 Mar 2017 20:07:15 +0200 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: <20377CD5-040A-42FA-82A1-E9E80171FB31@laposte.net> Hello all, thanks for the helpful answers, especially the idea of a two-level lookup, path to hash and hash to blob. This would also naturally take care of data deduplication. On the other hand, it would require to keep a reverse index to map a given checksum to all the paths pointing to it. Storing only the path, without notion of intermediate node, would also make very time consuming doing simple operations such as listing the contents of a given directory, I would have to lookup _all_ the paths... So the idea is nice, but it requires refinements :-) Now time to think a bit more about it. marco > On 26 Mar 2017, at 14:45, Marco Molteni wrote: > > Hello colleagues, > > I would like to start from the presentation "Build an FTP Server with Ranch in 30 Minutes" [1] and really build an FTP server, so at a certain point I need to hit the filesystem. > > Since the idea is still as a presentation (as opposed to building a production FTP server) I don't want to touch the real filesystem of the host, I want to use a as simple as possible virtual filesystem. It could be backed by DETS or Mnesia for example, or it could stay only in memory. > > On the other hand, it must still behave as a filesystem, that is, it must be hierarchical, so a direct mapping to a key/value store would not be enough. In the spirit of simplicity, I don't need any concept of read/write permission, I simply need a sort of graph with two types: inner nodes are directories, leaf nodes are files or empty directories. > > I am thinking to use DETS and somehow introduce a very simple intermediate layer that would offer the impression to be in a graph (each non-leaf node a directory) and map it to the DETS key/value API. > > Any suggestions? > > thanks > marco > > [1] https://ninenines.eu/articles/ranch-ftp/ From erlang@REDACTED Mon Mar 27 21:16:08 2017 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 27 Mar 2017 12:16:08 -0700 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <20377CD5-040A-42FA-82A1-E9E80171FB31@laposte.net> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> <20377CD5-040A-42FA-82A1-E9E80171FB31@laposte.net> Message-ID: As regards your comments of performance - I would be extremely cautious about predicting performance before any code is written and measured. This is the one area where predictions can be wildly incorrect. Any operating system worth the name caches large parts of the working set of the file system in memory which makes predicting performance very difficult. Even using a RAM file buffer might not help since the OS is itself caching file blocks in RAM - I'd go for the simplest and most beautiful design and then measure. Cheers /Joe On Mon, Mar 27, 2017 at 11:07 AM, Marco Molteni wrote: > Hello all, > > thanks for the helpful answers, especially the idea of a two-level lookup, path to hash and hash to blob. This would also naturally take care of data deduplication. On the other hand, it would require to keep a reverse index to map a given checksum to all the paths pointing to it. Storing only the path, without notion of intermediate node, would also make very time consuming doing simple operations such as listing the contents of a given directory, I would have to lookup _all_ the paths... So the idea is nice, but it requires refinements :-) > > Now time to think a bit more about it. > > marco > >> On 26 Mar 2017, at 14:45, Marco Molteni wrote: >> >> Hello colleagues, >> >> I would like to start from the presentation "Build an FTP Server with Ranch in 30 Minutes" [1] and really build an FTP server, so at a certain point I need to hit the filesystem. >> >> Since the idea is still as a presentation (as opposed to building a production FTP server) I don't want to touch the real filesystem of the host, I want to use a as simple as possible virtual filesystem. It could be backed by DETS or Mnesia for example, or it could stay only in memory. >> >> On the other hand, it must still behave as a filesystem, that is, it must be hierarchical, so a direct mapping to a key/value store would not be enough. In the spirit of simplicity, I don't need any concept of read/write permission, I simply need a sort of graph with two types: inner nodes are directories, leaf nodes are files or empty directories. >> >> I am thinking to use DETS and somehow introduce a very simple intermediate layer that would offer the impression to be in a graph (each non-leaf node a directory) and map it to the DETS key/value API. >> >> Any suggestions? >> >> thanks >> marco >> >> [1] https://ninenines.eu/articles/ranch-ftp/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Mon Mar 27 22:35:53 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 27 Mar 2017 20:35:53 +0000 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: On Sun, Mar 26, 2017 at 7:31 PM Joe Armstrong wrote: > Somebody tweeted me me a link to upspin.io > > https://upspin.io/doc/overview.md > > This has a very good way of naming files - well worth reading > > > Upspin does two things right: 1. It uses protobufs for its protocol on top of a simple HTTP layer: https://github.com/upspin/upspin/blob/master/upspin/proto/upspin.proto (it is very close to UBF in many ways) 2. It exposes file systems, not final solutions, through Key, Directory and Storage servers. The latter is especially powerful. It means you can evolve the system as long as it fits inside the rules of a file-system-like object. It abstracts the problem away and provides just a protocol for a client to follow. The naming scheme in Upspin is indeed brilliant since it allows everyone their own name space. As a side note: you don't need something like hex.pm if you have something like upspin. Really, all you have to provide is a nice UI layer and the underlying data layer will take care of the rest of the stuff. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Mon Mar 27 22:52:52 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 27 Mar 2017 16:52:52 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?simple_virtual_file_system_in_Erlang?= =?utf-8?q?=3F?= In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: <1490647972.985825289@apps.rackspace.com> Hi all, This is flying over my head, but sounds super interesting. How I'd love to see a tutorial that demonstrates a simple application based on upspin. Jesper, I know you're hyper busy, but I ask stupid questions can you help sketch out such a tutorial? Many thanks, LRP -----Original Message----- From: "Jesper Louis Andersen" Sent: Monday, March 27, 2017 4:35pm To: "Joe Armstrong" Cc: "Erlang-Questions Questions" Subject: Re: [erlang-questions] simple virtual file system in Erlang? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions On Sun, Mar 26, 2017 at 7:31 PM Joe Armstrong wrote: > Somebody tweeted me me a link to upspin.io > > https://upspin.io/doc/overview.md > > This has a very good way of naming files - well worth reading > > > Upspin does two things right: 1. It uses protobufs for its protocol on top of a simple HTTP layer: https://github.com/upspin/upspin/blob/master/upspin/proto/upspin.proto (it is very close to UBF in many ways) 2. It exposes file systems, not final solutions, through Key, Directory and Storage servers. The latter is especially powerful. It means you can evolve the system as long as it fits inside the rules of a file-system-like object. It abstracts the problem away and provides just a protocol for a client to follow. The naming scheme in Upspin is indeed brilliant since it allows everyone their own name space. As a side note: you don't need something like hex.pm if you have something like upspin. Really, all you have to provide is a nice UI layer and the underlying data layer will take care of the rest of the stuff. From ok@REDACTED Mon Mar 27 23:12:05 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 28 Mar 2017 10:12:05 +1300 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: Message-ID: <40BB09D0-9D3E-4A5C-81DA-D2BE70B383C3@cs.otago.ac.nz> > On 28/03/2017, at 5:03 AM, Bekes, Andras G wrote: > I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. Be careful there. OSX 10.11.3 manual page: UNIX-domain addresses are variable-length filesystem pathnames of at most 104 characters. The Single Unix Specification, version 4, section un.h, says The size of sun_path has intentionally been left undefined. This is because different implementations use different sizes. For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104. Since most implementations originate from BSD versions, the size is typically in the range 92 to 108. Applications should not assume a particular length for sun_path or assume that it can hold {_POSIX_PATH_MAX} bytes (256). Expect the last byte to be used for the NUL terminator of a C string, so the length of the actual name could be limited to 91..107. From lloyd@REDACTED Mon Mar 27 23:18:13 2017 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 27 Mar 2017 17:18:13 -0400 (EDT) Subject: [erlang-questions] Not quite ANN: ErlPress - work--in-progress toward an Erlang-based page make-up program Message-ID: <1490649493.65823253@apps.rackspace.com> Hello, I've been working for several months now on a major revision/extension of Joe Armstrong/Carl Wright/Hugh Watkins' erlguten program. The goal is a fairly full-functioned Erlang-based page make-up program suitable for creating print-ready *.pdf files. So far I have implemented: -- A markdown parser for text input -- Structures and functions to define page grids -- Functions that implement text jumps across two or more columns of different widths (think "continued on page xxx" functionality). -- Isolated many useful functions that are currently not well documented and buried in erlguten source. Tasks ahead involve wrapping up these functions into a documented and easy-to-use interface for creating postcards, reports, books, and other print documents. Ideally I see ErlPress as the basis for a web-based program similar to Scribus. But I'm not ready to go that far yet. The code is still sloppy so I'm not ready to release it. But I would love to get ideas and input on high-level design/API from folks experienced with and interested in print media. If that's you, please contact me off-line. All the best, LRP From ok@REDACTED Mon Mar 27 23:23:50 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 28 Mar 2017 10:23:50 +1300 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> Message-ID: <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> > On 27/03/2017, at 4:31 AM, Joe Armstrong wrote: > To store , then later > > GET /blob/sha256/ > > to recover the blob. is the SHA256 checksum of the data (the > path contains the > type of the checksum - so you might say GETblob/md5/ It seems to me that this amounts to using a checksum of the contents of a file instead of an inode number. What I don't understand is how I would ever use this. When I go looking for a file, I want its *contents*, which I don't know, otherwise I wouldn't need it. So I *can't* compute its checksum. With things like Spotlight, I can use a known *part* of the content to look for the rest of the content, but again, if I already knew enough of the content to compute the checksum, I wouldn't bother looking at the file. The only way I can think of to use something like this is to maintain some sort of name-based directory structure on top, or some sort of IR-like inverted index based on part of the content. From mjtruog@REDACTED Tue Mar 28 00:10:55 2017 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 27 Mar 2017 15:10:55 -0700 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> Message-ID: <58D98DEF.4020001@gmail.com> On 03/27/2017 02:23 PM, Richard A. O'Keefe wrote: >> On 27/03/2017, at 4:31 AM, Joe Armstrong wrote: >> To store , then later >> >> GET /blob/sha256/ >> >> to recover the blob. is the SHA256 checksum of the data (the >> path contains the >> type of the checksum - so you might say GETblob/md5/ > It seems to me that this amounts to using a checksum of the contents > of a file instead of an inode number. > > What I don't understand is how I would ever use this. > When I go looking for a file, I want its *contents*, > which I don't know, otherwise I wouldn't need it. > So I *can't* compute its checksum. With things like > Spotlight, I can use a known *part* of the content to > look for the rest of the content, but again, if I > already knew enough of the content to compute the > checksum, I wouldn't bother looking at the file. > > The only way I can think of to use something like this > is to maintain some sort of name-based directory > structure on top, or some sort of IR-like inverted index > based on part of the content. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Yes, it is normally for a name-based directory related to peer-to-peer file sharing. The links have been called "magnet links" as described at https://en.wikipedia.org/wiki/Magnet_URI_scheme . From ok@REDACTED Tue Mar 28 06:22:42 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 28 Mar 2017 17:22:42 +1300 Subject: [erlang-questions] Reliable kernels? Message-ID: I was introduced to the concept of a "million-year bug" today. That's a bug in a program that, if you were running it on a single CPU, would be expected to show up once in a million years. With a couple of thousand million Linux kernels around the world in phones &c, we can expect a million year bug in the kernel to show up tens of times a day. There's been a spate of problems with 911 in Dallas being overloaded with bogus calls apparently sent autonomously by certain mobile phones, to the point where a man and a child are thought to have died because genuine 911 callers were put on hold for a long time. That's probably *not* a million-year bug, but a million-year bug might do that kind of thing. Now me, I'm still happily pottering away on 4-core and 16-core machines (even a 1-core machine that sees a lot of use because it Just Keeps Working). But I'm talking to people who want to use unbelievable amounts of computing power. I understand the Erlang Way: write your software as lots of small things communicating through narrow protocols, *expect* failure and deal with it. I believe! Praise Joe, I believe! That's not the way the people I'm talking to think. They've got a somewhat resilient data flow scheme they're proud of that has thousands and tens of thousands of nodes hooked up through Python, where the protocol between the nodes is Pyro. Not, "uses Pyro", "IS Pyro". I'm supposed to tell these people what would be a good stripped down kernel to use (or what would be a good kernel to strip further), and I'm tempted to start by saying "strip away Python" which certainly won't make me popular (;-). But thinking about error rates and million-year bugs has me thinking harder. It seems as if we have to think in terms of *expecting* the kernel itself to be unreliable-at-scale, so that something like JailHouse might be the right level to start. Does anyone have any experience with trying to harden a large system against faults in the distribution layer and in the kernel, and have any advice they'd care to share? None of this is Erlang-specific, it's just that I think the Erlang community are likely to have more relevant experience than most. From mjtruog@REDACTED Tue Mar 28 07:41:55 2017 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 27 Mar 2017 22:41:55 -0700 Subject: [erlang-questions] Reliable kernels? In-Reply-To: References: Message-ID: <58D9F7A3.3010403@gmail.com> On 03/27/2017 09:22 PM, Richard A. O'Keefe wrote: > I was introduced to the concept of a "million-year bug" today. > That's a bug in a program that, if you were running it on a > single CPU, would be expected to show up once in a million years. > > With a couple of thousand million Linux kernels around the world > in phones &c, we can expect a million year bug in the kernel to > show up tens of times a day. > > There's been a spate of problems with 911 in Dallas being overloaded > with bogus calls apparently sent autonomously by certain mobile > phones, to the point where a man and a child are thought to have > died because genuine 911 callers were put on hold for a long time. > That's probably *not* a million-year bug, but a million-year bug > might do that kind of thing. > > Now me, I'm still happily pottering away on 4-core and 16-core > machines (even a 1-core machine that sees a lot of use because it > Just Keeps Working). But I'm talking to people who want to use > unbelievable amounts of computing power. > > I understand the Erlang Way: write your software as lots of > small things communicating through narrow protocols, *expect* > failure and deal with it. I believe! Praise Joe, I believe! > > That's not the way the people I'm talking to think. They've got > a somewhat resilient data flow scheme they're proud of that has > thousands and tens of thousands of nodes hooked up through > Python, where the protocol between the nodes is Pyro. Not, > "uses Pyro", "IS Pyro". > > I'm supposed to tell these people what would be a good stripped > down kernel to use (or what would be a good kernel to strip > further), and I'm tempted to start by saying "strip away Python" > which certainly won't make me popular (;-). But thinking about > error rates and million-year bugs has me thinking harder. > > It seems as if we have to think in terms of *expecting* the > kernel itself to be unreliable-at-scale, so that something like > JailHouse might be the right level to start. > > Does anyone have any experience with trying to harden a large system > against faults in the distribution layer and in the kernel, and have > any advice they'd care to share? > > None of this is Erlang-specific, it's just that I think the Erlang > community are likely to have more relevant experience than most. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > The expectation seems to be that microkernels would eventually replace monolithic kernels and the change would provide better reliability and security. The two most accessible, well-known, and complete approaches appear to be: 1) seL4 which is open-source and formally verified (https://en.wikipedia.org/wiki/L4_microkernel_family) available at https://github.com/seL4/seL4 2) MINIX 3 (https://en.wikipedia.org/wiki/MINIX_3) available at http://www.minix3.org/ For normal UNIX use of an operating system you would likely need to go with MINIX 3 and use its ability to install from the NetBSD ports tree. MINIX has a history of mainly being used for teaching, so this approach is likely not something most people would agree with immediately. There are other attempts to pursue microkernels but it doesn't appear like they have been able to receive much attention and keep their SLOC low. A more typical choice would be FreeBSD instead of Linux (though both are monolithic kernels) since FreeBSD is perceived as being more reliable (with the explanation that features are added to Linux at a much quicker rate which includes the addition of bugs). From Andras.Bekes@REDACTED Tue Mar 28 09:52:06 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Tue, 28 Mar 2017 07:52:06 +0000 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <40BB09D0-9D3E-4A5C-81DA-D2BE70B383C3@cs.otago.ac.nz> References: <40BB09D0-9D3E-4A5C-81DA-D2BE70B383C3@cs.otago.ac.nz> Message-ID: Thank you very much for the heads up. I am happy with a non-portable implementation. I only need to target Linux. I suspect the below holds for unix domain specific paths in general. The abstract namespace feature seems to be Linux-specific. -----Original Message----- From: Richard A. O'Keefe [mailto:ok@REDACTED] Sent: Monday, March 27, 2017 11:12 PM To: Bekes, Andras G (IST) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes > On 28/03/2017, at 5:03 AM, Bekes, Andras G wrote: > I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. Be careful there. OSX 10.11.3 manual page: UNIX-domain addresses are variable-length filesystem pathnames of at most 104 characters. The Single Unix Specification, version 4, section un.h, says The size of sun_path has intentionally been left undefined. This is because different implementations use different sizes. For example, 4.3 BSD uses a size of 108, and 4.4 BSD uses a size of 104. Since most implementations originate from BSD versions, the size is typically in the range 92 to 108. Applications should not assume a particular length for sun_path or assume that it can hold {_POSIX_PATH_MAX} bytes (256). Expect the last byte to be used for the NUL terminator of a C string, so the length of the actual name could be limited to 91..107. -------------------------------------------------------------------------------- From Andras.Bekes@REDACTED Tue Mar 28 09:54:52 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Tue, 28 Mar 2017 07:54:52 +0000 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> Message-ID: I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux. Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109. From: Alex S. [mailto:alex0player@REDACTED] Sent: Monday, March 27, 2017 8:00 PM To: Bekes, Andras G (IST) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes I have a strong suspicion the binary might be automatically zero-terminated. 27 ????? 2017 ?., ? 19:03, Bekes, Andras G > ???????(?): Hi All, I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). http://man7.org/linux/man-pages/man7/unix.7.html Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: > Sun_path_length=108. 108 > UDS_path = <<"whatever">>. <<"whatever">> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). 792 > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,...>>} > Port = 0. 0 > gen_tcp:connect(Address,Port,[local],infinity). (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} ** exception exit: badarg in function gen_tcp:connect/4 (gen_tcp.erl, line 149) (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. Seems to me that this is a bug in the Erlang implementation. What do you think? Thank you very much, Andras G. Bekes _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Mar 28 12:05:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 28 Mar 2017 12:05:09 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: Message-ID: <20170328100509.GA94857@erix.ericsson.se> On Mon, Mar 27, 2017 at 04:03:40PM +0000, Bekes, Andras G wrote: > Hi All, > > I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. > If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). > http://man7.org/linux/man-pages/man7/unix.7.html That is correct. The third argument shall always be the size of the address structure. > > Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: Yes. Since it is so unspecified whether the sun_path should be zero terminated or not, inet_drv tries to do something sensible. It first zeros the whole sun_path, which on linux is 108 bytes, and then copies the whole given address binary into it. But if the binary is longer than 107 bytes it is rejected since that might overwrite the last zero, which might be dangerous if the kernel was expecting a zero terminated address. And the length is set to the length of the binary (without zero termination) since that seems to be expected for file names. I see no good way to know for all platforms if the address is supposed to be zero terminated or not. Linux utilizes the special case that if the first byte is zero then it is not zero terminated. I do not know if that is a safe indicator on all platforms. An fix that sounds safe and would work in this case is to check if the last byte is zero then allow it to fill sun_path and use its actual length. A riskier fix would be to allow it to fill sun_path and use its acual length if the first byte is zero since that is what Linux says. Riskier since *BSD says it should be zero terminated and the calculated length to use should not count the zero termination. / Raimo Niskanen, Erlang/OTP > > > Sun_path_length=108. > 108 > > > UDS_path = <<"whatever">>. > <<"whatever">> > > > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). > 792 > > > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. > {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,...>>} > > > Port = 0. > 0 > > > gen_tcp:connect(Address,Port,[local],infinity). > (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) > (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) > (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) > (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} > ** exception exit: badarg > in function gen_tcp:connect/4 (gen_tcp.erl, line 149) > (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} > (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} > > If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: > connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) > When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. > > Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. > Seems to me that this is a bug in the Erlang implementation. What do you think? > > Thank you very much, > > Andras G. Bekes > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlang@REDACTED Tue Mar 28 12:45:40 2017 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 28 Mar 2017 12:45:40 +0200 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> Message-ID: You're quite right - you need to know in advance the checksum of the data. But obtaining the checksum from "somewhere" is orthogonal to the problem of storing or obtaining the data. Indexing etc. is completely different problem. /Joe On Mon, Mar 27, 2017 at 11:23 PM, Richard A. O'Keefe wrote: > >> On 27/03/2017, at 4:31 AM, Joe Armstrong wrote: >> To store , then later >> >> GET /blob/sha256/ >> >> to recover the blob. is the SHA256 checksum of the data (the >> path contains the >> type of the checksum - so you might say GETblob/md5/ > > It seems to me that this amounts to using a checksum of the contents > of a file instead of an inode number. > > What I don't understand is how I would ever use this. > When I go looking for a file, I want its *contents*, > which I don't know, otherwise I wouldn't need it. > So I *can't* compute its checksum. With things like > Spotlight, I can use a known *part* of the content to > look for the rest of the content, but again, if I > already knew enough of the content to compute the > checksum, I wouldn't bother looking at the file. > > The only way I can think of to use something like this > is to maintain some sort of name-based directory > structure on top, or some sort of IR-like inverted index > based on part of the content. > > From jesper.louis.andersen@REDACTED Tue Mar 28 17:46:00 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 28 Mar 2017 15:46:00 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: Message-ID: On Fri, Mar 24, 2017 at 11:37 PM Dmitry Kakurin wrote: > Two of them that seem especially useful are *underspecs *and *overspecs*. > > If you think about it, an overspecification is not really a problem with the code. The dialyzer uses inference analysis and makes a best effort at determining the type of a given function. Your contract specifies your usage of the function. When a overspec happens, it is because your contract is more restrictive than the inference. This is usually not a problem. A small change in your function body might change the inferred type entirely. But unless you want to go change your contract all the time, it may be better to keep it more restrictive. Lots of functions are more general than their intended use. Another problem is when the dialyzer fails to make a precise inference, so falls back to type any(). Enabling overspecs will make the system complain all the time in this situation, but it may not be a problem. Underspecs, on the other hand, could spell trouble. Here the inference is more restrictive, and thus there are inputs to the function which is within the contract, but won't make the code terminate as expected according to the inference scheme. If your code never makes use of the extended contract, there is no problem, but as soon as some user tries to do so, things will go bad. The dialyzer will usually catch such a use quickly, but it is nicer to be strict in the contract if possible. There are, however, situations where underspecs isn't that desirable as a property either. Suppose you are returning a map, such as #{ a => 37 }. The dialyzer can infer its success type to be #{ a := pos_integer() }[0]. But you may have written a specification where you say a map() is returned. This is more general than the inferred type, and as such it is an underspec violation. Your contract isn't wrong, but on the other hand, it isn't precise either. A common problem that also hits a lot are when you are declaring a binary(), but the dialyzer is able to figure out a more precise type scheme, for instance <<_:64, _:_*24>>. You will perhaps be more happy with just saying "this is a binary". My usual recommendation is to start with the default set of warnings. Remove every error there, because the slogan is The dialyzer is never wrong! and this is absolutely true. Once you have a dialyzer clean piece of code with the default warnings, you start turning on underspecs, no_return, and so on. But often this is more for understanding if there is anything that sticks out as being too general. [0] This is strictly not true. If you *always* return 37, then it figures out the type is exactly the same response every time and tags its type as #{ a := 37 }. But for this exposition, I'm assuming your code returns "some positive integer" where 37 is an example. -------------- next part -------------- An HTML attachment was scrubbed... URL: From krzysztof.jurewicz@REDACTED Tue Mar 28 19:26:38 2017 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Tue, 28 Mar 2017 19:26:38 +0200 Subject: [erlang-questions] [ANN] gb_merkle_trees Message-ID: <8760ituxbl.fsf@gmail.com> gb_merkle_trees has been released. It provides a dictionary-like storage for binaries using general balanced binary Merkle trees. The library is available at: https://github.com/KrzysiekJ/gb_merkle_trees Its interface and mechanism are similar to stdlib?s gb_trees, with a notable difference that all trees are also Merkle trees. What follows, values are stored only on leaf nodes. Having an agreement only on the root hash of a tree, one can prove that a particular key and value combination belongs to the tree. Applications include storing states for blockchains ? see Tendermint?s Application Blockchain Interface which represents application state as Merkle root hash ( https://github.com/tendermint/abci ). Snapshot of the documentation for 0.1.0 release has been put on IPFS: http://ipfs.io/ipfs/QmUsr6wsULroXXh6kD8nGjWQQierck1uoe1GMQaVxETcaJ/ Patches, comments and other feedback are welcome. From dima_kakurin@REDACTED Tue Mar 28 19:29:59 2017 From: dima_kakurin@REDACTED (Dmitry Kakurin) Date: Tue, 28 Mar 2017 17:29:59 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: , Message-ID: > What you want is provided as a default behaviour. Hi Alex, My observations don't match your statement above: * Removing -Wunderspecs switch makes warning in both under_ methods disappear * Removing -Woverspecs switch makes warning in both over_ methods disappear * Removing both -W makes all 4 warnings disappear And I'll reply to Jasper's email on usefulness of both underspecs and overspecs. Thank you, Dmitry. ________________________________ From: Alex S. Sent: Monday, March 27, 2017 1:09 AM To: Dmitry Kakurin Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Dialyzer underspecs and overspecs options What you want is provided as a default behaviour. Overspecs you shouldn?t really enable for production needs, it?s ?debug the Dialyzer? feature. 24 ????? 2017 ?., ? 21:42, Dmitry Kakurin > ???????(?): Hi all, I've started using Dialyzer and tried to enable as many warnings as possible. Two of them that seem especially useful are underspecs and overspecs. But after struggling with warnings produced by them for a while I've realized that what I really need is a subset of each. In the example below (sorry, I'm using Elixir syntax but hope it's still obvious) the 2 warnings I want to enable are under_in and over_out. They are the ones that will break callers relying on the spec. While over_in and under_out warnings are more of a hints to the function author about current implementation and are much less serious. The current Dialyzer options allow me to enable either both over_* warnings or both under_* warnings. But that's not useful. Is there a way to configure Dialyzer the way I want? Here is the demo code: # suboptimal implementation, fine for caller @spec over_in(x :: integer) :: nil def over_in(x) when is_number(x) do nil end # broken caller contract, could return unexpected value @spec over_out(x :: number) :: integer def over_out(x) when is_number(x) do x end # broken caller contract, rejects input allowed by the spec @spec under_in(x :: number) :: nil def under_in(x) when is_integer(x) do nil end # current implementation detail, does not concern caller @spec under_out(x :: integer) :: number def under_out(x) when is_integer(x) do x end and here are the warnings: demo.ex:4: Type specification 'Elixir.Virt':over_in(x::integer()) -> 'nil' is a subtype of the success typing: 'Elixir.Virt':over_in(number()) -> 'nil' demo.ex:9: Type specification 'Elixir.Virt':over_out(x::number()) -> integer() is a subtype of the success typing: 'Elixir.Virt':over_out(number()) -> number() demo.ex:14: Type specification 'Elixir.Virt':under_in(x::number()) -> 'nil' is a supertype of the success typing: 'Elixir.Virt':under_in(integer()) -> 'nil' demo.ex:19: The specification for 'Elixir.Virt':under_out/1 states that the function might also return float() but the inferred return is integer() Thanks, Dmitry. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dima_kakurin@REDACTED Tue Mar 28 21:03:19 2017 From: dima_kakurin@REDACTED (Dmitry Kakurin) Date: Tue, 28 Mar 2017 19:03:19 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: , Message-ID: Hi Jesper, We need to separate what constitutes a specified contract violation for function inputs and function outputs. I have reworked my demo example (below) and produced warnings (also below) to make it absolutely clear. Let SpecIn be a set of @spec inputs and RealIn be a set of inputs as inferred by Dialyzer from real code, then: * The Input Contract is satisfied when SpecIn <= RealIn (where <= is a non-strict subset operation). See over_in in demo code below. * The Input Contract violation is detected by -Wunderspecs option when SpecIn > RealIn. See under_in below. It is easy to see in the code: * It's OK for over_in to declare that it only accepts :a and :b while it also happens to accept :c. Maybe suboptimal, but fine. * It's NOT OK for under_in to claim that it accepts :a, :b and :c and break if :c is passed. Rejecting :c would break the caller. Let SpecOut be a set of @spec outputs and RealOut be a set of outputs as inferred by Dialyzer from real code, then: * The Output Contract is satisfied when SpecOut >= RealOut (where >= is a non-strict superset operation). See under_out below. * The Output Contract violation is detected by -Woverspecs option when SpecOut < RealOut. See over_out below. It is easy to see in the code: * It's OK for under_out to declare that it returns :a, :b and :c, while currently it only returns :a and :b. Maybe future implementations will return :c as well. * It's NOT OK for over_out to declare that it returns :a and :b, but to also return :c sometimes. Returning :c would break the caller. We can see the perfect asymmetry for inputs and outputs. These are pretty-much covariance and contravariance rules: you cannot accept less than you claim and you cannot produce more than you claim. Also we see how: * -Wunderspecs option detects Input Contract violations and produces false positives for acceptable Output contracts * -Woverspecs option detects Output Contract violations and produces false positives for acceptable Input contracts That's why I'd like to enable underspec_in and overspec_out options (if they existed), as they are the 2 options detecting real problems (out of 4 possible combinations). # suboptimal implementation, fine for caller @spec over_in(x :: :a | :b) :: nil def over_in(x) when x == :a or x == :b or x == :c do nil end # BROKEN caller contract, rejects input allowed by the spec @spec under_in(x :: :a | :b | :c) :: nil def under_in(x) when x == :a or x == :b do nil end # current implementation detail, does not concern caller @spec under_out() :: :a | :b | :c def under_out() do produce_ab() end # BROKEN caller contract, could return unexpected value @spec over_out() :: :a | :b def over_out() do produce_abc() end # helpers @spec produce_ab() :: :a | :b def produce_ab() do case :rand.uniform(2) do 1 -> :a _ -> :b end end @spec produce_abc() :: :a | :b | :c def produce_abc() do case :rand.uniform(3) do 1 -> :a 2 -> :b _ -> :c end end and the warnings are: demo.ex:6: Type specification 'Elixir.Virt':over_in('a' | 'b') -> 'nil' is a subtype of the success typing: 'Elixir.Virt':over_in('a' | 'b' | 'c') -> 'nil' demo.ex:12: Type specification 'Elixir.Virt':under_in('a' | 'b' | 'c') -> 'nil' is a supertype of the success typing: 'Elixir.Virt':under_in('a' | 'b') -> 'nil' demo.ex:18: The specification for 'Elixir.Virt':under_out/0 states that the function might also return 'c' but the inferred return is 'a' | 'b' demo.ex:24: Type specification 'Elixir.Virt':over_out() -> 'a' | 'b' is a subtype of the success typing: 'Elixir.Virt':over_out() -> 'a' | 'b' | 'c' Thank you, Dmitry ________________________________ From: Jesper Louis Andersen Sent: Tuesday, March 28, 2017 8:46:00 AM To: Dmitry Kakurin; erlang-questions@REDACTED Subject: Re: [erlang-questions] Dialyzer underspecs and overspecs options On Fri, Mar 24, 2017 at 11:37 PM Dmitry Kakurin > wrote: Two of them that seem especially useful are underspecs and overspecs. If you think about it, an overspecification is not really a problem with the code. The dialyzer uses inference analysis and makes a best effort at determining the type of a given function. Your contract specifies your usage of the function. When a overspec happens, it is because your contract is more restrictive than the inference. This is usually not a problem. A small change in your function body might change the inferred type entirely. But unless you want to go change your contract all the time, it may be better to keep it more restrictive. Lots of functions are more general than their intended use. Another problem is when the dialyzer fails to make a precise inference, so falls back to type any(). Enabling overspecs will make the system complain all the time in this situation, but it may not be a problem. Underspecs, on the other hand, could spell trouble. Here the inference is more restrictive, and thus there are inputs to the function which is within the contract, but won't make the code terminate as expected according to the inference scheme. If your code never makes use of the extended contract, there is no problem, but as soon as some user tries to do so, things will go bad. The dialyzer will usually catch such a use quickly, but it is nicer to be strict in the contract if possible. There are, however, situations where underspecs isn't that desirable as a property either. Suppose you are returning a map, such as #{ a => 37 }. The dialyzer can infer its success type to be #{ a := pos_integer() }[0]. But you may have written a specification where you say a map() is returned. This is more general than the inferred type, and as such it is an underspec violation. Your contract isn't wrong, but on the other hand, it isn't precise either. A common problem that also hits a lot are when you are declaring a binary(), but the dialyzer is able to figure out a more precise type scheme, for instance <<_:64, _:_*24>>. You will perhaps be more happy with just saying "this is a binary". My usual recommendation is to start with the default set of warnings. Remove every error there, because the slogan is The dialyzer is never wrong! and this is absolutely true. Once you have a dialyzer clean piece of code with the default warnings, you start turning on underspecs, no_return, and so on. But often this is more for understanding if there is anything that sticks out as being too general. [0] This is strictly not true. If you *always* return 37, then it figures out the type is exactly the same response every time and tags its type as #{ a := 37 }. But for this exposition, I'm assuming your code returns "some positive integer" where 37 is an example. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Mar 29 01:20:47 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 28 Mar 2017 23:20:47 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: Message-ID: On Tue, Mar 28, 2017 at 9:03 PM Dmitry Kakurin wrote: > These are pretty-much *c**ovariance* and *contravariance* rules: you *cannot > **accept less *than you claim and you *cannot **produce more* than you > claim. > > > Ah of course. There is no way to specify the "cross" of the two options as of now I think. What you want is basically to check for contravariance in inputs and covariance in outputs if I read what you say correctly. This is normal errors for a subtype hierarchy where you have functions as first class values. That is the relation is that S1 -> T1 <: S2 -> T2 Given that T1 <: T2 (covariance) S2 <: S1 (contravariance) Do note that the dialyzer is an approximate analysis however. Consider these functions: f(a) -> true; f(b) -> false; f(c) -> true. g(a) -> true; g(b) -> true; g(c) -> false; g(d) -> true; g(e) -> false; g(f) -> true; g(g) -> true; g(h) -> true; g(i) -> false; g(j) -> false; g(k) -> true; g(l) -> true; g(m) -> false; g(n) -> true; g(o) -> false. The dialyzer infers the following types for these: -spec f('a' | 'b' | 'c') -> boolean(). -spec g(atom()) -> boolean(). The point is that 'g' contains "too much" information so the dialyzer just moves one up in the type lattice rather than running with a precise specification. Clearly g/1 fails if called as 'g(x)' for instance, but this is not represented in the type specification. Rather, the dialyzer has approximated that the function is probably from atom() to boolean(). This is what usually makes underspecs/overspecs a bit like a false positive in the system. As long as you travel along a subtyping chain in the lattice, you may be witnessing an approximation. Where we are sure of a mistake is if you suddenly have an input that doesn't match the type lattice at all. Say we called 'g(<<>>)' which would obviously be wrong since g doesn't accept binary() as input. It may also be worth mentioning that a success type f : A -> B is defined in a specific manner: Suppose f(X) reduces to a value V. Then V is among type B and X is among type A. It is subtly different from a standard static type system. Note that 'g/1' above fulfills this notion: if it terminates with a boolean() value, then its input was definitely an atom(). Also note that the succes type any() -> any is valid for 'g/1'. But it doesn't establish a relation of being well-typed and thus having certain desirable meta-theoretic properties. On the other hand, the key aspect is that while it over-approximates, we can say something about a function application such as g(<<>>) because we know a priori it will fail. This is program defect and we better report such defects! If you are interested, Lindahl and Sagonas has a paper on the implementation, and its exposition is better than mine given here: https://it.uu.se/research/group/hipe/papers/succ_types.pdf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Mar 29 03:08:42 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 29 Mar 2017 14:08:42 +1300 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170328100509.GA94857@erix.ericsson.se> References: <20170328100509.GA94857@erix.ericsson.se> Message-ID: <44C5A8B1-523A-4B56-AE50-6162FC49B073@cs.otago.ac.nz> > On 28/03/2017, at 11:05 PM, Raimo Niskanen wrote: > I see no good way to know for all platforms if the address is supposed to > be zero terminated or not. Linux utilizes the special case that if the > first byte is zero then it is not zero terminated. I do not know if that > is a safe indicator on all platforms. The Linux manual page is quite specific: "a Unix domain socket can be bound to a >>>null-terminated<<< file system pathname" "an abstract socket address" begins with a NUL byte. "All of the remaining bytes define the 'name' of the socket." "The abstract socket namespace is a non-portable Linux extension." Sounds like any "fix" in Erlang's guts would need to be Linux-specific and leading NUL bytes should probably be reported as errors on other systems. From mbike2000ru@REDACTED Wed Mar 29 07:22:00 2017 From: mbike2000ru@REDACTED (Dmitry) Date: Wed, 29 Mar 2017 05:22:00 +0000 (UTC) Subject: [erlang-questions] How to save Socket data in ETS References: <282457631.6231291.1490764920705.ref@mail.yahoo.com> Message-ID: <282457631.6231291.1490764920705@mail.yahoo.com> Hello, What is correct way to save tcp socket data in ETS? I tried this way: ets:insert(socket, {"79514484992",#Port<0.11255>}), But it gives "syntax error". When I place commas around - everything is ok: ets:insert(socket, {"79514484992",'#Port<0.11255>'}) - this is ok, but how to add commas around Port? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 29 08:09:01 2017 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 29 Mar 2017 06:09:01 +0000 Subject: [erlang-questions] How to save Socket data in ETS In-Reply-To: <282457631.6231291.1490764920705@mail.yahoo.com> References: <282457631.6231291.1490764920705.ref@mail.yahoo.com> <282457631.6231291.1490764920705@mail.yahoo.com> Message-ID: Hi, Do not type it in Erlang shell but use variable bound to Port. #Port<0.11255> is a way how Erlang visualize Port value but it is not a Port value. You can't make Port from "#Port<0.11255>" which is what Erlang shell received from your keyboard. You have to use what Erlang shell received from (erlang:)open_port/2 and then transformed to "#Port<0.11255>" and sent to IO server which shows it to the terminal. You have to bound it to the variable or use v(N) pseudofunction. Hynek Vychodil On Wed, Mar 29, 2017 at 7:28 AM Dmitry wrote: > Hello, > > What is correct way to save tcp socket data in ETS? > > I tried this way: > > ets:insert(socket, {"79514484992",#Port<0.11255>}), But it gives "syntax > error". > > When I place commas around - everything is ok: > > ets:insert(socket, {"79514484992",'#Port<0.11255>'}) - this is ok, but how > to add commas around Port? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Mar 29 11:55:21 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 29 Mar 2017 11:55:21 +0200 Subject: [erlang-questions] Multiple application configurations in multiple files Message-ID: Hello! I'm looking into the following corner case: what if the Erlang VM is started with multiple configuration files, at least one of the containing multiple configurations for the same application? For example I have these two configuration files: 3.config: [ {sasl, [ {key, value3}, {key3, value3} ]}, {sasl, [ {key, value3}, {key3, value3} ]} ]. 4.config: [ {sasl, [ {key, value4}, {key4, value4} ]} ]. I can start erlang with this command, load the sasl application and check the environment: erl -config 3 -config 4 Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.3 (abort with ^G) 1> application:load(sasl). ok 2> application:get_all_env(sasl). [{key,value3}, {key3,value3}, {sasl_error_logger,tty}, {included_applications,[]}, {errlog_type,all}] Based on the comments in application_controller.erl ("Env2 overrides Env1"), I expected to get {key, value3}, {key3, value3} and {key4, value4} in the environment, not {key, value3} and {key3, value3}. Interestingly, when there's only one sasl configuration in 3.config, I do get the merged environment: 2> application:get_all_env(sasl). [{key,value4}, {key3,value3}, {sasl_error_logger,tty}, {key4,value4}, {included_applications,[]}, {errlog_type,all}] Is it a bug or a feature that if there's multiple configuration for the same application in a config file, only the last one is used? From yueyoum@REDACTED Wed Mar 29 12:29:33 2017 From: yueyoum@REDACTED (=?UTF-8?B?5pyI5b+n6IyX?=) Date: Wed, 29 Mar 2017 18:29:33 +0800 Subject: [erlang-questions] [ANN] eipp: A C++11 Header Only Library for using erlang ei library. Message-ID: https://github.com/yueyoum/eipp This library to decode/encode erlang extern format more convenient. Encode: you can encode std::list, std::vector, std::map, std::tuple to ext driectly. e.g. auto data = std::make_tuple(1, "abc"); encoder.encoder(data); std::string output = encoder.get_data(); Decode: Just define the schema of the ext structs. and just one api call. e.g. using T = eipp::Tuple>; auto result = decoder.parse(); More examples in the github link above. -- My GitHub https://github.com/yueyoum -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Wed Mar 29 13:53:51 2017 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 29 Mar 2017 13:53:51 +0200 Subject: [erlang-questions] simple virtual file system in Erlang? In-Reply-To: References: <666762DD-812F-42CA-9729-53A9C5EE99EF@laposte.net> <32FFD42D-076A-45E1-BC69-0D270447FEB9@cs.otago.ac.nz> Message-ID: Two papers worth reading when thinking about storage and file systems: Venti, a content addressable archival storage: http://doc.cat-v.org/plan_9/4th_edition/papers/venti/ Fossil, a filesystem that can use venti for snapshot archives: http://doc.cat-v.org/plan_9/4th_edition/papers/fossil/ On 28 March 2017 at 12:45, Joe Armstrong wrote: > You're quite right - you need to know in advance the checksum of the data. > > But obtaining the checksum from "somewhere" is orthogonal to the problem > of storing or obtaining the data. Indexing etc. is completely different > problem. > > /Joe > > On Mon, Mar 27, 2017 at 11:23 PM, Richard A. O'Keefe > wrote: > > > >> On 27/03/2017, at 4:31 AM, Joe Armstrong wrote: > >> To store , then later > >> > >> GET /blob/sha256/ > >> > >> to recover the blob. is the SHA256 checksum of the data (the > >> path contains the > >> type of the checksum - so you might say GETblob/md5/ > > > > It seems to me that this amounts to using a checksum of the contents > > of a file instead of an inode number. > > > > What I don't understand is how I would ever use this. > > When I go looking for a file, I want its *contents*, > > which I don't know, otherwise I wouldn't need it. > > So I *can't* compute its checksum. With things like > > Spotlight, I can use a known *part* of the content to > > look for the rest of the content, but again, if I > > already knew enough of the content to compute the > > checksum, I wouldn't bother looking at the file. > > > > The only way I can think of to use something like this > > is to maintain some sort of name-based directory > > structure on top, or some sort of IR-like inverted index > > based on part of the content. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Mar 29 15:29:27 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 29 Mar 2017 15:29:27 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <44C5A8B1-523A-4B56-AE50-6162FC49B073@cs.otago.ac.nz> References: <20170328100509.GA94857@erix.ericsson.se> <44C5A8B1-523A-4B56-AE50-6162FC49B073@cs.otago.ac.nz> Message-ID: <20170329132927.GA19866@erix.ericsson.se> On Wed, Mar 29, 2017 at 02:08:42PM +1300, Richard A. O'Keefe wrote: > > > On 28/03/2017, at 11:05 PM, Raimo Niskanen wrote: > > I see no good way to know for all platforms if the address is supposed to > > be zero terminated or not. Linux utilizes the special case that if the > > first byte is zero then it is not zero terminated. I do not know if that > > is a safe indicator on all platforms. > > The Linux manual page is quite specific: > "a Unix domain socket can be bound to a >>>null-terminated<<< > file system pathname" > > "an abstract socket address" begins with a NUL byte. "All of the > remaining bytes define the 'name' of the socket." "The abstract > socket namespace is a non-portable Linux extension." > > Sounds like any "fix" in Erlang's guts would need to be Linux-specific > and leading NUL bytes should probably be reported as errors on other > systems. That is uncomfortable since the spirit of configure tests is to test for features and to not rely on a particular system name implying a feature. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From hellkvist@REDACTED Wed Mar 29 21:55:16 2017 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Wed, 29 Mar 2017 21:55:16 +0200 Subject: [erlang-questions] REPLACE_OS_VARS in erlexec? Message-ID: Hello Erlangers, For different reasons (for instance to be able to run distributed Erlang inside Kubernetes for instance, like here: https://substance.brpx.com/clustering-elixir-nodes-on-kubernetes-e85d0c26b0cf), there is a need to be able to override or replace the settings in vm.args (or the args_file if you use the name of the argument to erlexec) with environment variables. For instance like so (example taken from the above article): ----begin vm.args---- ## Name of the node -name ${MY_POD_NAMESPACE}@${MY_POD_IP} ## Cookie for distributed erlang -setcookie ${ERLANG_COOKIE} ... ----end vm.args----- The need for this functionality seems to be big enough for there to now be support for it in the release builders (relx for instance offers a way to turn on vm.args environment variable replacement with the RELX_REPLACE_OS_VARS environment variable when running the start script). My question though is if this functionality (to set variables such as -name or -setcookie in args_file through environment variables) has ever been considered for erlexec itself? For various reasons (mostly to reduce dependencies to other programs such as various shell utilities) I would like to start my erlang nodes without the generated release script by calling erlexec directly, but I realize I would, by doing so, miss out on the power to set variables in arg-file with environment variables (like in relx above). I guess it would not be extremely difficult to modify erlexec for this purpose (erlexec already has functions for reading environment variables so it would be a small change I guess) but I'm not sure if there would be enough interest in this feature to accept a pull-request and I would not like to roll my own fork of OTP only for this rather limited purpose. So, does anyone know if this this feature has ever been discussed or considered for inclusion in OTP? /Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Mar 30 00:42:31 2017 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 29 Mar 2017 15:42:31 -0700 Subject: [erlang-questions] [ANN] CloudI 1.7.0 Released! Message-ID: Download 1.7.0 from http://sourceforge.net/projects/cloudi/files/latest/download (checksums at the bottom of this email) CloudI (http://cloudi.org/) is a "universal integrator" using an Erlang core to provide fault-tolerance with efficiency and scalability. The CloudI API provides a minimal interface to communicate among services so programming language agnostic and protocol agnostic integration can occur. CloudI currently integrates with the programming languages: C/C++, Elixir, Erlang, Go, Haskell, Java, JavaScript/node.js, OCaml, PHP, Perl, Python, and Ruby, Many reusable services are included that rely on the CloudI service bus. The details for this release are below: * backwards compatibility difference: * The C CloudI API (not the C++ CloudI API) had a small change to facilitate the use of thread local data with the callback function arguments order changing to match the other programming languages that lack (or avoid) state objects (C, Erlang, Go, Haskell, OCaml). All CloudI API callback functions have the arguments: Type, Name, Pattern, RequestInfo, Request, Timeout, Priority, TransId, Source (in this order). The programming languages with state objects have the two arguments: State, API, before the common arguments with State as a state object (normally not present as a function argument). The programming languages without state objects have the two arguments (State, API) after the common arguments. The C examples have been updated to show the changes (http://cloudi.org/tutorials.html#cloudi_examples). More function argument discussion is at http://cloudi.org/faq.html#1_Reactive * The CloudI API implementations had the function request_http_qs_parse name change to info_key_value_parse * The Go CloudI API was added (requires the configure argument --enable-go-support and Go (gc) >= 1.6) * The Haskell CloudI API was added (requires the configure argument --enable-haskell-support and GHC >= 7.10.3, cabal-install >= 1.22) * The OCaml CloudI API was added (requires the configure argument --enable-ocaml-support and OCaml >= 4.03.0) * The CloudI Service API function logging_stdout_set was added to allow all CloudI logging to be sent to stdout * The count integration test was added to show how thread local state works in all supported programming languages (http://cloudi.org/tutorials.html#cloudi_examples). * A bug with CloudI Service API services_update usage on external services was fixed, to allow the service configuration values: timeout_async, timeout_sync, and the service configuration options: priority_default, request_timeout_adjustment, to dynamically change during runtime if no new OS process is created (to utilize the new values). * A bug was fixed to make the use of the service configuration option count_process_dynamic more dependable for external services written in Perl, PHP, Python, and Ruby. * lager was removed; hut is now used for exometer logging. Erlang logging integration was added to the modules cloudi_logger_lager and cloudi_logger_hut (to use CloudI logging instead of lager or with hut) * The default external service configuration buffer_size setting changed from 16384 to 65536 (to match the same Linux localhost MTU change) * UTF8 is now logged properly from Erlang source code * Bugs were fixed and other improvements were added (see the ChangeLog for more detail) Please mention any problems, issues, or ideas! Thanks, Michael SHA256 CHECKSUMS cloudi-1.7.0.tar.gz (16571701 bytes) 05ca9bb7772d11f26a8cceb364c32b50bb4f37b158d8dee801f621f8c828c7a6 cloudi-1.7.0.tar.bz2 (13504415 bytes) f626e85dce7896cad3cf5338c7e80ad6b0291d751602fbfaa7d17ae621408b21 From ok@REDACTED Thu Mar 30 01:12:15 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 30 Mar 2017 12:12:15 +1300 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170329132927.GA19866@erix.ericsson.se> References: <20170328100509.GA94857@erix.ericsson.se> <44C5A8B1-523A-4B56-AE50-6162FC49B073@cs.otago.ac.nz> <20170329132927.GA19866@erix.ericsson.se> Message-ID: <8BB800D5-4A36-4483-9675-18F976607FA0@cs.otago.ac.nz> I wrote that any support in Erlang for the 'abstract socket namespace' of Linux would have to depend on the system *being* Linux. > On 30/03/2017, at 2:29 AM, Raimo Niskanen wrote: > > That is uncomfortable since the spirit of configure tests is to test for > features and to not rely on a particular system name implying a feature. Where did I say anything about testing for a system name? What I meant is simply that *IF* Erlang ever supports this Linux feature (and I do not say that it should; the last time I wanted to write system-specific code was the last day I programmed a DEC-10) it should take care to check for a leading NUL byte and give an informative error on systems that DON'T support this "feature". I have no idea what the best way to detect such support might be. What I cared about is that Erlang shouldn't just go ahead and accept leading NUL bytes on all systems because someone might be expecting the Linux "feature", so an informative error message is needed. From dima_kakurin@REDACTED Thu Mar 30 03:52:29 2017 From: dima_kakurin@REDACTED (Dmitry Kakurin) Date: Thu, 30 Mar 2017 01:52:29 +0000 Subject: [erlang-questions] Dialyzer underspecs and overspecs options In-Reply-To: References: , Message-ID: > There is no way to specify the "cross" of the two options as of now I think. How realistic do you think it would be to add this functionality to Dialyzer? Thank you, Dmitry. ________________________________ From: Jesper Louis Andersen Sent: Tuesday, March 28, 2017 4:20:47 PM To: Dmitry Kakurin; erlang-questions@REDACTED Subject: Re: [erlang-questions] Dialyzer underspecs and overspecs options On Tue, Mar 28, 2017 at 9:03 PM Dmitry Kakurin > wrote: These are pretty-much covariance and contravariance rules: you cannot accept less than you claim and you cannot produce more than you claim. Ah of course. There is no way to specify the "cross" of the two options as of now I think. What you want is basically to check for contravariance in inputs and covariance in outputs if I read what you say correctly. This is normal errors for a subtype hierarchy where you have functions as first class values. That is the relation is that S1 -> T1 <: S2 -> T2 Given that T1 <: T2 (covariance) S2 <: S1 (contravariance) Do note that the dialyzer is an approximate analysis however. Consider these functions: f(a) -> true; f(b) -> false; f(c) -> true. g(a) -> true; g(b) -> true; g(c) -> false; g(d) -> true; g(e) -> false; g(f) -> true; g(g) -> true; g(h) -> true; g(i) -> false; g(j) -> false; g(k) -> true; g(l) -> true; g(m) -> false; g(n) -> true; g(o) -> false. The dialyzer infers the following types for these: -spec f('a' | 'b' | 'c') -> boolean(). -spec g(atom()) -> boolean(). The point is that 'g' contains "too much" information so the dialyzer just moves one up in the type lattice rather than running with a precise specification. Clearly g/1 fails if called as 'g(x)' for instance, but this is not represented in the type specification. Rather, the dialyzer has approximated that the function is probably from atom() to boolean(). This is what usually makes underspecs/overspecs a bit like a false positive in the system. As long as you travel along a subtyping chain in the lattice, you may be witnessing an approximation. Where we are sure of a mistake is if you suddenly have an input that doesn't match the type lattice at all. Say we called 'g(<<>>)' which would obviously be wrong since g doesn't accept binary() as input. It may also be worth mentioning that a success type f : A -> B is defined in a specific manner: Suppose f(X) reduces to a value V. Then V is among type B and X is among type A. It is subtly different from a standard static type system. Note that 'g/1' above fulfills this notion: if it terminates with a boolean() value, then its input was definitely an atom(). Also note that the succes type any() -> any is valid for 'g/1'. But it doesn't establish a relation of being well-typed and thus having certain desirable meta-theoretic properties. On the other hand, the key aspect is that while it over-approximates, we can say something about a function application such as g(<<>>) because we know a priori it will fail. This is program defect and we better report such defects! If you are interested, Lindahl and Sagonas has a paper on the implementation, and its exposition is better than mine given here: https://it.uu.se/research/group/hipe/papers/succ_types.pdf -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Mar 30 08:44:33 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 30 Mar 2017 08:44:33 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <8BB800D5-4A36-4483-9675-18F976607FA0@cs.otago.ac.nz> References: <20170328100509.GA94857@erix.ericsson.se> <44C5A8B1-523A-4B56-AE50-6162FC49B073@cs.otago.ac.nz> <20170329132927.GA19866@erix.ericsson.se> <8BB800D5-4A36-4483-9675-18F976607FA0@cs.otago.ac.nz> Message-ID: <20170330064433.GA69507@erix.ericsson.se> On Thu, Mar 30, 2017 at 12:12:15PM +1300, Richard A. O'Keefe wrote: > I wrote that any support in Erlang for the 'abstract socket namespace' > of Linux would have to depend on the system *being* Linux. > > > On 30/03/2017, at 2:29 AM, Raimo Niskanen wrote: > > > > That is uncomfortable since the spirit of configure tests is to test for > > features and to not rely on a particular system name implying a feature. > > Where did I say anything about testing for a system name? > What I meant is simply that > *IF* Erlang ever supports this Linux feature (and I do not say that it > should; the last time I wanted to write system-specific code was the > last day I programmed a DEC-10) > it should take care to check for a leading NUL byte and give an > informative error on systems that DON'T support this "feature". > I have no idea what the best way to detect such support might be. > What I cared about is that Erlang shouldn't just go ahead and > accept leading NUL bytes on all systems because someone might be > expecting the Linux "feature", so an informative error message is > needed. Ah. Sorry, I misread you. I also have no idea how to detect if this "feature" is supported, so I thought you actually suggested to check if the system was "Linux" since you had no other suggestion... -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From russell@REDACTED Thu Mar 30 09:04:10 2017 From: russell@REDACTED (Russell Brown) Date: Thu, 30 Mar 2017 08:04:10 +0100 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record Message-ID: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> Hi, I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. The error is: riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} riak_object:object/0 The code in question is: %% @private generate an epoch actor, and update the vnode state. -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> {false, VId, State}; new_key_epoch(State) -> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), EpochId = key_epoch_actor(VId, Cntr), {true, EpochId, NewState}. What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: %% @private generate an epoch actor, and update the vnode state. -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> {VId, State}; new_key_epoch(State) -> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), EpochId = key_epoch_actor(VId, Cntr), {EpochId, NewState}. The record definition for #counter_state is: -record(counter_state, { use = true :: boolean(), cnt = 0 :: non_neg_integer(), lease = 0 :: non_neg_integer(), lease_size = 0 :: non_neg_integer(), leasing = false :: boolean() }). I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? Many thanks in advance if you can help Cheers Russell From alex0player@REDACTED Thu Mar 30 10:00:29 2017 From: alex0player@REDACTED (Alex S.) Date: Thu, 30 Mar 2017 11:00:29 +0300 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> Message-ID: <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> > #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()} Dialyzer somehow figured out that counter_state is never constructed with ?false? (and has no explicit contract). > 30 ????? 2017 ?., ? 10:04, Russell Brown ???????(?): > > Hi, > > I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. > > The error is: > > riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} > riak_object:object/0 > > > The code in question is: > > %% @private generate an epoch actor, and update the vnode state. > -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. > new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> > {false, VId, State}; > new_key_epoch(State) -> > NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), > EpochId = key_epoch_actor(VId, Cntr), > {true, EpochId, NewState}. > > > What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: > > %% @private generate an epoch actor, and update the vnode state. > -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. > new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> > {VId, State}; > new_key_epoch(State) -> > NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), > EpochId = key_epoch_actor(VId, Cntr), > {EpochId, NewState}. > > The record definition for #counter_state is: > > -record(counter_state, { > use = true :: boolean(), > cnt = 0 :: non_neg_integer(), > lease = 0 :: non_neg_integer(), > lease_size = 0 :: non_neg_integer(), > leasing = false :: boolean() > }). > > I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. > > If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? > > Many thanks in advance if you can help > > Cheers > > Russell > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From russell@REDACTED Thu Mar 30 10:06:37 2017 From: russell@REDACTED (Russell Brown) Date: Thu, 30 Mar 2017 09:06:37 +0100 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> Message-ID: <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> On 30 Mar 2017, at 09:00, Alex S. wrote: >> #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()} > > Dialyzer somehow figured out that counter_state is never constructed with ?false? (and has no explicit contract). Thanks for the reply, there is a code path where counter state can be updated to have `use=false`. I?m sorry but I don?t know what ?no explicit contract? means in this context, can you explain? Do you know how changing the return type tipped dialyzer off? >> 30 ????? 2017 ?., ? 10:04, Russell Brown ???????(?): >> >> Hi, >> >> I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. >> >> The error is: >> >> riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} >> riak_object:object/0 >> >> >> The code in question is: >> >> %% @private generate an epoch actor, and update the vnode state. >> -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. >> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >> {false, VId, State}; >> new_key_epoch(State) -> >> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >> EpochId = key_epoch_actor(VId, Cntr), >> {true, EpochId, NewState}. >> >> >> What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: >> >> %% @private generate an epoch actor, and update the vnode state. >> -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. >> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >> {VId, State}; >> new_key_epoch(State) -> >> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >> EpochId = key_epoch_actor(VId, Cntr), >> {EpochId, NewState}. >> >> The record definition for #counter_state is: >> >> -record(counter_state, { >> use = true :: boolean(), >> cnt = 0 :: non_neg_integer(), >> lease = 0 :: non_neg_integer(), >> lease_size = 0 :: non_neg_integer(), >> leasing = false :: boolean() >> }). >> >> I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. >> >> If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? >> >> Many thanks in advance if you can help >> >> Cheers >> >> Russell >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From alex0player@REDACTED Thu Mar 30 10:08:42 2017 From: alex0player@REDACTED (Alex S.) Date: Thu, 30 Mar 2017 11:08:42 +0300 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> Message-ID: <524EC174-AB4D-414C-B10F-4D3B4BC30D9E@gmail.com> > 30 ????? 2017 ?., ? 11:06, Russell Brown ???????(?): > > > On 30 Mar 2017, at 09:00, Alex S. wrote: > >>> #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()} >> >> Dialyzer somehow figured out that counter_state is never constructed with ?false? (and has no explicit contract). > > Thanks for the reply, there is a code path where counter state can be updated to have `use=false`. I?m sorry but I don?t know what ?no explicit contract? means in this context, can you explain? > > Do you know how changing the return type tipped dialyzer off? That means dialyzer figured out that this code path is never executed successfully, and contains a typing error. It is not a mistake to have a dead code, according to Dialyzer, just having all code dead in a function. Explicit contract means declaring types for counter_state, though maybe it is ignored for local records. Changing the return type might?ve broken the exact code path that can change ?use? to ?false?. > >>> 30 ????? 2017 ?., ? 10:04, Russell Brown ???????(?): >>> >>> Hi, >>> >>> I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. >>> >>> The error is: >>> >>> riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} >>> riak_object:object/0 >>> >>> >>> The code in question is: >>> >>> %% @private generate an epoch actor, and update the vnode state. >>> -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. >>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>> {false, VId, State}; >>> new_key_epoch(State) -> >>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>> EpochId = key_epoch_actor(VId, Cntr), >>> {true, EpochId, NewState}. >>> >>> >>> What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: >>> >>> %% @private generate an epoch actor, and update the vnode state. >>> -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. >>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>> {VId, State}; >>> new_key_epoch(State) -> >>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>> EpochId = key_epoch_actor(VId, Cntr), >>> {EpochId, NewState}. >>> >>> The record definition for #counter_state is: >>> >>> -record(counter_state, { >>> use = true :: boolean(), >>> cnt = 0 :: non_neg_integer(), >>> lease = 0 :: non_neg_integer(), >>> lease_size = 0 :: non_neg_integer(), >>> leasing = false :: boolean() >>> }). >>> >>> I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. >>> >>> If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? >>> >>> Many thanks in advance if you can help >>> >>> Cheers >>> >>> Russell >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > From russell@REDACTED Thu Mar 30 10:12:26 2017 From: russell@REDACTED (Russell Brown) Date: Thu, 30 Mar 2017 09:12:26 +0100 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <524EC174-AB4D-414C-B10F-4D3B4BC30D9E@gmail.com> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> <524EC174-AB4D-414C-B10F-4D3B4BC30D9E@gmail.com> Message-ID: <234FF6E2-E6A7-4665-B87A-FF6EF34BBCDD@wombat.me> On 30 Mar 2017, at 09:08, Alex S. wrote: >> >> 30 ????? 2017 ?., ? 11:06, Russell Brown ???????(?): >> >> >> On 30 Mar 2017, at 09:00, Alex S. wrote: >> >>>> #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()} >>> >>> Dialyzer somehow figured out that counter_state is never constructed with ?false? (and has no explicit contract). >> >> Thanks for the reply, there is a code path where counter state can be updated to have `use=false`. I?m sorry but I don?t know what ?no explicit contract? means in this context, can you explain? >> >> Do you know how changing the return type tipped dialyzer off? > > That means dialyzer figured out that this code path is never executed successfully, and contains a typing error. It is not a mistake to have a dead code, according to Dialyzer, just having all code dead in a function. > > Explicit contract means declaring types for counter_state, though maybe it is ignored for local records. > > Changing the return type might?ve broken the exact code path that can change ?use? to ?false?. I checked, it does not. This is very confusing. I can quite easily call the function with use=false, by changing a setting in riak?s advanced.config. Is the fact that this value comes from an app variable the problem? > >> >>>> 30 ????? 2017 ?., ? 10:04, Russell Brown ???????(?): >>>> >>>> Hi, >>>> >>>> I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. >>>> >>>> The error is: >>>> >>>> riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} >>>> riak_object:object/0 >>>> >>>> >>>> The code in question is: >>>> >>>> %% @private generate an epoch actor, and update the vnode state. >>>> -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. >>>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>>> {false, VId, State}; >>>> new_key_epoch(State) -> >>>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>>> EpochId = key_epoch_actor(VId, Cntr), >>>> {true, EpochId, NewState}. >>>> >>>> >>>> What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: >>>> >>>> %% @private generate an epoch actor, and update the vnode state. >>>> -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. >>>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>>> {VId, State}; >>>> new_key_epoch(State) -> >>>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>>> EpochId = key_epoch_actor(VId, Cntr), >>>> {EpochId, NewState}. >>>> >>>> The record definition for #counter_state is: >>>> >>>> -record(counter_state, { >>>> use = true :: boolean(), >>>> cnt = 0 :: non_neg_integer(), >>>> lease = 0 :: non_neg_integer(), >>>> lease_size = 0 :: non_neg_integer(), >>>> leasing = false :: boolean() >>>> }). >>>> >>>> I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. >>>> >>>> If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? >>>> >>>> Many thanks in advance if you can help >>>> >>>> Cheers >>>> >>>> Russell >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions From aronisstav@REDACTED Thu Mar 30 10:53:10 2017 From: aronisstav@REDACTED (Stavros Aronis) Date: Thu, 30 Mar 2017 10:53:10 +0200 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> Message-ID: Hi Russell, >From your original warning: On Thu, Mar 30, 2017 at 9:04 AM, Russell Brown wrote: > riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, > _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', > _, _, _, _}, _, _} tagged with a record name violates the declared type of > #state{idx::'undefined' | integer(),mod::atom() | > tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | > binary(),delete_mode::'immediate' | 'keep' | 'undefined' | > pos_integer(),bucket_buf_size::'undefined' | > pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' > | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff:: > boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() > | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() > | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | > #counter_state{use::'true',cnt::non_neg_integer(),lease:: > non_neg_integer(),lease_size::non_neg_integer(),leasing:: > boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | > tuple()} > riak_object:object/0 > my guess would be that in the definition of the #state record, the annotated #state.counter field's type is a #counter_state record whose #counter_state.use field annotated type is just the atom 'true'. Can you send some context from that part of the module? (e.g. definition of #state and #counter_state records) Best, Stavros -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell@REDACTED Thu Mar 30 11:32:56 2017 From: russell@REDACTED (Russell Brown) Date: Thu, 30 Mar 2017 10:32:56 +0100 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> Message-ID: <103CA58D-3162-4A5F-8F0A-4AA9EE5DC1A2@wombat.me> Hi Stavros, Thanks for the reply. On 30 Mar 2017, at 09:53, Stavros Aronis wrote: > Hi Russell, > > From your original warning: > > On Thu, Mar 30, 2017 at 9:04 AM, Russell Brown wrote: > riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} > riak_object:object/0 > > my guess would be that in the definition of the #state record, the annotated #state.counter field's type is a #counter_state record whose #counter_state.use field annotated type is just the atom 'true'. > > Can you send some context from that part of the module? (e.g. definition of #state and #counter_state records) The record definition for #counter_state is: -record(counter_state, { use = true :: boolean(), cnt = 0 :: non_neg_integer(), lease = 0 :: non_neg_integer(), lease_size = 0 :: non_neg_integer(), leasing = false :: boolean() }). The #state record: -record(state, {idx :: partition(), mod :: module(), async_put :: boolean(), modstate :: term(), mrjobs :: term(), vnodeid :: undefined | binary(), delete_mode :: keep | immediate | pos_integer(), bucket_buf_size :: pos_integer(), index_buf_size :: pos_integer(), key_buf_size :: pos_integer(), async_folding :: boolean(), in_handoff = false :: boolean(), handoff_target :: node(), handoffs_rejected = 0 :: integer(), forward :: node() | [{integer(), node()}], hashtrees :: pid(), upgrade_hashtree = false :: boolean(), md_cache :: ets:tab(), md_cache_size :: pos_integer(), counter :: #counter_state{}, status_mgr_pid :: pid(), %% a process that manages vnode status persistence update_hook = riak_kv_noop_update_hook :: update_hook() }). What I?m _most_ confused about is how the change of return type for that function surfaced this dialyzer warning. Thanks again for your time Russell > > Best, > > Stavros From chandrashekhar.mullaparthi@REDACTED Thu Mar 30 12:20:57 2017 From: chandrashekhar.mullaparthi@REDACTED (Chandrashekhar Mullaparthi) Date: Thu, 30 Mar 2017 11:20:57 +0100 Subject: [erlang-questions] ANN: OpenTracing Toolkit for ERlang Message-ID: <0D71EE6D-43C0-4FBF-AAE4-24D0E3DCDE3F@gmail.com> We are pleased to announce support for OpenTracing in Erlang and Elixir by the open source release of a library 'otter' (OpenTracing Toolkit for Erlang) under the Apache-2.0 license. This work has been in progress for the past few months and we are thrilled to announce this as part of the "CloudNativeCon + KubeCon Europe" event in Berlin today. OpenTracing is gaining a lot of traction as a distributed tracing framework in a cloud environment. This library will enable Erlang/Elixir based systems to play nicely within a heterogeneous computing environment which is typical of most cloud based deployments these days. The development of 'otter' was championed by Holger Winkelmann of Travelping. Travelping and bet365 kindly provided sponsorship for the initial development. Code is available at https://github.com/Bluehouse-Technology/otter OpenTracing : http://opentracing.io Travelping: http://www.travelping.com/ bet365: https://www.bet365.com/ Bluehouse Technology: https://www.bluehouse-technology.co.uk It is our intention to actively maintain this library and use it in our own products. Requests for features and pull requests are most welcome! Regards, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From ted.ml.erlang@REDACTED Thu Mar 30 13:41:05 2017 From: ted.ml.erlang@REDACTED (Ted Burghart) Date: Thu, 30 Mar 2017 07:41:05 -0400 Subject: [erlang-questions] Implementing enif_whereis Message-ID: Hello, I've implemented an enif_whereis function on a branch of master (OTP-20), and I have a couple of questions. This turned out to be pretty simple, and I'm wondering if something this small needs to go through the full EEP process or if it can be PR'd in, and does it need a Jira issue? The current function implementation (in erl_nif.c) is as follows: int enif_whereis(ErlNifEnv *env, ERL_NIF_TERM name, ERL_NIF_TERM *out) { if (is_atom(name)) { Process* c_p; Eterm res; execution_state(env, &c_p, NULL); res = erts_whereis_name_to_id(c_p, name); if (is_not_atom(res)) { *out = enif_make_copy(env, res); return 1; } } return 0; } This works fine for pids and ports, and one of my tests calls it from lots of parallel processes without any issues (both with and without lock checking enabled). It's got docs and tests, and I'm preparing the EEP Lite commit comment to be pushed shortly. So, anyone want to throw some darts at it? ? Ted From alex0player@REDACTED Thu Mar 30 16:26:43 2017 From: alex0player@REDACTED (Alex S.) Date: Thu, 30 Mar 2017 17:26:43 +0300 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <234FF6E2-E6A7-4665-B87A-FF6EF34BBCDD@wombat.me> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> <524EC174-AB4D-414C-B10F-4D3B4BC30D9E@gmail.com> <234FF6E2-E6A7-4665-B87A-FF6EF34BBCDD@wombat.me> Message-ID: <3394E307-4572-436E-BFE5-03562A5B7829@gmail.com> > 30 ????? 2017 ?., ? 11:12, Russell Brown ???????(?): > > > On 30 Mar 2017, at 09:08, Alex S. > wrote: > >>> >>> 30 ????? 2017 ?., ? 11:06, Russell Brown ???????(?): >>> >>> >>> On 30 Mar 2017, at 09:00, Alex S. wrote: >>> >>>>> #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()} >>>> >>>> Dialyzer somehow figured out that counter_state is never constructed with ?false? (and has no explicit contract). >>> >>> Thanks for the reply, there is a code path where counter state can be updated to have `use=false`. I?m sorry but I don?t know what ?no explicit contract? means in this context, can you explain? >>> >>> Do you know how changing the return type tipped dialyzer off? >> >> That means dialyzer figured out that this code path is never executed successfully, and contains a typing error. It is not a mistake to have a dead code, according to Dialyzer, just having all code dead in a function. >> >> Explicit contract means declaring types for counter_state, though maybe it is ignored for local records. >> >> Changing the return type might?ve broken the exact code path that can change ?use? to ?false?. > > I checked, it does not. This is very confusing. I can quite easily call the function with use=false, by changing a setting in riak?s advanced.config. Is the fact that this value comes from an app variable the problem? So, it executed correctly with use=false, without errors, and dialyzer still reports it? Then that is a bug. Dialyzer should never report bugs in working code. > >> >>> >>>>> 30 ????? 2017 ?., ? 10:04, Russell Brown ???????(?): >>>>> >>>>> Hi, >>>>> >>>>> I have a dialyzer error I can?t figure out, at all, it?s truly very weird, even for dialyzer. >>>>> >>>>> The error is: >>>>> >>>>> riak_kv_vnode.erl:2695: Matching of pattern State = {'state', _, _, _, _, _, VId, _, _, _, _, _, _, _, _, _, _, _, _, _, {'counter_state', 'false', _, _, _, _}, _, _} tagged with a record name violates the declared type of #state{idx::'undefined' | integer(),mod::atom() | tuple(),async_put::'false' | 'true' | 'undefined',vnodeid::'undefined' | binary(),delete_mode::'immediate' | 'keep' | 'undefined' | pos_integer(),bucket_buf_size::'undefined' | pos_integer(),index_buf_size::'undefined' | pos_integer(),key_buf_size::'undefined' | pos_integer(),async_folding::'false' | 'true' | 'undefined',in_handoff::boolean(),handoff_target::atom(),handoffs_rejected::integer(),forward::atom() | [{integer(),atom()}],hashtrees::'undefined' | pid(),upgrade_hashtree::boolean(),md_cache::atom() | tid(),md_cache_size::'undefined' | pos_integer(),counter::'undefined' | #counter_state{use::'true',cnt::non_neg_integer(),lease::non_neg_integer(),lease_size::non_neg_integer(),leasing::boolean()},status_mgr_pid::'undefined' | pid(),update_hook::atom() | tuple()} >>>>> riak_object:object/0 >>>>> >>>>> >>>>> The code in question is: >>>>> >>>>> %% @private generate an epoch actor, and update the vnode state. >>>>> -spec new_key_epoch(#state{}) -> {NewEpoch::boolean(), EpochActor :: binary(), #state{}}. >>>>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>>>> {false, VId, State}; >>>>> new_key_epoch(State) -> >>>>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>>>> EpochId = key_epoch_actor(VId, Cntr), >>>>> {true, EpochId, NewState}. >>>>> >>>>> >>>>> What is really freaky (to me) is that this code (the previous version) does not provoke a dialyzer warning: >>>>> >>>>> %% @private generate an epoch actor, and update the vnode state. >>>>> -spec new_key_epoch(#state{}) -> {EpochActor :: binary(), #state{}}. >>>>> new_key_epoch(State=#state{vnodeid=VId, counter=#counter_state{use=false}}) -> >>>>> {VId, State}; >>>>> new_key_epoch(State) -> >>>>> NewState=#state{counter=#counter_state{cnt=Cntr}, vnodeid=VId} = update_counter(State), >>>>> EpochId = key_epoch_actor(VId, Cntr), >>>>> {EpochId, NewState}. >>>>> >>>>> The record definition for #counter_state is: >>>>> >>>>> -record(counter_state, { >>>>> use = true :: boolean(), >>>>> cnt = 0 :: non_neg_integer(), >>>>> lease = 0 :: non_neg_integer(), >>>>> lease_size = 0 :: non_neg_integer(), >>>>> leasing = false :: boolean() >>>>> }). >>>>> >>>>> I have missed out the record definition for #state as it is huge, but crucially, unchanged between the two definitions of new_key_epoch/1 above. >>>>> >>>>> If the match in the function head, and the record, are exactly the same between the two versions of the code, why does the one that returns a three tuple provoke a warning? >>>>> >>>>> Many thanks in advance if you can help >>>>> >>>>> Cheers >>>>> >>>>> Russell >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Thu Mar 30 16:30:16 2017 From: duncan@REDACTED (duncan@REDACTED) Date: Thu, 30 Mar 2017 07:30:16 -0700 Subject: [erlang-questions] =?utf-8?q?Reliable_kernels=3F?= Message-ID: <20170330073016.7e43b23f706d1a78218bd3e1c66e57ee.7f81e18b4d.wbe@email23.godaddy.com> An HTML attachment was scrubbed... URL: From russell@REDACTED Thu Mar 30 16:32:22 2017 From: russell@REDACTED (Russell Brown) Date: Thu, 30 Mar 2017 15:32:22 +0100 Subject: [erlang-questions] Dialyzer warning on pattern match with sub-record In-Reply-To: <3394E307-4572-436E-BFE5-03562A5B7829@gmail.com> References: <144F5A48-7DC6-4E83-83DA-3E1856FC1D55@wombat.me> <4B28E02B-B49F-49D7-A3E7-45595FC9CD42@gmail.com> <6E6BD763-0D86-4156-ADC6-39DEB3DED862@wombat.me> <524EC174-AB4D-414C-B10F-4D3B4BC30D9E@gmail.com> <234FF6E2-E6A7-4665-B87A-FF6EF34BBCDD@wombat.me> <3394E307-4572-436E-BFE5-03562A5B7829@gmail.com> Message-ID: <3868A344-0E81-4727-91D4-B65CEE31EC1D@wombat.me> On 30 Mar 2017, at 15:26, Alex S. wrote: >> >> 30 ????? 2017 ?., ? 11:12, Russell Brown ???????(?): >> >> >> On 30 Mar 2017, at 09:08, Alex S. wrote: >> >>>> >> >> I checked, it does not. This is very confusing. I can quite easily call the function with use=false, by changing a setting in riak?s advanced.config. Is the fact that this value comes from an app variable the problem? > > So, it executed correctly with use=false, without errors, and dialyzer still reports it? Then that is a bug. Dialyzer should never report bugs in working code. Current thinking from the reviewer (Doug Rohrer, I know he?s on this list!) is that this head/match is now redundant, since the functions that call it check for use=true before calling new_key_epoch. So dialzyer is correct, the function only ever gets called with `true` now. Once again the age old refrain ?never argue with dialyzer?. I hope that someone learned something so this thread wasn?t a complete waste of time. Thanks, and thanks to all who replied. Cheers Russell > >> >>>>>> >>>>>> Russell >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions From ted.ml.erlang@REDACTED Thu Mar 30 17:01:41 2017 From: ted.ml.erlang@REDACTED (Ted Burghart) Date: Thu, 30 Mar 2017 11:01:41 -0400 Subject: [erlang-questions] Implementing enif_whereis In-Reply-To: References: Message-ID: <4369DB3F-5B30-494D-AB73-A26CC7B05473@tedb.net> My working code is at https://github.com/tburghart/otp/tree/trb/erts/enif_whereis I'm adding tests for behavior on dirty schedulers and background threads, which may result in checks in enif_whereis to bail out in contexts where it can't execute properly. Comments are welcome! > On 30-Mar 2017, at 7:41 AM, Ted Burghart wrote: > > Hello, > > I've implemented an enif_whereis function on a branch of master (OTP-20), and I have a couple of questions. > > This turned out to be pretty simple, and I'm wondering if something this small needs to go through the full EEP process or if it can be PR'd in, and does it need a Jira issue? > > The current function implementation (in erl_nif.c) is as follows: > > int enif_whereis(ErlNifEnv *env, ERL_NIF_TERM name, ERL_NIF_TERM *out) > { > if (is_atom(name)) { > Process* c_p; > Eterm res; > > execution_state(env, &c_p, NULL); > res = erts_whereis_name_to_id(c_p, name); > > if (is_not_atom(res)) { > *out = enif_make_copy(env, res); > return 1; > } > } > return 0; > } > > This works fine for pids and ports, and one of my tests calls it from lots of parallel processes without any issues (both with and without lock checking enabled). > > It's got docs and tests, and I'm preparing the EEP Lite commit comment to be pushed shortly. > > So, anyone want to throw some darts at it? > > ? Ted > -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Thu Mar 30 19:23:23 2017 From: duncan@REDACTED (duncan@REDACTED) Date: Thu, 30 Mar 2017 10:23:23 -0700 Subject: [erlang-questions] Using test data with ct Message-ID: <20170330102323.7e43b23f706d1a78218bd3e1c66e57ee.526e3e1827.wbe@email23.godaddy.com> An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Mar 30 19:34:24 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 30 Mar 2017 17:34:24 +0000 (UTC) Subject: [erlang-questions] getting args with erlang:get_stacktrace or similar? References: <50040593.8328512.1490895264527.ref@mail.yahoo.com> Message-ID: <50040593.8328512.1490895264527@mail.yahoo.com> When we call erlang:get_stacktrace we can get the function + line where we crashed. Is there a way to get the arguments the function we crashed in was called, or a way to get the current line we crashed on, what the arguments are currently? This would make narrowing down pesky crashes so much easier. From dmkolesnikov@REDACTED Thu Mar 30 20:20:04 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 30 Mar 2017 21:20:04 +0300 Subject: [erlang-questions] Using test data with ct In-Reply-To: <20170330102323.7e43b23f706d1a78218bd3e1c66e57ee.526e3e1827.wbe@email23.godaddy.com> References: <20170330102323.7e43b23f706d1a78218bd3e1c66e57ee.526e3e1827.wbe@email23.godaddy.com> Message-ID: <137DDB6D-B6E1-4578-AD86-1E173B4E72D8@gmail.com> Hello, CT has built support for it. Please check this tutorial http://learnyousomeerlang.com/common-test-for-uncommon-tests In the nutshell, you create a folder and store your file there mytest_SUTE_data and you can access this files from SUITE: Scenario = filename:join([?config(data_dir, Config), "my.json"]), Best Regards, Dmitry > On Mar 30, 2017, at 8:23 PM, wrote: > > I'm building a rebar3 application and using ct to test it. I have a bunch of test files containing data (sample json) to use as part of testing. When it was just a few, I converted them to erlang terms, and used -define in .hrl files in the include directory. But that doesn't scale well nor make for easy configuration management. I'd prefer to store each json (for a particular test) in a file since those files already exist created by others for validation. Then the test could read the file, and use jsx to convert text to json for comparison within the test. > > Where can I stick a bunch of files and then access them as part of test? I understand there is a data_dir. Is that the proper way to do it? Where in the tree do I put a file so it ends up in data_dir? > > Duncan Sparrell > sFractal Consulting LLC > iPhone, iTypo, iApologize > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dch@REDACTED Thu Mar 30 23:50:12 2017 From: dch@REDACTED (Dave Cottlehuber) Date: Thu, 30 Mar 2017 23:50:12 +0200 Subject: [erlang-questions] REPLACE_OS_VARS in erlexec? Message-ID: <1490910612.532813.929186248.628E3415@webmail.messagingengine.com> On Wed, 29 Mar 2017, at 21:55, Stefan Hellkvist wrote: > Hello Erlangers, > > For different reasons (for instance to be able to run distributed Erlang > inside Kubernetes for instance, like here: > https://substance.brpx.com/clustering-elixir-nodes-on-kubernetes-e85d0c26b0cf), > there is a need to be able to override or replace the settings in vm.args > (or the args_file if you use the name of the argument to erlexec) with > environment variables. > > For instance like so (example taken from the above article): > > ----begin vm.args---- > ## Name of the node > -name ${MY_POD_NAMESPACE}@${MY_POD_IP} > > ## Cookie for distributed erlang > -setcookie ${ERLANG_COOKIE} > ... > ----end vm.args----- > > > The need for this functionality seems to be big enough for there to now > be > support for it in the release builders (relx for instance offers a way to > turn on vm.args environment variable replacement with the > RELX_REPLACE_OS_VARS environment variable when running the start script). > > > My question though is if this functionality (to set variables such as > -name > or -setcookie in args_file through environment variables) has ever been > considered for erlexec itself? > > For various reasons (mostly to reduce dependencies to other programs such > as various shell utilities) I would like to start my erlang nodes without > the generated release script by calling erlexec directly, but I realize I > would, by doing so, miss out on the power to set variables in arg-file > with > environment variables (like in relx above). > > I guess it would not be extremely difficult to modify erlexec for this > purpose (erlexec already has functions for reading environment variables > so > it would be a small change I guess) but I'm not sure if there would be > enough interest in this feature to accept a pull-request and I would not > like to roll my own fork of OTP only for this rather limited purpose. So, > does anyone know if this this feature has ever been discussed or > considered > for inclusion in OTP? > > /Stefan Sounds very useful. I have at least a couple of times removed all the release script cruft and written a simple erlexec wrapper, for debugging purposes. I assume using erlexec directly is not supported as I don't find it in the usual place like http://erlang.org/doc/man/erl.html My main gripe with relx and descendants is that simply replacing variables and creating a new vm.args file is a horrible hack. It would be nice to have that support built into the emulator directly, as you propose. BTW my latest version of this (with Elixir goodness) is here: - https://gist.github.com/dch/aee49d986e6acf7621c0b48366dab727 and you can pass arbitrary parameters to erlexec, such as -env KEY VALUE, -setcookie ... and -[s]name .... (in fact any other parameter that erl accepts) directly, but I suppose you already know this. A+ Dave From christopher.meiklejohn@REDACTED Fri Mar 31 10:51:09 2017 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Fri, 31 Mar 2017 10:51:09 +0200 Subject: [erlang-questions] Google Summer of Code for Lasp Message-ID: There's only a few days left to apply, but we're looking for more students to come be part of the Google Summer of Code project for Lasp. In terms of proposals, I'm happy to help any eager students who are interested in submitting something write a proposal. For ideas, we've got a lot of things that are interesting happening: * I'd like to see someone bring SSL/TLS support to our partisan library -- partisan is a highly scalable work-in-progress replacement for distributed Erlang for edge and mobile computing. * We've got a newish work-in-progress report out on our Loquat system -- recently presented at Erlang Factory 2016 in SF -- and we've love to have students help us build out a first functioning prototype of the system and evaluate it. * Documentation: help us write documentation about how to use Lasp from Erlang and Elixir. * Finally, we'd love to see some students work on building real applications in Lasp: we've got several ideas for applications, but haven't had time to implement them yet. (Ideally, we'd love to see these example applications in Elixir and Erlang!) Last year was super fun: not only did we have great success with two student projects, one of them resulted in a draft paper and presentation at the AGERE! workshop located at ACM's SPLASH conference. [Working on open source, on a research project, where your results will possibly contribute to a conference paper and you as co-author, not bad!] Please reach out, and I'm looking forward to your submissions! Thanks, Christopher Meiklejohn