From ok@REDACTED Tue Mar 1 00:33:55 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 1 Mar 2016 12:33:55 +1300 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: My mail system is sufficiently messed up that I can't attribute this. Sorry. > I have been led to understand by people far more familiar with Erlang than > I am that in order to use certain behaviours from OTP one has to write > boilerplate code for several callbacks whether one wants to modify the > behavior from defaults or not. This is seriously weird. The whole point of behaviours is to *not* write stuff. If you have defaults that you find yourself using more than once, you could do things like sticking them in -include files, or even write your own module to wrap the behaviour. Or you could use the e2 behaviours instead of the OTP ones. Or you could copy the behaviour module you want and change its interface, then use the copy. You could even discuss the interface in this mailing list and propose changes. It would be nice to have more details, like >which< behaviours and >what< defaults (or rather, such the point at issue is that there _aren't_ defaults, what the defaults should be). From siraaj@REDACTED Tue Mar 1 03:43:30 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 29 Feb 2016 21:43:30 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> Message-ID: <56D501D2.3070805@khandkar.net> On 2/29/16 1:39 PM, Jesper Louis Andersen wrote: > On Mon, Feb 29, 2016 at 4:28 PM, Siraaj Khandkar > wrote: > >> In other words, I'm saying it is very possible to have this cake (of >> public structure) and eat it too (in private, without anyone accessing the >> rapidly-changing, secret cream) :) > > > You are so close to the idea of view types, and Torben hinted me I should > say something about them :) The basic idea is indeed very similar, but, of course, far less ambitious :) My main point is that the idea of an API compatibility is a social contract to, once published, not remove parts of a signature of a component. The following two components make equivalent contracts with their users: -module(a). -export_type([foo/0]). -export([new/0, ..., get_a/1, get_b/1]). -record(foo, {a :: a(), b :: b(), c :: super_secret()}). -opaque foo() :: #foo{}. -spec new() -> foo(). -spec get_a(foo()) -> a(). -spec get_b(foo()) -> b(). -module(b). -export_type([foo/0, bar/0]). -export([new/0, ..., view/1]). -record(foo, {a :: a(), b :: b(), c :: super_secret()}). -record(bar, {a :: a(), b :: b()}). -opaque foo() :: #foo{}. -type bar() :: #bar{}. -spec new() -> foo(). -spec view(foo()) -> bar(). Here, for simplicity, bar is just a subset of foo, but, in practice, it's fine for them to have nothing in common, of course (so one can do more complicated things, if desired). Which one to choose is pretty much art - all depends on author's other constraints/plans. > > In other words: exporting records across application boundaries tend to > lock them down, so be pretty sure they are set in stone. Precisely. Just like any API should be, within bounds of it's promise to the users (such as semver). From siraaj@REDACTED Tue Mar 1 04:15:26 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 29 Feb 2016 22:15:26 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: <56d4a3dc.45ce190a.394c6.609a@mx.google.com> References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> <56d4a3dc.45ce190a.394c6.609a@mx.google.com> Message-ID: <56D5094E.4070607@khandkar.net> On 2/29/16 3:02 PM, Torben Hoffmann wrote: > > > Jesper Louis Andersen writes: > >> [ text/plain ] >> On Mon, Feb 29, 2016 at 4:28 PM, Siraaj Khandkar >> wrote: >> >>> In other words, I'm saying it is very possible to have this cake (of >>> public structure) and eat it too (in private, without anyone accessing the >>> rapidly-changing, secret cream) :) >> >> >> You are so close to the idea of view types, and Torben hinted me I should >> say something about them :) >> >> One advantage of a function is that it doesn't have to give you the field >> outright, but can provide ways to dynamically cast that structure into >> another structure you export. Two really good examples is that you can >> return some kind of tuple depending on the contents of the data and thus >> you can build a case match on the data. This ad-hoc >> for-the-purpose-construction of a datatype can often yield far more precise >> code since it splits analysis of what is *in* the datastructure from >> computation which says what to *do* about that data. Say we have a #uri{} >> record. The path component has many possible representations: a binary(), a >> list of binaries, an iterator, and so on. Different view-functions would >> give you different ways to handle that binary. >> >> Another nice thing is that functions can give you are >> zipper/derivative/delimiting-continuation over the data in the structure. >> The path-component can then be unfolded one path-entry at a time: >> >> pth(X) -> >> case http_uri:unfold_path(X) of >> none -> ...; >> {some, C, X2} -> >> ... C ... pth(X2) >> end. >> >> or you can imagine a gb_trees/gb_sets like iterator over the data structure. >> >> A plain map() cannot give any of these ideas, and its transparency also >> tightly couples your code to the map() structure. So one has to carefully >> weigh what kind of interface you want in the long run. I much prefer views >> of the data structures if possible since it allows for more freedom in the >> long run, but it requires you to be able to figure out what kind of >> functions you would need a priori on the data. On the other hand, it >> provides for much better hiding of what is going on inside the module, >> which allows you more flexibility regarding backwards compatibility. >> >> In other words: exporting records across application boundaries tend to >> lock them down, so be pretty sure they are set in stone. Also think hard if >> you want every user to implement the analysis code. The view is much like >> in a database: do you export the raw records, or do you expose a >> transactional API through stored procedures which tell you what you can do >> with the data? >> >> > Why attempt to write a half-baked answer when you can ask Jesper to > explain it well?!?! [1] > > Look at Cowboy and Webmachine for examples of how to provide functions > to work with a request. > > Both have a very rich API for extracting relevant things out of the > request record. > No need to start poking into that record on your own. > I think you missed the point, see my reply to Jesper :) That said, since you bring-up Cowboy, it is a good example of what I mean by other constraints and plans - an author might want to fill-in some of the fields lazily (such body in cowboy_req), in which case per-field accessor functions are their only good choice, unless, of course, they want to consider compound subsets (of, say, only the eagerly calculated parts, but I digress - it isn't the important point)... :) Anyhow, without a specific application and vision for it's future, which API design direction to take is not something you or I can finalize here - it depends on a bunch of things. The important point is that, by using an abstract type, you're not at all limiting yourself to exposing per-field accessor functions - you can do other things (if you wish), as long as you make a clear distinction between internals and API. From davidnwelton@REDACTED Tue Mar 1 06:27:06 2016 From: davidnwelton@REDACTED (David Welton) Date: Mon, 29 Feb 2016 21:27:06 -0800 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: If you were going to do "web stuff", besides something really straightforward where you just use Cowboy directly, what else would you consider besides Phoenix? It comes with a lot of nice things out of the box. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From pavananms@REDACTED Tue Mar 1 08:57:13 2016 From: pavananms@REDACTED (Pavanan M S) Date: Tue, 1 Mar 2016 13:27:13 +0530 Subject: [erlang-questions] Rest endpoint behaving strangely Message-ID: Hi all, I created a few rest end points for an app and it works. Now I am starting to write the test cases. But there is a strange issue in my code. When I connect the rest client with url " http://localhost:8080/[resource_name] " it works perfectly. But if I use " http://localhost:8080/[resource_name]*/ *" , (A slash ( '/' ) in the end) it gives a 404. Now that I am trying to write test cases using common test, I just tried to make requests using ibrowse first. And there I found something strange again. When I used ibrowse to issue a post request, it returned 302, with the location of the resource. In the browser I am getting 200, I guess because of get. Now, since get is working, I guess the resource is being created, Then what might be the reason that I am not getting 201 ? Also, Is this related to the above problem ? Please help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Tue Mar 1 10:44:26 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Tue, 1 Mar 2016 10:44:26 +0100 Subject: [erlang-questions] [ANN] Announcing Erlang on Slack, the team communication tool Message-ID: Hello all We have recently launched a channel for Erlang on Slack. It is a team communication tool where you can chat with other Erlangers. You can chat in channels, send one-to-one messages, paste code snippets, share files and search for messages. We also integrated with the JIRA issue tracker bugs.erlang.org. Visit: https://erlanger.slack.com/ Click here to request an invitation: https://erlang-slack.herokuapp.com/ [image: Inline image 1] [image: Inline image 2] A warm welcome and hope to see you all on Slack! Best regards *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-03-01 at 10.12.43.png Type: image/png Size: 57637 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-03-01 at 10.17.31.png Type: image/png Size: 47315 bytes Desc: not available URL: From essen@REDACTED Tue Mar 1 11:16:46 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 1 Mar 2016 11:16:46 +0100 Subject: [erlang-questions] Rest endpoint behaving strangely In-Reply-To: References: Message-ID: <56D56C0E.8090703@ninenines.eu> I think you didn't say which server you are using. Not Cowboy considering the 404 you get. On 03/01/2016 08:57 AM, Pavanan M S wrote: > Hi all, > I created a few rest end points for an app and it works. Now I > am starting to write the test cases. But there is a strange issue in my > code. > > When I connect the rest client with url " > http://localhost:8080/[resource_name] " it works perfectly. > > But if I use " http://localhost:8080/[resource_name]*/ *" , (A > slash ( '/' ) in the end) it gives a 404. > > Now that I am trying to write test cases using common test, I > just tried to make requests using ibrowse first. And there I found > something strange again. > > When I used ibrowse to issue a post request, it returned 302, > with the location of the resource. In the browser I am getting 200, I > guess because of get. > > Now, since get is working, I guess the resource is being > created, Then what might be the reason that I am not getting 201 ? Also, > Is this related to the above problem ? > Please help. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From mremond@REDACTED Tue Mar 1 12:07:36 2016 From: mremond@REDACTED (=?UTF-8?B?TWlja2HDq2wgUsOpbW9uZCAu?=) Date: Tue, 01 Mar 2016 11:07:36 +0000 Subject: [erlang-questions] BEAM Community and Google Summer of Code Message-ID: Hello Erlanger, We have great news ! The BEAM Community has been accepted as a Google Summer of Code project. https://summerofcode.withgoogle.com/organizations/5748418722922496/ For those not familiar with the program, it allows students to get funded to get the opportunity to work on Open Source Source Software. This is a great opportunity to get funded to work on a cool project in Erlang, or Elixir programming languages. As a student, you should really consider participating in that program. Details are on the BEAM Community Github Page: http://beamcommunity.github.io Take the time to read through the idea, join the BEAM community mailing list (https://groups.google.com/forum/#!forum/beam-community). Your proposals are welcome ! Cheers, -- Micka?l R?mond -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavananms@REDACTED Tue Mar 1 12:38:36 2016 From: pavananms@REDACTED (Pavanan M S) Date: Tue, 1 Mar 2016 17:08:36 +0530 Subject: [erlang-questions] Rest endpoint behaving strangely In-Reply-To: <56D56C0E.8090703@ninenines.eu> References: <56D56C0E.8090703@ninenines.eu> Message-ID: Sorry the response code is *303 not 302* On 1 March 2016 at 15:46, Lo?c Hoguin wrote: > I think you didn't say which server you are using. Not Cowboy considering > the 404 you get. > > On 03/01/2016 08:57 AM, Pavanan M S wrote: > >> Hi all, >> I created a few rest end points for an app and it works. Now I >> am starting to write the test cases. But there is a strange issue in my >> code. >> >> When I connect the rest client with url " >> http://localhost:8080/[resource_name] " it works perfectly. >> >> But if I use " http://localhost:8080/[resource_name]*/ *" , (A >> slash ( '/' ) in the end) it gives a 404. >> >> Now that I am trying to write test cases using common test, I >> just tried to make requests using ibrowse first. And there I found >> something strange again. >> >> When I used ibrowse to issue a post request, it returned 302, >> with the location of the resource. In the browser I am getting 200, I >> guess because of get. >> >> Now, since get is working, I guess the resource is being >> created, Then what might be the reason that I am not getting 201 ? Also, >> Is this related to the above problem ? >> Please help. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Mar 1 12:52:23 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 1 Mar 2016 12:52:23 +0100 Subject: [erlang-questions] Rest endpoint behaving strangely In-Reply-To: References: <56D56C0E.8090703@ninenines.eu> Message-ID: <56D58277.2040503@ninenines.eu> See http://ninenines.eu/docs/en/cowboy/1.0/guide/rest_flowcharts/#put,_post_and_patch_methods for details as to why you get a status code instead of another. I'm not entirely sure the diagram is clear enough though. What status code is returned depends on what you return from the callback, what method it is, whether there's a body in the response and whether the resource existed previously. The diagram is at least missing the case where the resource didn't exist and the method is PUT, where you get a 201. On 03/01/2016 12:38 PM, Pavanan M S wrote: > Sorry the response code is *303 not 302* > > On 1 March 2016 at 15:46, Lo?c Hoguin > wrote: > > I think you didn't say which server you are using. Not Cowboy > considering the 404 you get. > > On 03/01/2016 08:57 AM, Pavanan M S wrote: > > Hi all, > I created a few rest end points for an app and it > works. Now I > am starting to write the test cases. But there is a strange > issue in my > code. > > When I connect the rest client with url " > http://localhost:8080/[resource_name] > " it works perfectly. > > But if I use " http://localhost:8080/[resource_name]*/ > *" , (A > slash ( '/' ) in the end) it gives a 404. > > Now that I am trying to write test cases using common > test, I > just tried to make requests using ibrowse first. And there I found > something strange again. > > When I used ibrowse to issue a post request, it > returned 302, > with the location of the resource. In the browser I am getting > 200, I > guess because of get. > > Now, since get is working, I guess the resource is being > created, Then what might be the reason that I am not getting 201 > ? Also, > Is this related to the above problem ? > Please help. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From siraaj@REDACTED Tue Mar 1 15:30:14 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 1 Mar 2016 09:30:14 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D18874.1080002@ninenines.eu> References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> Message-ID: <56D5A776.7090609@khandkar.net> Brilliant analysis, Lo?c! Is this the Scott Adams post you're referring to? http://blog.dilbert.com/post/136950092871/why-would-a-man-vote-for-hillary-clinton Which is also quite brilliant in its observations - thanks for the pointer! On 2/27/16 6:28 AM, Lo?c Hoguin wrote: > On 02/26/2016 09:21 PM, Jos? Valim wrote: >> But I may not be representative. Last time I counted I've >> used around 40 languages in anger over the years, yet >> I find Ruby bewildering. >> >> >> I am not sure Ruby is relevant here. Elixir is not Ruby (and it could >> never be in the Erlang VM). Elixir also isn't about Ruby syntax (the >> same way Erlang isn't about Prolog syntax)[4]. > > Rationally, Elixir is not Ruby, and Erlang isn't Prolog. Irrationally, > it is. Elixir has the same look and feel as Ruby, and Erlang has the > same look and feel as Prolog. > > When Ruby developers look at Elixir they feel right at home. If you call > yourself a Ruby developer, then you identify with certain values from > Ruby, many of which can be found in Elixir. It's familiar. Again, we are > on the irrational level here. > > Same goes for Erlang and Prolog. In fact a few days ago a few long-time > Prolog developers pointed out the exact same thing when they were > talking about Erlang. There is this familiarity that smoothes them in, > even though the languages are fundamentally different. > > The thing is, if you have to convince large groups of people, you need > to appeal to their irrational mind. As Scott Adams brilliantly pointed > out, identity beats analogy beats reason. If you want to convince people > to come to Elixir, you need to appeal to their identity, which is why > targeting Ruby on Rails developers is your best bet. If you don't then > you're just wasting valuable time and resources. > > I've pointed out a few years ago that Elixir was for Ruby developers. I > didn't know why at the time. If you look at the most recent survey > (http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), you > can see that Ruby developers dominate. Other languages are little more > than a statistical anomaly. Clearly you bring in a lot more Ruby > developers than any other combined, and the reason for that is identity. > > Stop fighting it. Use it to bring more people in. > From essen@REDACTED Tue Mar 1 15:57:33 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 1 Mar 2016 15:57:33 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D5A776.7090609@khandkar.net> References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> <56D5A776.7090609@khandkar.net> Message-ID: <56D5ADDD.8070001@ninenines.eu> One of them. He's been writing a lot about this, especially in the past 8-9 months. He's predicted nearly all the US Republican primaries so far and has been writing extensively about it, explaining everything in details. Very instructing. Highlights a lot about human behavior. Anyway this is off-topic so feel free to email me off-list. On 03/01/2016 03:30 PM, Siraaj Khandkar wrote: > Brilliant analysis, Lo?c! > > Is this the Scott Adams post you're referring to? > > http://blog.dilbert.com/post/136950092871/why-would-a-man-vote-for-hillary-clinton > > > Which is also quite brilliant in its observations - thanks for the pointer! > > > On 2/27/16 6:28 AM, Lo?c Hoguin wrote: >> On 02/26/2016 09:21 PM, Jos? Valim wrote: >>> But I may not be representative. Last time I counted I've >>> used around 40 languages in anger over the years, yet >>> I find Ruby bewildering. >>> >>> >>> I am not sure Ruby is relevant here. Elixir is not Ruby (and it could >>> never be in the Erlang VM). Elixir also isn't about Ruby syntax (the >>> same way Erlang isn't about Prolog syntax)[4]. >> >> Rationally, Elixir is not Ruby, and Erlang isn't Prolog. Irrationally, >> it is. Elixir has the same look and feel as Ruby, and Erlang has the >> same look and feel as Prolog. >> >> When Ruby developers look at Elixir they feel right at home. If you call >> yourself a Ruby developer, then you identify with certain values from >> Ruby, many of which can be found in Elixir. It's familiar. Again, we are >> on the irrational level here. >> >> Same goes for Erlang and Prolog. In fact a few days ago a few long-time >> Prolog developers pointed out the exact same thing when they were >> talking about Erlang. There is this familiarity that smoothes them in, >> even though the languages are fundamentally different. >> >> The thing is, if you have to convince large groups of people, you need >> to appeal to their irrational mind. As Scott Adams brilliantly pointed >> out, identity beats analogy beats reason. If you want to convince people >> to come to Elixir, you need to appeal to their identity, which is why >> targeting Ruby on Rails developers is your best bet. If you don't then >> you're just wasting valuable time and resources. >> >> I've pointed out a few years ago that Elixir was for Ruby developers. I >> didn't know why at the time. If you look at the most recent survey >> (http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), you >> can see that Ruby developers dominate. Other languages are little more >> than a statistical anomaly. Clearly you bring in a lot more Ruby >> developers than any other combined, and the reason for that is identity. >> >> Stop fighting it. Use it to bring more people in. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From t@REDACTED Tue Mar 1 16:42:20 2016 From: t@REDACTED (Tristan Sloughter) Date: Tue, 01 Mar 2016 09:42:20 -0600 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D5ADDD.8070001@ninenines.eu> References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> <56D5A776.7090609@khandkar.net> <56D5ADDD.8070001@ninenines.eu> Message-ID: <1456846940.1402310.536301490.6DC0C393@webmail.messagingengine.com> Yes, please get this off list, this is more disgusting than I ever thought Scott Adams (who I thought was disgusting already) would be. -- Tristan Sloughter t@REDACTED On Tue, Mar 1, 2016, at 08:57 AM, Lo?c Hoguin wrote: > One of them. He's been writing a lot about this, especially in the past > 8-9 months. He's predicted nearly all the US Republican primaries so far > and has been writing extensively about it, explaining everything in > details. Very instructing. Highlights a lot about human behavior. > > Anyway this is off-topic so feel free to email me off-list. > > On 03/01/2016 03:30 PM, Siraaj Khandkar wrote: > > Brilliant analysis, Lo?c! > > > > Is this the Scott Adams post you're referring to? > > > > http://blog.dilbert.com/post/136950092871/why-would-a-man-vote-for-hillary-clinton > > > > > > Which is also quite brilliant in its observations - thanks for the pointer! > > > > > > On 2/27/16 6:28 AM, Lo?c Hoguin wrote: > >> On 02/26/2016 09:21 PM, Jos? Valim wrote: > >>> But I may not be representative. Last time I counted I've > >>> used around 40 languages in anger over the years, yet > >>> I find Ruby bewildering. > >>> > >>> > >>> I am not sure Ruby is relevant here. Elixir is not Ruby (and it could > >>> never be in the Erlang VM). Elixir also isn't about Ruby syntax (the > >>> same way Erlang isn't about Prolog syntax)[4]. > >> > >> Rationally, Elixir is not Ruby, and Erlang isn't Prolog. Irrationally, > >> it is. Elixir has the same look and feel as Ruby, and Erlang has the > >> same look and feel as Prolog. > >> > >> When Ruby developers look at Elixir they feel right at home. If you call > >> yourself a Ruby developer, then you identify with certain values from > >> Ruby, many of which can be found in Elixir. It's familiar. Again, we are > >> on the irrational level here. > >> > >> Same goes for Erlang and Prolog. In fact a few days ago a few long-time > >> Prolog developers pointed out the exact same thing when they were > >> talking about Erlang. There is this familiarity that smoothes them in, > >> even though the languages are fundamentally different. > >> > >> The thing is, if you have to convince large groups of people, you need > >> to appeal to their irrational mind. As Scott Adams brilliantly pointed > >> out, identity beats analogy beats reason. If you want to convince people > >> to come to Elixir, you need to appeal to their identity, which is why > >> targeting Ruby on Rails developers is your best bet. If you don't then > >> you're just wasting valuable time and resources. > >> > >> I've pointed out a few years ago that Elixir was for Ruby developers. I > >> didn't know why at the time. If you look at the most recent survey > >> (http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), you > >> can see that Ruby developers dominate. Other languages are little more > >> than a statistical anomaly. Clearly you bring in a lot more Ruby > >> developers than any other combined, and the reason for that is identity. > >> > >> Stop fighting it. Use it to bring more people in. > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bshroyer@REDACTED Tue Mar 1 16:55:52 2016 From: bshroyer@REDACTED (Brandon Shroyer) Date: Tue, 1 Mar 2016 10:55:52 -0500 Subject: [erlang-questions] Erlang scheduler bindings on Windows Message-ID: Hello all, I am trying to set scheduler bindings by adding the '+stbt ts' flag to a Windows .bat file. When I try to run the application, though, I get the following error: setting scheduler bind type ' ts' failed: invalid type I'm not sure why the space is being counted as part of the argument here; does anyone have any suggestions on how to prevent that? Thanks, Brandon Shroyer -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Tue Mar 1 18:46:23 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Wed, 2 Mar 2016 01:46:23 +0800 Subject: [erlang-questions] Mnesia table replication scheme Message-ID: Hi, I'm storing user accounts and organizations (a tree of users, i.e. departments, sub-departments, etc) in mnesia tables. Modification to the data is not very frequent, i.e. not as frequent as a message queue. I can't afford losing the data. Assume I have 9 erlang nodes (Linux VMs in a clound), and I haven't decided how many of them should be used for handling client connections and how many should be used for data storage/backup. I know that the more nodes a table is replicated on, the slower writing to it will be; I also know that ram copies are faster than disk copies; I am seeking to find a balance between performance and data safety. How should the tables be replicated? Should I use ram copies or disk copies or a hybrid scheme? Can the data be in ram copies on some nodes and backed up in disk only copies on some other nodes? If this is possible, will it provide performance as ram copies while still have data safety like disk copies? What is the best practice here? Thanks Khitai From dangud@REDACTED Tue Mar 1 21:10:40 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 01 Mar 2016 20:10:40 +0000 Subject: [erlang-questions] Mnesia table replication scheme In-Reply-To: References: Message-ID: You can mix as you want, if ram space is not a problem I would use disc_copies on a two nodes. tis 1 mar 2016 18:46 Khitai Pang skrev: > Hi, > > I'm storing user accounts and organizations (a tree of users, i.e. > departments, sub-departments, etc) in mnesia tables. Modification to > the data is not very frequent, i.e. not as frequent as a message queue. > I can't afford losing the data. Assume I have 9 erlang nodes (Linux VMs > in a clound), and I haven't decided how many of them should be used for > handling client connections and how many should be used for data > storage/backup. > > I know that the more nodes a table is replicated on, the slower writing > to it will be; I also know that ram copies are faster than disk copies; > I am seeking to find a balance between performance and data safety. How > should the tables be replicated? Should I use ram copies or disk copies > or a hybrid scheme? Can the data be in ram copies on some nodes and > backed up in disk only copies on some other nodes? If this is possible, > will it provide performance as ram copies while still have data safety > like disk copies? > > What is the best practice here? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowdhury918@REDACTED Tue Mar 1 22:02:58 2016 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Tue, 1 Mar 2016 16:02:58 -0500 Subject: [erlang-questions] DTLSex Message-ID: Hi, I need to use DTLS in Erlang. Does anyone have used dtlsex( https://github.com/RoadRunnr/dtlsex/tree/master/src)? Is it properly tested? Is it usable/reliable? I need to use it for server side. Any one has any experience? Any input will be highly appreciated. Looking forward to it. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.carrone@REDACTED Wed Mar 2 00:53:27 2016 From: federico.carrone@REDACTED (Federico Carrone) Date: Tue, 01 Mar 2016 23:53:27 +0000 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test Message-ID: Hi, I would like to update the HTML/CSS/Javascript of the Erlang documents and also the html generated by CT. I am not a frontend developer or designer, but I have fought a few good fights on the frontend frontline (I have already created http://spawnedshelter.com/ for Erlang for example) For example Elixir docs are really pretty and easy documents to nativage: http://elixir-lang.org/docs/stable/elixir/Kernel.SpecialForms.html practicaltypography python's example are great too: http://practicaltypography.com/python-docs-after/index.html I am sure this is not a priority for most developers. However I do think this type of things are important for some newcomers. I would like to check if a pull request doing this type of changes would be accepted if I met all the required requirements. I have no idea who I have to talk to. Any help or recommendation would be appreciated. Thanks in advance, Federico. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Wed Mar 2 04:15:04 2016 From: sdl.web@REDACTED (Leo Liu) Date: Wed, 02 Mar 2016 11:15:04 +0800 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test References: Message-ID: On 2016-03-01 23:53 +0000, Federico Carrone wrote: > I am sure this is not a priority for most developers. However I do > think this type of things are important for some newcomers. I think it is very important to the Erlang community. The bar has been raised so high that there is no excuse not to make the documentation pleasant to read. Hope there is a way to facilitate your contribution. Cheers, Leo From zxq9@REDACTED Wed Mar 2 08:37:14 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 02 Mar 2016 16:37:14 +0900 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: References: Message-ID: <1867637.adRS8g8MDV@burrito> On Wednesday 02 March 2016 11:15:04 Leo Liu wrote: > On 2016-03-01 23:53 +0000, Federico Carrone wrote: > > I am sure this is not a priority for most developers. However I do > > think this type of things are important for some newcomers. > > I think it is very important to the Erlang community. The bar has been > raised so high that there is no excuse not to make the documentation > pleasant to read. Hope there is a way to facilitate your contribution. I've wished I had time to do the same. If you actually *do* have time to work on this it would be great (and if you need a hand, ASK -- I've mentioned it more than once, just never got around to doing anything). You may be able to create a pull request directly against the OTP project on github. Maybe not. Anyway, good luck! -Craig From jean.parpaillon@REDACTED Wed Mar 2 13:05:10 2016 From: jean.parpaillon@REDACTED (Jean Parpaillon) Date: Wed, 02 Mar 2016 13:05:10 +0100 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: <1867637.adRS8g8MDV@burrito> References: <1867637.adRS8g8MDV@burrito> Message-ID: <1456920310.17621.32.camel@free.fr> mkdocs is used for generating documentation suitable for, for instance, readthedocs.org platform. I've had started a mkdocs doclet module some time ago but it is stalled now. At that time, I was not really familiar with xmerl, and this is one of the reasons I've not continued. Anybody interested in going on ? https://github.com/erocci/edoc_mkdocs Cheers, Jean Le mercredi 02 mars 2016 ? 16:37 +0900, zxq9 a ?crit?: > On Wednesday 02 March 2016 11:15:04 Leo Liu wrote: > > On 2016-03-01 23:53 +0000, Federico Carrone wrote: > > > I am sure this is not a priority for most developers. However I > > > do > > > think this type of things are important for some newcomers. > > > > I think it is very important to the Erlang community. The bar has > > been > > raised so high that there is no excuse not to make the > > documentation > > pleasant to read. Hope there is a way to facilitate your > > contribution. > > I've wished I had time to do the same. If you actually *do* have time > to > work on this it would be great (and if you need a hand, ASK -- I've > mentioned it more than once, just never got around to doing > anything). > > You may be able to create a pull request directly against the OTP > project on github. Maybe not. > > Anyway, good luck! > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jean Parpaillon -- Director @ OW2 Consortium OCCIware Strategic Orientation Committee Chairman Research Engineer @ Inria -- Phone: +33 6 30 10 92 86 im: jean.parpaillon@REDACTED skype: jean.parpaillon linkedin: http://www.linkedin.com/in/jeanparpaillon/en From ali.sabil@REDACTED Wed Mar 2 13:38:52 2016 From: ali.sabil@REDACTED (Ali Sabil) Date: Wed, 02 Mar 2016 12:38:52 +0000 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: <1456920310.17621.32.camel@free.fr> References: <1867637.adRS8g8MDV@burrito> <1456920310.17621.32.camel@free.fr> Message-ID: Thank you for bringing this up. An improvement to the presentation of the Erlang documentation is very much needed. I have started looking at this myself sometime ago, here is what I gathered: ? The OTP documentation is using erl_docgen which is mostly XSLT based ? Edoc is not that much used in OTP, the applications that do rely on it, use a custom layout (otpsgml_layout) to generate output that can be consumed by erl_docgen. ? Experimented with adding a modern HTML5 output to erl_docgen here: https://github.com/asabil/otp/tree/feature/html5-docs but this proved itself challenging. ? Started working on a modern doclet for edoc, based on the ex_doc layout: https://bitbucket.org/asabil/edoc-modern Hope this helps. Cheers, Ali On Wed, Mar 2, 2016 at 1:05 PM Jean Parpaillon wrote: > mkdocs is used for generating documentation suitable for, for instance, > readthedocs.org platform. > > I've had started a mkdocs doclet module some time ago but it is stalled > now. At that time, I was not really familiar with xmerl, and this is > one of the reasons I've not continued. > > Anybody interested in going on ? > https://github.com/erocci/edoc_mkdocs > > > Cheers, > Jean > > > Le mercredi 02 mars 2016 ? 16:37 +0900, zxq9 a ?crit : > > On Wednesday 02 March 2016 11:15:04 Leo Liu wrote: > > > On 2016-03-01 23:53 +0000, Federico Carrone wrote: > > > > I am sure this is not a priority for most developers. However I > > > > do > > > > think this type of things are important for some newcomers. > > > > > > I think it is very important to the Erlang community. The bar has > > > been > > > raised so high that there is no excuse not to make the > > > documentation > > > pleasant to read. Hope there is a way to facilitate your > > > contribution. > > > > I've wished I had time to do the same. If you actually *do* have time > > to > > work on this it would be great (and if you need a hand, ASK -- I've > > mentioned it more than once, just never got around to doing > > anything). > > > > You may be able to create a pull request directly against the OTP > > project on github. Maybe not. > > > > Anyway, good luck! > > > > -Craig > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -- > Jean Parpaillon > -- > Director @ OW2 Consortium > OCCIware Strategic Orientation Committee Chairman > Research Engineer @ Inria > -- > Phone: +33 6 30 10 92 86 > im: jean.parpaillon@REDACTED > skype: jean.parpaillon > linkedin: http://www.linkedin.com/in/jeanparpaillon/en > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Mar 2 13:58:31 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 02 Mar 2016 21:58:31 +0900 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: References: <1456920310.17621.32.camel@free.fr> Message-ID: <291389291.rCvldqcINk@burrito> On Wednesday 02 March 2016 12:38:52 Ali Sabil wrote: > Thank you for bringing this up. An improvement to the presentation of the > Erlang documentation is very much needed. > > I have started looking at this myself sometime ago, here is what I gathered: > ? The OTP documentation is using erl_docgen which is mostly XSLT based > ? Edoc is not that much used in OTP, the applications that do rely on > it, use a custom layout (otpsgml_layout) to generate output that can be > consumed by erl_docgen. > ? Experimented with adding a modern HTML5 output to erl_docgen here: > https://github.com/asabil/otp/tree/feature/html5-docs but this proved > itself challenging. > ? Started working on a modern doclet for edoc, based on the ex_doc > layout: https://bitbucket.org/asabil/edoc-modern Hi Ali, I ran into the exact same thing and concluded it would be easier to treat rendered docs as their own data source rather than try to decobble all the mix-and-match that exists within the OTP sources themselves (though the *right* approach would be to actually make the docs across OTP uniform -- I felt this would be futile to even attempt alone and unsupported, who has that much (un)paid time on their hands?). Tricky, to say the least. -Craig From jean.parpaillon@REDACTED Wed Mar 2 14:07:28 2016 From: jean.parpaillon@REDACTED (Jean Parpaillon) Date: Wed, 02 Mar 2016 14:07:28 +0100 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: <291389291.rCvldqcINk@burrito> References: <1456920310.17621.32.camel@free.fr> <291389291.rCvldqcINk@burrito> Message-ID: <1456924048.14749.5.camel@free.fr> Hi, Le mercredi 02 mars 2016 ? 21:58 +0900, zxq9 a ?crit?: > On Wednesday 02 March 2016 12:38:52 Ali Sabil wrote: > > Thank you for bringing this up. An improvement to the presentation > > of the > > Erlang documentation is very much needed. > > > > I have started looking at this myself sometime ago, here is what I > > gathered: > > ????? The OTP documentation is using erl_docgen which is mostly > > XSLT based > > ????? Edoc is not that much used in OTP, the applications that do > > rely on > > it, use a custom layout (otpsgml_layout) to generate output that > > can be > > consumed by erl_docgen. > > ????? Experimented with adding a modern HTML5 output to erl_docgen > > here: > > https://github.com/asabil/otp/tree/feature/html5-docs but this prov > > ed > > itself challenging. > > ????? Started working on a modern doclet for edoc, based on the > > ex_doc > > layout: https://bitbucket.org/asabil/edoc-modern > > Hi Ali, > > I ran into the exact same thing and concluded it would be easier to > treat > rendered docs as their own data source rather than try to decobble > all the > mix-and-match that exists within the OTP sources themselves (though > the > *right* approach would be to actually make the docs across OTP > uniform -- > I felt this would be futile to even attempt alone and unsupported, > who has > that much (un)paid time on their hands?). > > Tricky, to say the least. > > -Craig The approach I've had with mkdocs is to translate into more-or-less standard documentation intermediate format, like mkdocs or *cough* docbook ;) This would have the advantage of benefiting from existing ecosystem and let uniformity come later (we could later develop a specific mkdocs theme, for instance). My 2c, > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jean Parpaillon -- Director @ OW2 Consortium OCCIware Strategic Orientation Committee Chairman Research Engineer @ Inria -- Phone: +33 6 30 10 92 86 im: jean.parpaillon@REDACTED skype: jean.parpaillon linkedin: http://www.linkedin.com/in/jeanparpaillon/en From jesper.louis.andersen@REDACTED Wed Mar 2 15:47:41 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 2 Mar 2016 15:47:41 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 Message-ID: Hi Erlangers, I'd really like to add two functions to the lists module from Haskell: intersperse(List, Seperator) produces a list where each element is separated by separator, i.e. X = [1,2,3] [1, x, 2, x, 3] = lists:intersperse(X, x), and it's cousin, intercalate(ListOfLists, Separator) is append(intersperse(ListOfLists, Seperator)), i.e, Y = ["a", "b", "c"] "a, b, c" = lists:intercalate(Y, ", "), The implementations are straightforward and easy to write tests for, even property based tests if needed. The rationale for this proposal is that I find myself implementing this function again and again in every project I write, and it is highly generic. It belongs in a typical list module. OCaml libraries add it. Haskell's Data.List has it. I believe Erlang, being a practical language, should have it as well. Thoughts? -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Wed Mar 2 15:54:11 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 2 Mar 2016 06:54:11 -0800 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <7938FDD2-5930-45D9-BF5D-62AB8B3BAE59@gmail.com> I find myself reimplementing the first one a lot, for use with iolists. The second one is string:join/2, right? > On 02 Mar 2016, at 06:47, Jesper Louis Andersen wrote: > > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is separated by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is append(intersperse(ListOfLists, Seperator)), i.e, > > Y = ["a", "b", "c"] > "a, b, c" = lists:intercalate(Y, ", "), > > The implementations are straightforward and easy to write tests for, even property based tests if needed. > > The rationale for this proposal is that I find myself implementing this function again and again in every project I write, and it is highly generic. It belongs in a typical list module. OCaml libraries add it. Haskell's Data.List has it. I believe Erlang, being a practical language, should have it as well. > > Thoughts? > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Wed Mar 2 16:05:23 2016 From: bob@REDACTED (Robert Wilkinson) Date: Wed, 2 Mar 2016 16:05:23 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <20160302150523.GA14264@fourtheye.org> On Wed, Mar 02, 2016 at 03:47:41PM +0100, Jesper Louis Andersen wrote: > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is > separated by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > Thoughts? > > -- > J. Hi Jesper Please spell Separator consistently and correctly :-) Bob -- Kiss me, Kate, we will be married o' Sunday. -- William Shakespeare, "The Taming of the Shrew" From erlangworkshop@REDACTED Wed Mar 2 15:12:53 2016 From: erlangworkshop@REDACTED (Erlang Workshop) Date: Wed, 2 Mar 2016 15:12:53 +0100 Subject: [erlang-questions] First Call For Papers -- Erlang Workshop 2016 Message-ID: <56D6F4E5.40405@gmail.com> Apologies for any duplicates you may receive. CALL FOR PAPERS =============== Fifteenth ACM SIGPLAN Erlang Workshop ----------------------------------------------------------- Nara, Japan, September 23, 2016 Satellite event of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016) September 18-24, 2016 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing interesting contributions either in theoretical work or real world applications. Submission related to Erlang, Elixir, Akka, CloudHaskell, Occam, and functional programming are welcome and encouraged. Topics of interest include (but are not limited to): - virtual machine extensions and compilation techniques - implementations and interfaces of Erlang in/with other languages - new tools (profilers, tracers, debuggers, testing frameworks etc.) - language extensions - formal semantics, correctness and verification - testing Erlang programs - program analysis and transformation - Erlang-like languages and technologies - functional languages and multi-processing - concurrency in functional languages - functional languages and distributed computing - parallel programming - pattern based programming - Erlang in education The maximum length for technical papers is restricted to 12 pages. 2. Experience reports describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the experience report is restricted to 2 pages. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Melinda T?th, E?tv?s Lor?nd University, Hungary Scott Lystig Fritchie, Basho Japan KK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Jamie Allen, Typesafe Laura M. Castro, University of A Coru?a, Spain Natalia Chechina, University of Glasgow Viktoria F?rd?s, Erlang Solutions Yosuke Hara, Rakuten, Inc. ? Sandhya Narayan, Infoblox Kenji Rikitake, KRPEO Bruce Tate, iCanMakeItBetter Simon Thompson, University of Kent, UK Important Dates ----------------------- Submissions due: Friday, 3 June, 2016 Author notification: Friday, 8 July, 2016 Final copy due: Sunday, 31 July, 2016 Workshop date: September 23, 2016 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2016" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2016 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2016 web site at: http://conf.researchr.org/home/icfp-2016 Related Links -------------------- CFP: http://conf.researchr.org/track/icfp-2016/erlang-2016-papers ICFP 2016 web site: http://conf.researchr.org/home/icfp-2016 Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2016 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Atendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/Anti-harassment -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Mar 2 16:21:58 2016 From: g@REDACTED (Garrett Smith) Date: Wed, 02 Mar 2016 15:21:58 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <7938FDD2-5930-45D9-BF5D-62AB8B3BAE59@gmail.com> References: <7938FDD2-5930-45D9-BF5D-62AB8B3BAE59@gmail.com> Message-ID: I recently implemented the second as "join" - ala string, but for arbitrary separators and list elements. I think the name "intersperse" is descriptive enough. "intercalculate" no so much. I'd go with "join". In the interest of consistency with other lists functions, I'd put the list argument last in each case. On Wed, Mar 2, 2016 at 8:54 AM Pierre Fenoll wrote: > I find myself reimplementing the first one a lot, for use with iolists. > The second one is string:join/2, right? > > > > On 02 Mar 2016, at 06:47, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > > Hi Erlangers, > > > > I'd really like to add two functions to the lists module from Haskell: > > > > intersperse(List, Seperator) produces a list where each element is > separated by separator, i.e. > > > > X = [1,2,3] > > [1, x, 2, x, 3] = lists:intersperse(X, x), > > > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > > > Y = ["a", "b", "c"] > > "a, b, c" = lists:intercalate(Y, ", "), > > > > The implementations are straightforward and easy to write tests for, > even property based tests if needed. > > > > The rationale for this proposal is that I find myself implementing this > function again and again in every project I write, and it is highly > generic. It belongs in a typical list module. OCaml libraries add it. > Haskell's Data.List has it. I believe Erlang, being a practical language, > should have it as well. > > > > Thoughts? > > > > -- > > J. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Wed Mar 2 18:02:49 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 2 Mar 2016 12:02:49 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <56D71CB9.2010209@khandkar.net> On 3/2/16 9:47 AM, Jesper Louis Andersen wrote: > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is > separated by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > Y = ["a", "b", "c"] > "a, b, c" = lists:intercalate(Y, ", "), > > The implementations are straightforward and easy to write tests for, even > property based tests if needed. > > The rationale for this proposal is that I find myself implementing this > function again and again in every project I write, and it is highly > generic. It belongs in a typical list module. OCaml libraries add it. > Haskell's Data.List has it. I believe Erlang, being a practical language, > should have it as well. > > Thoughts? +1 Though I prefer the name "interleave" to "intersperse", since its meaning is more-precise and closer to the intended behavior here. From dmkolesnikov@REDACTED Wed Mar 2 18:30:02 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 2 Mar 2016 19:30:02 +0200 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56D71CB9.2010209@khandkar.net> References: <56D71CB9.2010209@khandkar.net> Message-ID: <81CF3CC5-CAF6-4DC3-8AD5-F4590896AEF3@gmail.com> Hello, ?interleave? and ?join? are friendly for non-native English programmers. Otherwise +1 - Dmitry > On Mar 2, 2016, at 7:02 PM, Siraaj Khandkar wrote: > > On 3/2/16 9:47 AM, Jesper Louis Andersen wrote: >> Hi Erlangers, >> >> I'd really like to add two functions to the lists module from Haskell: >> >> intersperse(List, Seperator) produces a list where each element is >> separated by separator, i.e. >> >> X = [1,2,3] >> [1, x, 2, x, 3] = lists:intersperse(X, x), >> >> and it's cousin, intercalate(ListOfLists, Separator) is >> append(intersperse(ListOfLists, Seperator)), i.e, >> >> Y = ["a", "b", "c"] >> "a, b, c" = lists:intercalate(Y, ", "), >> >> The implementations are straightforward and easy to write tests for, even >> property based tests if needed. >> >> The rationale for this proposal is that I find myself implementing this >> function again and again in every project I write, and it is highly >> generic. It belongs in a typical list module. OCaml libraries add it. >> Haskell's Data.List has it. I believe Erlang, being a practical language, >> should have it as well. >> >> Thoughts? > > +1 > > Though I prefer the name "interleave" to "intersperse", since its meaning is more-precise and closer to the intended behavior here. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Mar 2 19:28:04 2016 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 2 Mar 2016 19:28:04 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen wrote: > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is separated > by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > Y = ["a", "b", "c"] > "a, b, c" = lists:intercalate(Y, ", "), > > The implementations are straightforward and easy to write tests for, even > property based tests if needed. > > The rationale for this proposal is that I find myself implementing this > function again and again in every project I write, and it is highly generic. > It belongs in a typical list module. OCaml libraries add it. Haskell's > Data.List has it. I believe Erlang, being a practical language, should have > it as well. > This is a splendid function - I have one myself called interleave, the only problem is that if somebody uses this and puts their code in a library on github and somebody fetches this code and runs it with an *old* version of erlang then they will get a error and probably blame your code. This is why I have a single library called elib2_misc.erl where I put *all* the stuff that should be somewhere else - then I ship this with my code. The whole story of dependency analysis seems to be be in a very sad state. I use the 'keep your fingers crossed and hope it will work' method. Years ago I wanted to add a -needs(Vsn) annotation to module For example: -needs(erl18). Means would mean you need version 18 of erlang. That way if you added a needs annotation to the code that used the updated lists then an earlier version of Erlang could give a meaningfull diagnostic and not just crash when the encountering a function 'from the future'. Actually I had another problem today - a program I wrote in 2003 did not work with the latest and greatest Erlang - not because the language had changed but because the library functions had been renamed and moved around. I guess a lint program would be useful here. It should be fairly doable to download all old erlangs and simple make lists of all the function names in all modules and do a bit of consistency checking. Cheers /Joe > Thoughts? > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gumm@REDACTED Wed Mar 2 21:01:22 2016 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 2 Mar 2016 14:01:22 -0600 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: I'd like to go ahead and throw in a big fat "same". I've also reimplemented the wheel in Nitrogen with a `join` function that interleaves a list of things with some other things. So cast my vote for a lists:join. `join` I would argue is much more accessible, as 1) it's quite common in other languages, 2) stays remains consistent with string:join and filename:join, and 3) brevity is awesome. -Jesse On Wed, Mar 2, 2016 at 12:28 PM, Joe Armstrong wrote: > On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen > wrote: >> Hi Erlangers, >> >> I'd really like to add two functions to the lists module from Haskell: >> >> intersperse(List, Seperator) produces a list where each element is separated >> by separator, i.e. >> >> X = [1,2,3] >> [1, x, 2, x, 3] = lists:intersperse(X, x), >> >> and it's cousin, intercalate(ListOfLists, Separator) is >> append(intersperse(ListOfLists, Seperator)), i.e, >> >> Y = ["a", "b", "c"] >> "a, b, c" = lists:intercalate(Y, ", "), >> >> The implementations are straightforward and easy to write tests for, even >> property based tests if needed. >> >> The rationale for this proposal is that I find myself implementing this >> function again and again in every project I write, and it is highly generic. >> It belongs in a typical list module. OCaml libraries add it. Haskell's >> Data.List has it. I believe Erlang, being a practical language, should have >> it as well. >> > > This is a splendid function - I have one myself called interleave, the only > problem is that if somebody uses this and puts their code in a library on github > and somebody fetches this code and runs it with an *old* version of erlang > then they will get a error and probably blame your code. > > This is why I have a single library called elib2_misc.erl where I put *all* the > stuff that should be somewhere else - then I ship this with my code. > > The whole story of dependency analysis seems to be be in a very sad state. > > I use the 'keep your fingers crossed and hope it will work' method. > > Years ago I wanted to add a -needs(Vsn) annotation to module > > For example: > > -needs(erl18). > > Means would mean you need version 18 of erlang. > > That way if you added a needs annotation to the code that used the updated lists > then an earlier version of Erlang could give a meaningfull diagnostic > and not just > crash when the encountering a function 'from the future'. > > Actually I had another problem today - a program I wrote in 2003 did not work > with the latest and greatest Erlang - not because the language had changed > but because the library functions had been renamed and moved around. > > I guess a lint program would be useful here. It should be fairly > doable to download > all old erlangs and simple make lists of all the function names in all modules > and do a bit of consistency checking. > > Cheers > > /Joe > > > > >> Thoughts? >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From vladdu55@REDACTED Wed Mar 2 21:05:12 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 2 Mar 2016 21:05:12 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: On Wed, Mar 2, 2016 at 7:28 PM, Joe Armstrong wrote: > Years ago I wanted to add a -needs(Vsn) annotation to module > > For example: > > -needs(erl18). > > Means would mean you need version 18 of erlang. > > That way if you added a needs annotation to the code that used the updated > lists > then an earlier version of Erlang could give a meaningfull diagnostic > and not just > crash when the encountering a function 'from the future'. > > Actually I had another problem today - a program I wrote in 2003 did not > work > with the latest and greatest Erlang - not because the language had changed > but because the library functions had been renamed and moved around. > > I guess a lint program would be useful here. It should be fairly > doable to download > all old erlangs and simple make lists of all the function names in all > modules > and do a bit of consistency checking. > One such tool is geas https://github.com/crownedgrouse/geas. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Mar 2 21:41:49 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 02 Mar 2016 21:41:49 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: Message-ID: <4882a98b-53d1-4de0-89cd-33d91f08e6e2@email.android.com> An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Mar 2 23:01:42 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 11:01:42 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <940CE93F-68F6-46DC-9634-1DBD57994D5D@pixie.co.za> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> <3c3cc4e9432dd3c8541953b60c86f026.squirrel@chasm.otago.ac.nz> <940CE93F-68F6-46DC-9634-1DBD57994D5D@pixie.co.za> Message-ID: On 27/02/16 3:53 am, Valentin Micic wrote: Far from being an oxymoron, ensuring atomicity is surprisingly tricky. > > And when you say COMPLETELY INACCESSIBLE FROM THE OUTSIDE, I assume > this would exclude read access as well. Yes. > If the whole point of registering processes is to be able to provide > for a static way of sending a message to a process, how would that > work under these circumstances? I'm not sure what you mean when you refer to 'a static way'. The existing registry is *dynamic*, and so are pidnames. There are presently many modules, and books and training materials recommend creating more, where a module creates a process which *that* module needs to communicate with but which *no* other module should be able to contact directly. That's the problem -pidnames solve. > > Are you saying that one can have more than one -pidname declaration > per module? Yes. > > Don't you think this would mean that you would have to know in advance > (that is, compile-time) about all the names the process powered by > this module would ever have? Yes. (Except that you could support an unbounded number of names by creating a private registry *process* and binding that to a pidname.) > If so, I am quite surprised that you do not see a problem with that. I don't see a problem with that because there is an abundance of modules where there *is* a design-time-known set of processes to refer to. As far as I can tell, this is in fact the *usual* case. The -pidname proposal is NOT a replacement for the existing registry, but a much safer alternative for the many modules that need *private* names. If that's not your use-case, just keep on using the existing registry. > > Also, seeing that module name and named pidname variable are not the > same, what would happen if two different modules uses the same > name for a pidname? Nothing, The whole *point* of -pidnames is to be *private* to their containing module. > Of course, you may solve this problem by indicating that these are > COMPLETELY INACCESSIBLE FROM THE OUTSIDE, but again, assuming that the > reason for registration is to be able to have a "static" way to send a > message to a process, what would be the point of having them in the > first place if they are COMPLETELY INACCESSIBLE FROM THE OUTSIDE. Because you don't WANT them to be accessible from the outside. Because in fact you are terrified that somebody might access them from the outside, causing horrible breakage. A very common pattern is that a module contains two kinds of code: - code to be executed by a process - code for the protocols used to communicate with that process In order to find the process, the protocol code needs to find it. Currently, people use the registry, but that makes the module vulnerable to * accidental or malicious deletion of the registry entry * accidental or malicious replacement of the registry entry * accidental or malicious sending of messages to the process without the mediation of the protocol code. In order to prevent that, you need something like a registry entry that is private to the module. "The point of having them in the first place" is *precisely* to be "completely inaccessible from the outside". There is no other reason for having them. > > Given the Erlang standard syntax, it stands to reason that > pidname:set( , self() ) invokes a function set, > that belongs to another module (pidname), and, as such, goes against > your earlier stipulation that -pidname declarations are COMPLETELY > INACCESSIBLE FROM THE OUTSIDE. False. I used function call syntax (a) as a quick expedient in exposition and (b) because there isn't really anything else to use. The intention is that they would really be special syntax that just happens to look like a function call. > > See a contradiction that I attempted to point out above... I could find no contradiction. > When I say instance, well, I actually mean a process, as this is what > we register today (and we do that so we can use a symbolic reference > instead of pid() in order to send messages to it). > In ideal situation, you would have 1:1 correspondence between a module > and a process. I don't see why. I can't think of any concurrent programming language I've ever used that has had such a correspondence, or any text on concurrent programming that has recommended such a correspondence. I certainly have modules that exist to let a caller create concurrent data structures, as many as they want, and I don't understand why that would be disparaged. > However, this is rarely the case; e.g. what we register as a > gen_server is not the gen_server module, but the actual behavior that > runs on top of gen_server. So, which -pidname variable should we use? Where did you get the idea that -pidnames are meant to do *everything*? > And let me just reiterate a point that I've made commenting on your > Second point above -- are we to provide gen_server with as many names > as we have behaviors? I repeat, where did you get the idea that -pidnames are meant to do *everything*? Pidnames are for use when you need *private* names for a *known* set of processes. If that's not your use-case, use something else. > The point is, you cannot possibly predict all deployments any given > module may have. Of course. And I don't *care*. All that matters is whether the designer of a module knows about a set of processes s/he wants to *hide*. (Another version of this may have appeared. ThunderBird seemed to have completely lost it.) From eric.pailleau@REDACTED Wed Mar 2 23:12:19 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 02 Mar 2016 23:12:19 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: Message-ID: <322b91f8-bafb-480b-a0c3-fc85574f3560@email.android.com> An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Mar 2 23:17:41 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 02 Mar 2016 23:17:41 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <322b91f8-bafb-480b-a0c3-fc85574f3560@email.android.com> Message-ID: An HTML attachment was scrubbed... URL: From valentin@REDACTED Wed Mar 2 23:36:46 2016 From: valentin@REDACTED (Valentin Micic) Date: Thu, 3 Mar 2016 00:36:46 +0200 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: References: Message-ID: <564F73D0-9E8B-4BB3-B6CA-63057BE60C58@pixie.co.za> On 03 Mar 2016, at 12:17 AM, ?ric Pailleau wrote: > And for local pidname, a simple put() can store the spawned Pid in parent process dictionary and retrieve with get(). > > Le 2 mars 2016 11:12 PM, ?ric Pailleau a ?crit : > Hi, > Could not this be solved by spawn_priv/x functions ? I mean functions that do same than spawn/x but spawned process accept only messages from parent process ? > No need to change things in process registry then. Good luck supporting/debugging the code that may register itself, but able to receive messages only from the process that spawned it. Did April 1st arrived early this year? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Mar 2 23:55:37 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 02 Mar 2016 23:55:37 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <564F73D0-9E8B-4BB3-B6CA-63057BE60C58@pixie.co.za> Message-ID: <30e856f5-1807-4fee-b7c1-3900b0f989b3@email.android.com> An HTML attachment was scrubbed... URL: From valentin@REDACTED Thu Mar 3 00:57:05 2016 From: valentin@REDACTED (Valentin Micic) Date: Thu, 3 Mar 2016 01:57:05 +0200 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <30e856f5-1807-4fee-b7c1-3900b0f989b3@email.android.com> References: <30e856f5-1807-4fee-b7c1-3900b0f989b3@email.android.com> Message-ID: <691A810E-182E-4D2E-99DA-D576EC7B694A@pixie.co.za> Non sequitur! On 03 Mar 2016, at 12:55 AM, ?ric Pailleau wrote: > I suppose that process_flag(sensitive, true) was also invented a 1st April too, then. > > Le 2 mars 2016 11:36 PM, Valentin Micic a ?crit : > On 03 Mar 2016, at 12:17 AM, ?ric Pailleau wrote: > > And for local pidname, a simple put() can store the spawned Pid in parent process dictionary and retrieve with get(). > > Le 2 mars 2016 11:12 PM, ?ric Pailleau a ?crit : > Hi, > Could not this be solved by spawn_priv/x functions ? I mean functions that do same than spawn/x but spawned process accept only messages from parent process ? > No need to change things in process registry then. > Good luck supporting/debugging the code that may register itself, but able to receive messages only from the process that spawned it. > Did April 1st arrived early this year? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Mar 3 01:22:38 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 03 Mar 2016 01:22:38 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <691A810E-182E-4D2E-99DA-D576EC7B694A@pixie.co.za> Message-ID: An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 3 01:32:13 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 13:32:13 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: On 3/03/16 3:47 am, Jesper Louis Andersen wrote: > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) Why have you swapped the arguments? The Haskellfunction Data.List.intersperse :: a -> [a] -> [a] puts the separator *first*. > produces a list where each element is separated by separator, You can't separate one thing. The elements *are* separated by the separator. > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), Once again, why have you switched the arguments? Data.List.intercalate :: [a] -> [[a]] -> [a] puts the separator *first*. I've never really been happy with the name "intercalate". To the extent that it's acceptableto use "intercalate" for anything other than adding chunks to the calendar in order to fix itup, it means what the intersperse function does. However, the name _is_ established inHaskell, and there's no point in gratuitous difference. Which is why it is important to get the argument order right. intersperse and intercalate are very nearly the same function. intersperse sep xs = interwhatsit (:) [] (sep:) xs intercalate sep xs = interwhatsit (++) [] (sep++) xs interwhatsit :: (a -> b -> b) -> b -> (b -> b) -> [a] -> b -- Combine End Separate Items interwhatsit _ e _ [] = e interwhatsit c e s (x:xs) = loop xs x where loop [] x = c x e loop (y:ys) x = c x $ s $ loop ys y Do we want to have the special cases without the general case? (There has to be a better name than interwhatsit. sepfoldr?) > > The rationale for this proposal is that I find myself implementing > this function again and again in every project I write Why would you do that rather than putting it in a utilities module and just reusing that module? > , and it is highly generic. That sounds as though you have some other use in mind for these functions than pasting strings together. I'd love to know what. I've been using Haskell since before it had intersperse and intercalate, and can't remember ever using them. From ok@REDACTED Thu Mar 3 01:41:08 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 13:41:08 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56D71CB9.2010209@khandkar.net> References: <56D71CB9.2010209@khandkar.net> Message-ID: <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> On 3/03/16 6:02 am, Siraaj Khandkar wrote: > Though I prefer the name "interleave" to "intersperse", since its > meaning is more-precise and closer to the intended behavior here. No, interleaving requires *two* sequences. This is interleave: interleave([], []) -> []; interleave([X|Xs], [Y|Ys]) -> [X,Y|interleave(Xs,Ys)]. interleave, v, OED, sense 3: "Computing and Telecommunications. To interfile (two or more digitized signals or sequences of information) by alternating between them; to alternate (one such signal or sequence) with others. Also, to divide (memory, etc.) between a number of different tasks by allocating successive segments to each in turn." (I must admit I hadn't encountered the word 'interfile' before.) From ok@REDACTED Thu Mar 3 01:50:42 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 13:50:42 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <322b91f8-bafb-480b-a0c3-fc85574f3560@email.android.com> References: <322b91f8-bafb-480b-a0c3-fc85574f3560@email.android.com> Message-ID: <6f9a235c-7d42-6e2f-a394-b165f9784dec@cs.otago.ac.nz> On 3/03/16 11:12 am, ?ric Pailleau wrote: > > Hi, > Could not this be solved by spawn_priv/x functions ? I mean functions > that do same than spawn/x but spawned process accept only messages > from parent process ? > No need to change things in process registry then. > If we're still talking about pidnames, then the answer is NO. (That's not to say that some such thing could not be useful, only that it would not solve the same problem.) Let's suppose we have process P calls M:foobar() which uses a local pidname to communicate with *module*-private process R. process Q calls M:zarkov() which uses the same local pidname to communicate with the same process R. This is GOOD. We're happy with it. We are delighted that code inside M but executed by P sends a message to R and that code inside M but executed by a different process Q also sends a message to R. Not a problem! In both cases, the message was sent by code inside M, which knows R's protocol. If it's a matter of a process remembering the pid of a child process, you'd never use the process registry for that in the first place. You'd use the process dictionary of the parent process. We don't even need a new kind of spawn: without using subversive debugging machinery, the only processes that can send you messages are yourself and your parent, until one of you passes the pid on to some other process. Note that a process that only accepts messages from its parent, while it *could* be useful, would not be able to create its own child and receive replies from that. From ok@REDACTED Thu Mar 3 01:52:09 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 13:52:09 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: References: Message-ID: <54ab62c8-b9c9-740e-866b-1f7996dc34d5@cs.otago.ac.nz> On 3/03/16 11:17 am, ?ric Pailleau wrote: > > And for local pidname, a simple put() can store the spawned Pid in > parent process dictionary and retrieve with get(). > which would be amazingly useless for the problem -pidnames were devised to solve, which is not a PROCESS knowing a particular pid, but a MODULE. Modules don't *have* a process dictionary. From siraaj@REDACTED Thu Mar 3 02:23:55 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 2 Mar 2016 20:23:55 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> Message-ID: <56D7922B.1090102@khandkar.net> On 3/2/16 7:41 PM, Richard A. O'Keefe wrote: > > > On 3/03/16 6:02 am, Siraaj Khandkar wrote: >> Though I prefer the name "interleave" to "intersperse", since its >> meaning is more-precise and closer to the intended behavior here. > > No, interleaving requires *two* sequences. This is interleave: > > interleave([], []) -> []; > interleave([X|Xs], [Y|Ys]) -> [X,Y|interleave(Xs,Ys)]. > > interleave, v, OED, sense 3: > "Computing and Telecommunications. To interfile (two or more > digitized signals or sequences of information) by alternating > between them; to alternate (one such signal or sequence) with > others. Also, to divide (memory, etc.) between a number of > different tasks by allocating successive segments to each in turn." > > (I must admit I hadn't encountered the word 'interfile' before.) The way I understand it is: - interleave: put 1 B between 2 As - intersperse: put M Bs between N As i.e. "intersperse" is more-general - interval length is unspecified. Is this wrong? From mjtruog@REDACTED Thu Mar 3 02:25:20 2016 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 02 Mar 2016 17:25:20 -0800 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <56D79280.2040202@gmail.com> On 03/02/2016 10:28 AM, Joe Armstrong wrote: > On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen > wrote: >> Hi Erlangers, >> >> I'd really like to add two functions to the lists module from Haskell: >> >> intersperse(List, Seperator) produces a list where each element is separated >> by separator, i.e. >> >> X = [1,2,3] >> [1, x, 2, x, 3] = lists:intersperse(X, x), >> >> and it's cousin, intercalate(ListOfLists, Separator) is >> append(intersperse(ListOfLists, Seperator)), i.e, >> >> Y = ["a", "b", "c"] >> "a, b, c" = lists:intercalate(Y, ", "), >> >> The implementations are straightforward and easy to write tests for, even >> property based tests if needed. >> >> The rationale for this proposal is that I find myself implementing this >> function again and again in every project I write, and it is highly generic. >> It belongs in a typical list module. OCaml libraries add it. Haskell's >> Data.List has it. I believe Erlang, being a practical language, should have >> it as well. >> > This is a splendid function - I have one myself called interleave, the only > problem is that if somebody uses this and puts their code in a library on github > and somebody fetches this code and runs it with an *old* version of erlang > then they will get a error and probably blame your code. > > This is why I have a single library called elib2_misc.erl where I put *all* the > stuff that should be somewhere else - then I ship this with my code. > > The whole story of dependency analysis seems to be be in a very sad state. > > I use the 'keep your fingers crossed and hope it will work' method. > > Years ago I wanted to add a -needs(Vsn) annotation to module > > For example: > > -needs(erl18). > > Means would mean you need version 18 of erlang. > > That way if you added a needs annotation to the code that used the updated lists > then an earlier version of Erlang could give a meaningfull diagnostic > and not just > crash when the encountering a function 'from the future'. > > Actually I had another problem today - a program I wrote in 2003 did not work > with the latest and greatest Erlang - not because the language had changed > but because the library functions had been renamed and moved around. https://github.com/erlang/eep/blob/master/eeps/eep-0044.md#the-otp_release-macro combined with https://github.com/erlang/eep/blob/master/eeps/eep-0044.md#the--error-directive should do what you want, though it is more verbose and flexible. > I guess a lint program would be useful here. It should be fairly > doable to download > all old erlangs and simple make lists of all the function names in all modules > and do a bit of consistency checking. Dialyzer seems like the Erlang lint program. If the Erlang install automatically generated a plt file for it, it might be a bit easier to use it as part of normal compilation. The alternative might be to use http://erlang.org/doc/man/xref.html as a command line tool, if the checking needs to only consider functions and not types. I am not aware of a standalone (escript) to run xref, but that may be useful for normal Erlang development (aside from rebar or other build tool usage). From ok@REDACTED Thu Mar 3 05:31:58 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Mar 2016 17:31:58 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56D7922B.1090102@khandkar.net> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> Message-ID: <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> On 3/03/16 2:23 pm, Siraaj Khandkar wrote: > >> interleave, v, OED, sense 3: >> "Computing and Telecommunications. To interfile (two or more >> digitized signals or sequences of information) by alternating >> between them; to alternate (one such signal or sequence) with >> others. Also, to divide (memory, etc.) between a number of >> different tasks by allocating successive segments to each in turn." >> >> (I must admit I hadn't encountered the word 'interfile' before.) > > The way I understand it is: > > - interleave: put 1 B between 2 As [A1,B1,A2,B2,A3,B3] will do. [A1,B1,A2,B2,A3,B3,A4] will do. Using the *same* B every time is not interleaving. A simple example of interleaving: put the fingers of one hand between the fingers of the other. Please people, do NOT use the word "interleaving" to refer to what the proposed "intersperse" function does. We need "interleaving" to refer to interleaving. > - intersperse: put M Bs between N As OED sense 1: "to scatter or sprinkle between or among other things; to place here and there in the course of something; to mingle dispersedly or at intervals." I don't think anyone looking at the OED would expect the Haskell function definition. The only justification for adopting the names intersperse and intercalate is that those *are* the names other not-entirely-dissimilar programming languages are using. From eric.pailleau@REDACTED Thu Mar 3 10:19:58 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 03 Mar 2016 10:19:58 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <6f9a235c-7d42-6e2f-a394-b165f9784dec@cs.otago.ac.nz> Message-ID: An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Mar 3 10:23:54 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 3 Mar 2016 10:23:54 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56D79280.2040202@gmail.com> References: <56D79280.2040202@gmail.com> Message-ID: <56D802AA.6040907@ninenines.eu> On 03/03/2016 02:25 AM, Michael Truog wrote: > The alternative > might be to use http://erlang.org/doc/man/xref.html as a command line > tool, if the checking > needs to only consider functions and not types. I am not aware of a > standalone (escript) > to run xref, but that may be useful for normal Erlang development (aside > from rebar or > other build tool usage). https://github.com/inaka/xref_runner -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From ola.a.andersson@REDACTED Thu Mar 3 10:25:24 2016 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Thu, 3 Mar 2016 09:25:24 +0000 Subject: [erlang-questions] On the wishlist was (RE: Proposal: add lists:intersperse/2 and lists:intercalate/2) Message-ID: <9DEFCD8EB8743E4EA623A12F453B66FC3D25CC61@ESESSMB207.ericsson.se> Hi List, While we're at it, there is a function I would like to see in proplists. I need a function that returns the same as this construct: [{Key, proplists:get_all_values(Key, PropList)} || Key <- proplists:get_keys(PropList)]. But of course with a more efficient implementation. I have implemented this in a number of different ways in pure erlang lately but a function in proplists would be really nice to have. No idea what to call it though. Maybe: proplists:get_all_key_values(proplist()). Not really that great. Better suggestions are welcome. /OLA. From hugo@REDACTED Thu Mar 3 10:29:22 2016 From: hugo@REDACTED (Hugo Mills) Date: Thu, 3 Mar 2016 09:29:22 +0000 Subject: [erlang-questions] On the wishlist was (RE: Proposal: add lists:intersperse/2 and lists:intercalate/2) In-Reply-To: <9DEFCD8EB8743E4EA623A12F453B66FC3D25CC61@ESESSMB207.ericsson.se> References: <9DEFCD8EB8743E4EA623A12F453B66FC3D25CC61@ESESSMB207.ericsson.se> Message-ID: <20160303092922.GE27786@carfax.org.uk> On Thu, Mar 03, 2016 at 09:25:24AM +0000, Ola Andersson A wrote: > Hi List, > While we're at it, there is a function I would like to see in proplists. > I need a function that returns the same as this construct: > > [{Key, proplists:get_all_values(Key, PropList)} || Key <- proplists:get_keys(PropList)]. > > But of course with a more efficient implementation. > I have implemented this in a number of different ways in pure erlang lately but a function in proplists would be really nice to have. > No idea what to call it though. Maybe: > > proplists:get_all_key_values(proplist()). > > Not really that great. Better suggestions are welcome. collect/1, maybe? You're collecting all the values for each key into a single list. Hugo. -- Hugo Mills | Klytus! Are your men on the right pills? Maybe you hugo@REDACTED carfax.org.uk | should execute their trainer! http://carfax.org.uk/ | PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From ola.a.andersson@REDACTED Thu Mar 3 10:30:56 2016 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Thu, 3 Mar 2016 09:30:56 +0000 Subject: [erlang-questions] On the wishlist was (RE: Proposal: add lists:intersperse/2 and lists:intercalate/2) In-Reply-To: <20160303092922.GE27786@carfax.org.uk> References: <9DEFCD8EB8743E4EA623A12F453B66FC3D25CC61@ESESSMB207.ericsson.se> <20160303092922.GE27786@carfax.org.uk> Message-ID: <9DEFCD8EB8743E4EA623A12F453B66FC3D25CC7E@ESESSMB207.ericsson.se> I actually have an implementation right now called key_collect ;-) /OLA. > -----Original Message----- > From: Hugo Mills [mailto:hugo@REDACTED] > Sent: den 3 mars 2016 10:29 > To: Ola Andersson A > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] On the wishlist was (RE: Proposal: add > lists:intersperse/2 and lists:intercalate/2) > > On Thu, Mar 03, 2016 at 09:25:24AM +0000, Ola Andersson A wrote: > > Hi List, > > While we're at it, there is a function I would like to see in proplists. > > I need a function that returns the same as this construct: > > > > [{Key, proplists:get_all_values(Key, PropList)} || Key <- > proplists:get_keys(PropList)]. > > > > But of course with a more efficient implementation. > > I have implemented this in a number of different ways in pure erlang lately > but a function in proplists would be really nice to have. > > No idea what to call it though. Maybe: > > > > proplists:get_all_key_values(proplist()). > > > > Not really that great. Better suggestions are welcome. > > collect/1, maybe? You're collecting all the values for each key into a single > list. > > Hugo. > > -- > Hugo Mills | Klytus! Are your men on the right pills? Maybe you > hugo@REDACTED carfax.org.uk | should execute their trainer! > http://carfax.org.uk/ | > PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon From Chandru.Mullaparthi@REDACTED Thu Mar 3 18:06:32 2016 From: Chandru.Mullaparthi@REDACTED (Chandru.Mullaparthi@REDACTED) Date: Thu, 3 Mar 2016 17:06:32 +0000 Subject: [erlang-questions] [ANN] soap library for Erlang Message-ID: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Hi all, We at bet365 have been using Erlang for a while now and enjoying the benefits it provides. So I'm pleased to announce our first open source contribution to the Erlang community. An application named 'soap' is now available on GitHub which allows you to consume and expose web services easily from Erlang. https://github.com/bet365/soap SOAP isn't exactly considered sexy these days but it is still quite widely used in the Enterprise space. I've personally known instances where development teams have shied away from using Erlang purely because of the complete lack of support for SOAP. So hopefully this application fills that gap and will encourage more enterprise developers to take up Erlang. We are also currently working on an improved version of the 'odbc' application which we will be releasing in the coming weeks. bet365 employed Willem de Jong (author of https://github.com/willemdj/erlsom ) to develop this application (and the required enhancements to erlsom) with a little bit of design input from me. Suggestions for improvements, pull requests and any feedback is most welcome. regards, Chandru -- Chandru Mullaparthi Head of Software Architecture Hillside (Technology) Limited bet365.com This email and any files transmitted with it are confidential and contain information which may be privileged or confidential and are intended solely to be for the use of the individual(s) or entity to which they are addressed. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information is strictly prohibited and may be illegal. If you have received this email in error, please notify us by telephone or email immediately and delete it from your system. Activity and use of our email system is monitored to secure its effective operation and for other lawful business purposes. Communications using this system will also be monitored and may be recorded to secure effective operation and for other lawful business purposes. Internet emails are not necessarily secure. We do not accept responsibility for changes made to this message after it was sent. You are advised to scan this message for viruses and we cannot accept liability for any loss or damage which may be caused as a result of any computer virus. This email is sent by a bet365 group entity. The bet365 group includes the following entities: Hillside (Shared Services) Limited (registration no. 3958393), Hillside (Spain New Media) Plc (registration no. 07833226), bet365 Group Limited (registration no. 4241161), Hillside (Technology) Limited (registration no. 8273456), Hillside (Media Services) Limited (registration no. 9171710), Hillside (Trader Services) Limited (registration no. 9171598) each registered in England and Wales with a registered office address at Hillside, Festival Way, Stoke-on-Trent, Staffordshire, ST1 5SH; Hillside (Gibraltar) Limited (registration no. 97927), Hillside (Sports) GP Limited (registration no. 111829) and Hillside (Gaming) GP Limited (registered no. 111830) each registered in Gibraltar with a registered office address at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside (UK Sports) LP (registration no. 117), Hillside (Sports) LP (registration no. 118), Hillside (International Sports) LP (registration no. 119), Hillside (Gaming) LP (registration no. 120) and Hillside (International Gaming) LP (registration no. 121) each registered in Gibraltar with a principal place of business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside Espa?a Leisure S.A (CIF no. A86340270) registered in Spain with a registered office address at C/ Conde de Aranda n?20, 2?, 28001 Madrid, Spain; and Hillside (Australia New Media) Pty Limited (registration no. 148 920 665) registered in Australia with a registered office address at Level 4, 90 Arthur Street, North Sydney, NSW 2060, Australia. Hillside (Shared Services) Limited and Hillside (Spain New Media) Plc also have places of business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar. For residents of Greece, this email is sent on behalf of B2B Gaming Services (Malta) Limited (registration number C41936) organised under the laws of Malta with a registered office at Apartment 21, Suite 41, Charles Court, St. Luke's Road, Piet?, Malta. From chandrashekhar.mullaparthi@REDACTED Thu Mar 3 20:34:48 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 3 Mar 2016 19:34:48 +0000 Subject: [erlang-questions] Fwd: [ANN] soap library for Erlang In-Reply-To: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: For some reason this ended up in my Gmail spam folder, and I suspect it will be the same problem for many subscribers. Chandru ---------- Forwarded message ---------- From: Date: 3 March 2016 at 17:06 Subject: [erlang-questions] [ANN] soap library for Erlang To: erlang-questions@REDACTED Hi all, We at bet365 have been using Erlang for a while now and enjoying the benefits it provides. So I'm pleased to announce our first open source contribution to the Erlang community. An application named 'soap' is now available on GitHub which allows you to consume and expose web services easily from Erlang. https://github.com/bet365/soap SOAP isn't exactly considered sexy these days but it is still quite widely used in the Enterprise space. I've personally known instances where development teams have shied away from using Erlang purely because of the complete lack of support for SOAP. So hopefully this application fills that gap and will encourage more enterprise developers to take up Erlang. We are also currently working on an improved version of the 'odbc' application which we will be releasing in the coming weeks. bet365 employed Willem de Jong (author of https://github.com/willemdj/erlsom ) to develop this application (and the required enhancements to erlsom) with a little bit of design input from me. Suggestions for improvements, pull requests and any feedback is most welcome. regards, Chandru -- Chandru Mullaparthi Head of Software Architecture Hillside (Technology) Limited bet365.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Fri Mar 4 02:24:27 2016 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 04 Mar 2016 09:24:27 +0800 Subject: [erlang-questions] [ANN] soap library for Erlang References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: On 2016-03-03 17:06 +0000, Chandru.Mullaparthi@REDACTED wrote: > We are also currently working on an improved version of the 'odbc' > application which we will be releasing in the coming weeks. I am looking forward to this. Can you make something like the following work in ODBC: odbc:sql_query(Ref, "select '??'") Thanks, Leo From jisaacstone@REDACTED Fri Mar 4 05:47:45 2016 From: jisaacstone@REDACTED (isaac stone) Date: Thu, 3 Mar 2016 20:47:45 -0800 Subject: [erlang-questions] Status of map pair support in dialyzer Message-ID: I noticed today that dialyzer was failing to catch errors. Research led me to a previous question on this mailing list where Jos? quoted a note in the typespec reference stating "No type information of maps pairs, only the containing map types, are used by Dialyzer in OTP 17." 2 Questions: - is this also true in OTP 18 - is there somewhere I can track and/or contribute to this issue? (sorry if duplicate - seems my question did not go through yesterday) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Mar 4 06:14:20 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 4 Mar 2016 18:14:20 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: References: Message-ID: <7fd22b60-3c5e-ce70-25da-c561bb3abdda@cs.otago.ac.nz> On 3/03/16 10:19 pm, ?ric Pailleau wrote: I think it was Griswold, in the context of the SL5 programming language (the one that came between SNOBOL and Icon), who coined the term "The Window to Hell". The SL5 group used the term for aspects of SL5 that let you look down into the workings of the implementation, in order to support tracing and debugging. Here's the reference: The WIndow to Hell in SL5. Ralph E. Griswold. SL5 Project Document s5LD8a The University of Arizona June 1, 1976 (If anyone has electronic copies of the SL5 project documents, I'd be interested in seeing them. I read this one years ago, but never owned a copy.) What's that got to do with Erlang? First, Erlang can accept user-defined C code. If you accept an incubus (otherwise known as a NIF) into your bed (otherwise known as the Erlang VM), you must not be surprised if you give birth to monsters. Whatever integrity guarantees Erlang might offer are completely void if you do that. (This is not to say that there could not be some safety-checked subset of C that could be safely added to Erlang, or better still, some sort of BitC or D or Go variant that gets close to C speed without C's reckless disregard for life, health, and sanity. It just hasn't been done yet. Maybe the right thing to do is like Squeak's Slang: use a variant of Erlang itself as the safe+fast extension language?) Second, even pure Erlang has a Window to Hell. exit(Pid, Reason) can kill *any* process that is not trapping exits. group_leader(Leader, Pid) can set the group leader of any process, willing or unwilling, knowing or unknowing list_to_pid(String) lets you forge pids erlang:ports() returns a list of all the ports on the local node erlang:processes() returns a list of all the processes on the local node erlang:resume_process(Pid) and erlang:suspend_process(Pid[, Options]) let you break things in a way that even Java's designers couldn't stomach any longer. and those are just the ones I know about. The result is that you can NEVER assume, for example, that a process that has no reason to suspend *isn't* permanently suspended. In the Erlang shell, type > [erlang:suspend_process(P) || P <- erlang:processes(), P /= self()]. and watch your shell never come back... IMHO all such dangerous operations should be moved out of 'erlang' into some separate module (dangerous_internal_erlang_stuff, perhaps) to be used by debugging and tracing tools, with the compiler warning by default if you mention that module. It ought to be even *easier* than getting a cross-reference to KNOW that your application (including any libraries or frameworks) doesn't use these operations. When I try to design "safe" patterns of communication in Erlang, I always have this tacit "no looking through the Window to Hell, not inviting succubi into my bed" qualification in mind. > > However when you say " only processes that can send you messages are > yourself and > your parent, until one of you passes the pid on to some > other process. " I do not understand. > You are right that in unrestrained Erlang this is not true. I had my "no hell, no succubi" qualification in mind. So in a non-demon-haunted Erlang, if process P spawns process C, then P knows C's pid and can send messages to it, and C knows its own pid, and can send messages to itself, but nobody else can until P or C tells them. In the actual demon-haunted Erlang, very bad things can happen and there is no way to stop them Much the same objection was raised to Quintus Prolog. The module system did many nice things for you, but it completely failed to provide *trustworthy* isolation, in order that the debugger (itself implemented in Prolog) could do the things programmers wanted the debugger to do. (The same thing with the Common Lisp systems I knew anything about, not that that was many of them.) I not saying "don't HAVE a window to hell", what I'm saying is that there needs to be a fence around it. > > Any process can list all running process at any time and potentially > send messages to them. > You mean those pids have to be hidden, while existing in process > registry ? > From ok@REDACTED Fri Mar 4 06:23:29 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 4 Mar 2016 18:23:29 +1300 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: <8d3d0d2e-6473-5e1f-1811-59507aeafeee@cs.otago.ac.nz> On 4/03/16 6:06 am, Chandru.Mullaparthi@REDACTED wrote: > So I'm pleased to announce our first open source contribution > to the Erlang community. An application named 'soap' is now > available on GitHub which allows you to consume and expose > web services easily from Erlang. I see that you've even written documentation worth having for it too. I want to say something nice without gushing. "How kind of you?" "Well done?" Something along those lines. Frankly, having tried to read the SOAP specification, I think you're programming _heroes_. From chandrashekhar.mullaparthi@REDACTED Fri Mar 4 08:28:04 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 4 Mar 2016 07:28:04 +0000 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: On 4 March 2016 at 01:24, Leo Liu wrote: > On 2016-03-03 17:06 +0000, Chandru.Mullaparthi@REDACTED wrote: > > We are also currently working on an improved version of the 'odbc' > > application which we will be releasing in the coming weeks. > > I am looking forward to this. > > Can you make something like the following work in ODBC: > > odbc:sql_query(Ref, "select '??'") > > I'll pass it on internally to the developer working on this and get back to you. regards, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri Mar 4 08:35:45 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 4 Mar 2016 07:35:45 +0000 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: <8d3d0d2e-6473-5e1f-1811-59507aeafeee@cs.otago.ac.nz> References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> <8d3d0d2e-6473-5e1f-1811-59507aeafeee@cs.otago.ac.nz> Message-ID: On 4 March 2016 at 05:23, Richard A. O'Keefe wrote: > On 4/03/16 6:06 am, Chandru.Mullaparthi@REDACTED wrote: > > > So I'm pleased to announce our first open source contribution > > to the Erlang community. An application named 'soap' is now > > available on GitHub which allows you to consume and expose > > web services easily from Erlang. > > I see that you've even written documentation worth having for > it too. I want to say something nice without gushing. > "How kind of you?" "Well done?" Something along those lines. > > Frankly, having tried to read the SOAP specification, I think > you're programming _heroes_. > > Thank you Richard, coming from you I'll take that as high praise :-) I used to have a quote (I think from the Tex manual by someone named Dick Brandon if I'm not mistaken) on my desk which said something like: "Documentation is like sex. When it is good, it is very very good. When it is bad it is better than nothing" and use that as a justification for not writing good (any) documentation. But I've grown up now and know better that without good documentation, the work is only half done, if that. To give credit where credit is due, Willem wrote all the code and the lion's share of the documentation. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From w.a.de.jong@REDACTED Fri Mar 4 08:47:16 2016 From: w.a.de.jong@REDACTED (Willem de Jong) Date: Fri, 4 Mar 2016 08:47:16 +0100 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: Hi Chandru, Your mail ended up in my Gmail Spam folder, and I suspect the same happened for many other subscribers. Probably Google don't approve of SOAP. I can understand that, but still it would be a pity if this message was lost. There may be people out there who don't particularly like SOAP either, but who never the less are forced to use it. Hopefully the new application will be useful for them. So here it is again. Apologies to those of you who are receiving this twice, regards, Willem On Thu, Mar 3, 2016 at 6:06 PM, wrote: > Hi all, > > We at bet365 have been using Erlang for a while now and enjoying the > benefits it provides. So I'm pleased to announce our first open source > contribution to the Erlang community. An application named 'soap' is now > available on GitHub which allows you to consume and expose web services > easily from Erlang. > > https://github.com/bet365/soap > > SOAP isn't exactly considered sexy these days but it is still quite widely > used in the Enterprise space. I've personally known instances where > development teams have shied away from using Erlang purely because of the > complete lack of support for SOAP. So hopefully this application fills that > gap and will encourage more enterprise developers to take up Erlang. > > We are also currently working on an improved version of the 'odbc' > application which we will be releasing in the coming weeks. > > bet365 employed Willem de Jong (author of > https://github.com/willemdj/erlsom ) to develop this application (and the > required enhancements to erlsom) with a little bit of design input from me. > Suggestions for improvements, pull requests and any feedback is most > welcome. > > regards, > Chandru > -- > Chandru Mullaparthi > Head of Software Architecture > Hillside (Technology) Limited > bet365.com > > This email and any files transmitted with it are confidential and contain > information which may be privileged or confidential and are intended solely > to be for the use of the individual(s) or entity to which they are > addressed. If you are not the intended recipient be aware that any > disclosure, copying, distribution or use of the contents of this > information is strictly prohibited and may be illegal. If you have received > this email in error, please notify us by telephone or email immediately and > delete it from your system. Activity and use of our email system is > monitored to secure its effective operation and for other lawful business > purposes. Communications using this system will also be monitored and may > be recorded to secure effective operation and for other lawful business > purposes. Internet emails are not necessarily secure. We do not accept > responsibility for changes made to this message after it was sent. You are > advised to scan this message for viruses and we cannot accept liability for > any loss or damage which may be caused as a result of any computer virus. > > This email is sent by a bet365 group entity. The bet365 group includes the > following entities: Hillside (Shared Services) Limited (registration no. > 3958393), Hillside (Spain New Media) Plc (registration no. 07833226), > bet365 Group Limited (registration no. 4241161), Hillside (Technology) > Limited (registration no. 8273456), Hillside (Media Services) Limited > (registration no. 9171710), Hillside (Trader Services) Limited > (registration no. 9171598) each registered in England and Wales with a > registered office address at Hillside, Festival Way, Stoke-on-Trent, > Staffordshire, ST1 5SH; Hillside (Gibraltar) Limited (registration no. > 97927), Hillside (Sports) GP Limited (registration no. 111829) and Hillside > (Gaming) GP Limited (registered no. 111830) each registered in Gibraltar > with a registered office address at Unit 1.1, First Floor, Waterport Place, > 2 Europort Avenue, Gibraltar; Hillside (UK Sports) LP (registration no. > 117), Hillside (Sports) LP (registration no. 118), Hillside (International > Sports) LP (registration no. 119), Hillside (Gaming) LP (registration no. > 120) and Hillside (International Gaming) LP (registration no. 121) each > registered in Gibraltar with a principal place of business at Unit 1.1, > First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside Espa?a > Leisure S.A (CIF no. A86340270) registered in Spain with a registered > office address at C/ Conde de Aranda n?20, 2?, 28001 Madrid, Spain; and > Hillside (Australia New Media) Pty Limited (registration no. 148 920 665) > registered in Australia with a registered office address at Level 4, 90 > Arthur Street, North Sydney, NSW 2060, Australia. Hillside (Shared > Services) Limited and Hillside (Spain New Media) Plc also have places of > business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, > Gibraltar. For residents of Greece, this email is sent on behalf of B2B > Gaming Services (Malta) Limited (registration number C41936) organised > under the laws of Malta with a registered office at Apartment 21, Suite 41, > Charles Court, St. Luke's Road, Piet?, Malta. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Mar 4 09:09:00 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 4 Mar 2016 09:09:00 +0100 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: This email and any files transmitted with it are confidential and contain >> information which may be privileged or confidential and are intended solely >> to be for the use of the individual(s) or entity to which they are >> addressed. If you are not the intended recipient be aware that any >> disclosure, copying, distribution or use of the contents of this >> information is strictly prohibited and may be illegal. If you have received >> this email in error, please notify us by telephone or email immediately and >> delete it from your system. Activity and use of our email system is >> monitored to secure its effective operation and for other lawful business >> purposes. Communications using this system will also be monitored and may >> be recorded to secure effective operation and for other lawful business >> purposes. Internet emails are not necessarily secure. We do not accept >> responsibility for changes made to this message after it was sent. You are >> advised to scan this message for viruses and we cannot accept liability for >> any loss or damage which may be caused as a result of any computer virus. >> >> This email is sent by a bet365 group entity. The bet365 group includes >> the following entities: Hillside (Shared Services) Limited (registration >> no. 3958393), Hillside (Spain New Media) Plc (registration no. 07833226), >> bet365 Group Limited (registration no. 4241161), Hillside (Technology) >> Limited (registration no. 8273456), Hillside (Media Services) Limited >> (registration no. 9171710), Hillside (Trader Services) Limited >> (registration no. 9171598) each registered in England and Wales with a >> registered office address at Hillside, Festival Way, Stoke-on-Trent, >> Staffordshire, ST1 5SH; Hillside (Gibraltar) Limited (registration no. >> 97927), Hillside (Sports) GP Limited (registration no. 111829) and Hillside >> (Gaming) GP Limited (registered no. 111830) each registered in Gibraltar >> with a registered office address at Unit 1.1, First Floor, Waterport Place, >> 2 Europort Avenue, Gibraltar; Hillside (UK Sports) LP (registration no. >> 117), Hillside (Sports) LP (registration no. 118), Hillside (International >> Sports) LP (registration no. 119), Hillside (Gaming) LP (registration no. >> 120) and Hillside (International Gaming) LP (registration no. 121) each >> registered in Gibraltar with a principal place of business at Unit 1.1, >> First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside Espa?a >> Leisure S.A (CIF no. A86340270) registered in Spain with a registered >> office address at C/ Conde de Aranda n?20, 2?, 28001 Madrid, Spain; and >> Hillside (Australia New Media) Pty Limited (registration no. 148 920 665) >> registered in Australia with a registered office address at Level 4, 90 >> Arthur Street, North Sydney, NSW 2060, Australia. Hillside (Shared >> Services) Limited and Hillside (Spain New Media) Plc also have places of >> business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, >> Gibraltar. For residents of Greece, this email is sent on behalf of B2B >> Gaming Services (Malta) Limited (registration number C41936) organised >> under the laws of Malta with a registered office at Apartment 21, Suite 41, >> Charles Court, St. Luke's Road, Piet?, Malta. > > This. This wins the corporate signature size game by a factor of 3 to the nearest contender! I'll buy you a round of beers in the SOAP team. One, because you are going to need it after having implemented SOAP. Two, because you have to handle THAT signature every day. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Fri Mar 4 09:18:54 2016 From: marc@REDACTED (Marc Worrell) Date: Fri, 4 Mar 2016 09:18:54 +0100 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: <6CC25655-E134-4E39-B02A-73AA43214841@worrell.nl> And above that this signature is a minefield of spam-filter trapping words. A wonder some mail still gets through? - Marc > On 4 mrt. 2016, at 09:09, Jesper Louis Andersen wrote: > > > > This email and any files transmitted with it are confidential and contain information which may be privileged or confidential and are intended solely to be for the use of the individual(s) or entity to which they are addressed. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information is strictly prohibited and may be illegal. If you have received this email in error, please notify us by telephone or email immediately and delete it from your system. Activity and use of our email system is monitored to secure its effective operation and for other lawful business purposes. Communications using this system will also be monitored and may be recorded to secure effective operation and for other lawful business purposes. Internet emails are not necessarily secure. We do not accept responsibility for changes made to this message after it was sent. You are advised to scan this message for viruses and we cannot accept liability for any loss or damage which may be caused as a result of any computer virus. > > This email is sent by a bet365 group entity. The bet365 group includes the following entities: Hillside (Shared Services) Limited (registration no. 3958393), Hillside (Spain New Media) Plc (registration no. 07833226), bet365 Group Limited (registration no. 4241161), Hillside (Technology) Limited (registration no. 8273456), Hillside (Media Services) Limited (registration no. 9171710), Hillside (Trader Services) Limited (registration no. 9171598) each registered in England and Wales with a registered office address at Hillside, Festival Way, Stoke-on-Trent, Staffordshire, ST1 5SH; Hillside (Gibraltar) Limited (registration no. 97927), Hillside (Sports) GP Limited (registration no. 111829) and Hillside (Gaming) GP Limited (registered no. 111830) each registered in Gibraltar with a registered office address at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside (UK Sports) LP (registration no. 117), Hillside (Sports) LP (registration no. 118), Hillside (International Sports) LP (registration no. 119), Hillside (Gaming) LP (registration no. 120) and Hillside (International Gaming) LP (registration no. 121) each registered in Gibraltar with a principal place of business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar; Hillside Espa?a Leisure S.A (CIF no. A86340270) registered in Spain with a registered office address at C/ Conde de Aranda n?20, 2?, 28001 Madrid, Spain; and Hillside (Australia New Media) Pty Limited (registration no. 148 920 665) registered in Australia with a registered office address at Level 4, 90 Arthur Street, North Sydney, NSW 2060, Australia. Hillside (Shared Services) Limited and Hillside (Spain New Media) Plc also have places of business at Unit 1.1, First Floor, Waterport Place, 2 Europort Avenue, Gibraltar. For residents of Greece, this email is sent on behalf of B2B Gaming Services (Malta) Limited (registration number C41936) organised under the laws of Malta with a registered office at Apartment 21, Suite 41, Charles Court, St. Luke's Road, Piet?, Malta. > > This. > > This wins the corporate signature size game by a factor of 3 to the nearest contender! > > I'll buy you a round of beers in the SOAP team. One, because you are going to need it after having implemented SOAP. Two, because you have to handle THAT signature every day. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ola.Backstrom@REDACTED Fri Mar 4 10:47:59 2016 From: Ola.Backstrom@REDACTED (=?utf-8?B?T2xhIELDpGNrc3Ryw7Zt?=) Date: Fri, 4 Mar 2016 09:47:59 +0000 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> <8d3d0d2e-6473-5e1f-1811-59507aeafeee@cs.otago.ac.nz> Message-ID: I have not yet had to do any Soap, but I must add praise for the published work: Well documented (with good figures!), clean source code from a quick glance, tests available and all without any NIFs. In the future, if I?d ever face a Soap-dragon, I feel I could use this library as armour and mount my trusty old Erlang-horse ? Best regards /Ola From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Chandru Sent: den 4 mars 2016 08:36 To: Richard A. O'Keefe Cc: Erlang Questions Subject: Re: [erlang-questions] [ANN] soap library for Erlang On 4 March 2016 at 05:23, Richard A. O'Keefe > wrote: On 4/03/16 6:06 am, Chandru.Mullaparthi@REDACTED wrote: > So I'm pleased to announce our first open source contribution > to the Erlang community. An application named 'soap' is now > available on GitHub which allows you to consume and expose > web services easily from Erlang. I see that you've even written documentation worth having for it too. I want to say something nice without gushing. "How kind of you?" "Well done?" Something along those lines. Frankly, having tried to read the SOAP specification, I think you're programming _heroes_. Thank you Richard, coming from you I'll take that as high praise :-) I used to have a quote (I think from the Tex manual by someone named Dick Brandon if I'm not mistaken) on my desk which said something like: "Documentation is like sex. When it is good, it is very very good. When it is bad it is better than nothing" and use that as a justification for not writing good (any) documentation. But I've grown up now and know better that without good documentation, the work is only half done, if that. To give credit where credit is due, Willem wrote all the code and the lion's share of the documentation. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Fri Mar 4 11:59:41 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Fri, 4 Mar 2016 11:59:41 +0100 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: References: Message-ID: <56D96A9D.6060502@ericsson.com> On 2016-03-04 05:47, isaac stone wrote: > 2 Questions: > - is this also true in OTP 18 Yes. > - is there somewhere I can track and/or contribute to this issue? I don't believe an issue has been filed at our public bug-tracker, http://bugs.erlang.org, but we do have it in our internal product-backlog. This does not mean it will be done by us anytime soon. I did the initial work for maps in Dialyzer but that application is not really part of my work area. If you want to contribute to Dialyzer you should probably talk to Hans Bolinder, Kostis Sagonas and/or Stavros Aronis. Hans talked about increasing Dialyzers knowledge about map associations the other day but I don't think it is on his agenda at the moment. I don't think it will be supported in 19.0 either though I have no clue if Kostis or Stavros, or anyone else for that matter, is working on it. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Mar 4 12:59:23 2016 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Fri, 4 Mar 2016 12:59:23 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <7fd22b60-3c5e-ce70-25da-c561bb3abdda@cs.otago.ac.nz> References: <7fd22b60-3c5e-ce70-25da-c561bb3abdda@cs.otago.ac.nz> Message-ID: <56D9789B.4060804@wanadoo.fr> Le 04/03/2016 06:14, Richard A. O'Keefe a ?crit : > Second, even pure Erlang has a Window to Hell. > exit(Pid, Reason) can kill *any* process that is not trapping exits. > group_leader(Leader, Pid) can set the group leader of any process, > willing or unwilling, knowing or unknowing > list_to_pid(String) lets you forge pids > erlang:ports() returns a list of all the ports on the local node > erlang:processes() returns a list of all the processes on the local > node > erlang:resume_process(Pid) and > erlang:suspend_process(Pid[, Options]) let you break things in a way > that even Java's designers couldn't stomach any longer. > and those are just the ones I know about. The result is that you can NEVER > assume, for example, that a process that has no reason to suspend *isn't* > permanently suspended. In the Erlang shell, type > > [erlang:suspend_process(P) || P <- erlang:processes(), P /= self()]. > and watch your shell never come back... not to mention : io:write(self(), 'something'). that never return. Good syntax is io:write(group_leader(), 'something'). Being in a shell and self() is not the pid that can handle io is very strange for new comers (even if it is understandable by reading the code and documentation). I proposed a fix that replace self() by group leader, but was rejected. My feeling : a soft realtime software that have functions that never return (at least a timeout) is really weird and in some way do not follow the 'let it crash' philosophy . But... C'est la vie. > I not saying "don't HAVE a window to hell", what I'm saying > is that there needs to be a fence around it. spawn_priv/x could be a brick in this goal, but as far security is not a subject in Erlang VM, we must live listening AC/DC's "Hell ain't a bad place to be" , doing orgies with succubi . ;>) From siraaj@REDACTED Fri Mar 4 16:33:57 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 4 Mar 2016 10:33:57 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> Message-ID: <56D9AAE5.8080500@khandkar.net> On 3/2/16 11:31 PM, Richard A. O'Keefe wrote: > > On 3/03/16 2:23 pm, Siraaj Khandkar wrote: >> >>> interleave, v, OED, sense 3: >>> "Computing and Telecommunications. To interfile (two or more >>> digitized signals or sequences of information) by alternating >>> between them; to alternate (one such signal or sequence) with >>> others. Also, to divide (memory, etc.) between a number of >>> different tasks by allocating successive segments to each in turn." >>> >>> (I must admit I hadn't encountered the word 'interfile' before.) >> >> The way I understand it is: >> >> - interleave: put 1 B between 2 As > [A1,B1,A2,B2,A3,B3] will do. > [A1,B1,A2,B2,A3,B3,A4] will do. > Using the *same* B every time is not interleaving. > The notion of sameness here is problematic, since you're now requiring that the above-quoted "sequences of information" are sets, which is unlikely the intended meaning. If we have no uniqueness or distribution (i.e. how long until a repeat is allowed) requirements for our two sequences, then there's no useful, semantic distinction between a sequence of copied elements and a sequence of randomly-generated elements. Neither "interleaving" nor "interspersing" say anything about the contents of the two sequences they operate on, where they differ is in the degree to which they care about intervals between elements: - "interleaving" promises a process which picks 1 element from A, then 1 element from B; - "interspersing" only promises that some (undefined number of) elements from A will end-up separated by some (undefined number of) elements from B. One real problem, which goes against the requirement of the function in-question, is that both, [a, b, a] and [a, b, a, b] seem to be allowed by "interleave", but maybe not by "intersperse", but I'm not really sure about this... >> - intersperse: put M Bs between N As > OED sense 1: "to scatter or sprinkle between or among other things; > to place here and there in the course of something; to mingle > dispersedly or at intervals." > > I don't think anyone looking at the OED would expect the Haskell > function definition. The only justification for adopting the names > intersperse and intercalate is that those *are* the names other > not-entirely-dissimilar programming languages are using. > Just merely copying what has been done before, without looking into the reasons, misses a valuable opportunity for improvement. From roberto.ostinelli@REDACTED Fri Mar 4 21:40:12 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 4 Mar 2016 21:40:12 +0100 Subject: [erlang-questions] mnesia:dirty_write in cluster Message-ID: <3BED4EFE-6403-4C7A-8EEE-5485BEA4AB12@widetag.com> Dear all, Why does mnesia:dirty_write/1,2 take longer the more nodes there are in a cluster? I assume it is because it is sending this info to the other nodes as well? If so, is there any way to instruct mnesia to do the synch to other nodes asynchronously and free the proc doing the write asap? Thank you, r. From roberto.ostinelli@REDACTED Fri Mar 4 21:46:07 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 4 Mar 2016 21:46:07 +0100 Subject: [erlang-questions] mnesia:dirty_write in cluster In-Reply-To: <3BED4EFE-6403-4C7A-8EEE-5485BEA4AB12@widetag.com> References: <3BED4EFE-6403-4C7A-8EEE-5485BEA4AB12@widetag.com> Message-ID: I'm assuming I have to use async_dirty, however reading the docs I still have some doubts if using mnesia:dirty_write/1,2 not wrapped into an activity uses by default sync_dirty or not. > On 04 mar 2016, at 21:40, Roberto Ostinelli wrote: > > Dear all, > Why does mnesia:dirty_write/1,2 take longer the more nodes there are in a cluster? I assume it is because it is sending this info to the other nodes as well? > > If so, is there any way to instruct mnesia to do the synch to other nodes asynchronously and free the proc doing the write asap? > > Thank you, > r. From jesper.louis.andersen@REDACTED Sat Mar 5 11:03:17 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 5 Mar 2016 11:03:17 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: So to catch up: * I need to learn how to spell separator :) * Indeed, the argument order should be intersperse(Sep, Xs), not the other way around. This looks consistent with most List functions. * On Joe's comments of why not simply a utility library: This would have been fine, were it not for the fact that Erlang is miserably bad at handling ad-hoc utility libraries for other modules. There is no way I can hide joe_utils.erl since it is a globally known name. This creates lots of versioning problems down the road. In e.g., Standard ML or OCaml, I could just package the modules such that 'module JoeUtils' is not exported outside the scope of the individual libraries and applications. But Erlang is not that language. I also think intersperse/intercalate are so often used they shouldn't be in everyones utils library but should be put in the stdlib base. * On Richard's comments of other uses than for a string: The obvious use in Erlang is to produce iolist() types for binary data: intersperse($, [<<"a">>, <<"b">>, <<"c">>]) is relatively common in my code. The slightly less obvious use is to interleave a simplifier step in between every optimization step of a compiler, but I've only done that once in my life and I'm not sure it is that useful :) One could argue we should just have tooling such as Hughes-combinators[0] for output rather than the singular intersperse function, but it is by the far the missing tool in the toolbox when trying to use standard Erlang as were it a pretty-printing combinator library. * On the name choice of 'intersperse/intercalate': I think the semantics I'm aiming for is so close to the OCaml(Core)/Haskell implementations I'm willing to reuse that name. Another valid name is 'join' which is common in many other languages. I don't think one is better than the other, but the FP roots in me says we should adopt FP-language naming. NEXT STEP: I'd like to produce the necessary PR for 'intersperse' first, which gives me an idea of where to put changes. This builds a small unit of work which should be easy to make concrete. We can always augment this with additional functions later.' We need at least: * The implementation in lists.erl (I don't think we need to make it fast right away, so plain Erlang it is) * Documentation of the function, with examples * Tests with good coverage of the function. Once this is done, adding 'intercalate' is easy, but the question is if it is needed since once can just 'append' after having interspersed. The power of having a special function is that it is amenable to certain optimizations if they are kept as a larger block. [0] The thing I have in mind here is John Hughes "The Design of a Pretty-printing library" http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.38.8777 On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is > separated by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > Y = ["a", "b", "c"] > "a, b, c" = lists:intercalate(Y, ", "), > > The implementations are straightforward and easy to write tests for, even > property based tests if needed. > > The rationale for this proposal is that I find myself implementing this > function again and again in every project I write, and it is highly > generic. It belongs in a typical list module. OCaml libraries add it. > Haskell's Data.List has it. I believe Erlang, being a practical language, > should have it as well. > > Thoughts? > > -- > J. > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Mar 5 11:46:44 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 05 Mar 2016 10:46:44 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: <56D5094E.4070607@khandkar.net> References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> <56d4a3dc.45ce190a.394c6.609a@mx.google.com> <56D5094E.4070607@khandkar.net> Message-ID: So I finally did it the old ways using records and accessor functions over it. To answer to joe and other, the main goal was to have more clarity and consistency over the time in the way you will access the data. Right now the data structure is pretty clear and can only be augmented if I have to introduce new properties. So I guess it will be OK. (this thread was very interesting to learn how others uses the records, we should have more topics like this :) which let me remember http://www.erlangpatterns.org/ ..., I wonder how the project works as of today) - benoit On Tue, Mar 1, 2016 at 4:15 AM Siraaj Khandkar wrote: > On 2/29/16 3:02 PM, Torben Hoffmann wrote: > > > > > > Jesper Louis Andersen writes: > > > >> [ text/plain ] > >> On Mon, Feb 29, 2016 at 4:28 PM, Siraaj Khandkar > >> wrote: > >> > >>> In other words, I'm saying it is very possible to have this cake (of > >>> public structure) and eat it too (in private, without anyone accessing > the > >>> rapidly-changing, secret cream) :) > >> > >> > >> You are so close to the idea of view types, and Torben hinted me I > should > >> say something about them :) > >> > >> One advantage of a function is that it doesn't have to give you the > field > >> outright, but can provide ways to dynamically cast that structure into > >> another structure you export. Two really good examples is that you can > >> return some kind of tuple depending on the contents of the data and thus > >> you can build a case match on the data. This ad-hoc > >> for-the-purpose-construction of a datatype can often yield far more > precise > >> code since it splits analysis of what is *in* the datastructure from > >> computation which says what to *do* about that data. Say we have a > #uri{} > >> record. The path component has many possible representations: a > binary(), a > >> list of binaries, an iterator, and so on. Different view-functions would > >> give you different ways to handle that binary. > >> > >> Another nice thing is that functions can give you are > >> zipper/derivative/delimiting-continuation over the data in the > structure. > >> The path-component can then be unfolded one path-entry at a time: > >> > >> pth(X) -> > >> case http_uri:unfold_path(X) of > >> none -> ...; > >> {some, C, X2} -> > >> ... C ... pth(X2) > >> end. > >> > >> or you can imagine a gb_trees/gb_sets like iterator over the data > structure. > >> > >> A plain map() cannot give any of these ideas, and its transparency also > >> tightly couples your code to the map() structure. So one has to > carefully > >> weigh what kind of interface you want in the long run. I much prefer > views > >> of the data structures if possible since it allows for more freedom in > the > >> long run, but it requires you to be able to figure out what kind of > >> functions you would need a priori on the data. On the other hand, it > >> provides for much better hiding of what is going on inside the module, > >> which allows you more flexibility regarding backwards compatibility. > >> > >> In other words: exporting records across application boundaries tend to > >> lock them down, so be pretty sure they are set in stone. Also think > hard if > >> you want every user to implement the analysis code. The view is much > like > >> in a database: do you export the raw records, or do you expose a > >> transactional API through stored procedures which tell you what you can > do > >> with the data? > >> > >> > > Why attempt to write a half-baked answer when you can ask Jesper to > > explain it well?!?! [1] > > > > Look at Cowboy and Webmachine for examples of how to provide functions > > to work with a request. > > > > Both have a very rich API for extracting relevant things out of the > > request record. > > No need to start poking into that record on your own. > > > > I think you missed the point, see my reply to Jesper :) > > That said, since you bring-up Cowboy, it is a good example of what I > mean by other constraints and plans - an author might want to fill-in > some of the fields lazily (such body in cowboy_req), in which case > per-field accessor functions are their only good choice, unless, of > course, they want to consider compound subsets (of, say, only the > eagerly calculated parts, but I digress - it isn't the important > point)... :) Anyhow, without a specific application and vision for it's > future, which API design direction to take is not something you or I can > finalize here - it depends on a bunch of things. > > The important point is that, by using an abstract type, you're not at > all limiting yourself to exposing per-field accessor functions - you can > do other things (if you wish), as long as you make a clear distinction > between internals and API. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Mar 5 11:48:20 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 05 Mar 2016 10:48:20 +0000 Subject: [erlang-questions] cross-compile with rebar3 Message-ID: Does anyone know how to cross compile with rebar3? I wonder how we can give to it a cross-compiled erlang... Also what about C dependencies? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Sat Mar 5 12:15:06 2016 From: sdl.web@REDACTED (Leo Liu) Date: Sat, 05 Mar 2016 19:15:06 +0800 Subject: [erlang-questions] [ANN] soap library for Erlang References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: On 2016-03-04 07:28 +0000, Chandru wrote: >> Can you make something like the following work in ODBC: >> >> odbc:sql_query(Ref, "select '??'") >> >> > I'll pass it on internally to the developer working on this and get back to > you. That would solve a category of errors? for me when I am experimenting with someone's SQL that may have some CJK chars in comments. So thanks in advance. Leo Footnotes: ? [[badmatch [error einval]] ([odbc odbc_send 2 ([file "odbc.erl"] [line 833])] ...)] From tuncer.ayaz@REDACTED Sat Mar 5 12:37:21 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 5 Mar 2016 12:37:21 +0100 Subject: [erlang-questions] cross-compile with rebar3 In-Reply-To: References: Message-ID: On 5 March 2016 at 11:48, Benoit Chesneau wrote: > Does anyone know how to cross compile with rebar3? I wonder how we > can give to it a cross-compiled erlang... Also what about C > dependencies? For C, the rebar3 plugin needs to be synced with the current rebar_port_compiler. It's an easy and welcoming task for casual contributors that nobody got around to yet. While doing so, you may find other things to port forward (in the plugin) as well. See https://github.com/blt/port_compiler/issues/22 From bchesneau@REDACTED Sat Mar 5 14:00:59 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 05 Mar 2016 13:00:59 +0000 Subject: [erlang-questions] cross-compile with rebar3 In-Reply-To: References: Message-ID: Hi, I see thanks :) I am also wondering if the trick `REBAR_TARGET_ARCH=` is working or not. At least it's not documented (will check the sources). - beno?t On Sat, Mar 5, 2016 at 12:38 PM Tuncer Ayaz wrote: > On 5 March 2016 at 11:48, Benoit Chesneau wrote: > > Does anyone know how to cross compile with rebar3? I wonder how we > > can give to it a cross-compiled erlang... Also what about C > > dependencies? > > For C, the rebar3 plugin needs to be synced with the current > rebar_port_compiler. It's an easy and welcoming task for casual > contributors that nobody got around to yet. While doing so, you > may find other things to port forward (in the plugin) as well. > See https://github.com/blt/port_compiler/issues/22 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Mar 5 15:07:06 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 05 Mar 2016 14:07:06 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> <56d4a3dc.45ce190a.394c6.609a@mx.google.com> <56D5094E.4070607@khandkar.net> Message-ID: On Sat, Mar 5, 2016 at 4:47 AM Benoit Chesneau wrote: > So I finally did it the old ways using records and accessor functions over > it. To answer to joe and other, the main goal was to have more clarity and > consistency over the time in the way you will access the data. Right now > the data structure is pretty clear and can only be augmented if I have to > introduce new properties. So I guess it will be OK. > > (this thread was very interesting to learn how others uses the records, > we should have more topics like this :) which let me remember > http://www.erlangpatterns.org/ ..., I wonder how the project works as of > today) > I was literally feeling the guilt of neglect just yesterday on this! Okay, I'll take this as a sign to find some time here. I'll setup an Erlang meetup right now to help move this forward. Garrett -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Mar 5 15:22:24 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 05 Mar 2016 14:22:24 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: On Sat, Mar 5, 2016 at 4:04 AM Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > So to catch up: > > * On Richard's comments of other uses than for a string: > > The obvious use in Erlang is to produce iolist() types for binary data: > intersperse($, [<<"a">>, <<"b">>, <<"c">>]) is relatively common in my > code. The slightly less obvious use is to interleave a simplifier step in > between every optimization step of a compiler, but I've only done that once > in my life and I'm not sure it is that useful :) > Yep, iolists is the application I have for this. > * On the name choice of 'intersperse/intercalate': > > I think the semantics I'm aiming for is so close to the > OCaml(Core)/Haskell implementations I'm willing to reuse that name. Another > valid name is 'join' which is common in many other languages. I don't think > one is better than the other, but the FP roots in me says we should adopt > FP-language naming. > In looking over the naming conventions in Erlang, one might assume that the design pattern is for diversity rather than uniformity. That's a nice way to look at it I think. Personally I'd vote to converge on some standards rather than diverge - and taking cues from Erlang, and not the many many other languages, will help there. The clear analog is string:join. It's also short - saves valuable typing cycles! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sat Mar 5 15:31:22 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 5 Mar 2016 15:31:22 +0100 Subject: [erlang-questions] cross-compile with rebar3 In-Reply-To: References: Message-ID: On 5 March 2016 at 14:00, Benoit Chesneau wrote: > Hi, > > I see thanks :) I am also wondering if the trick `REBAR_TARGET_ARCH=` is > working or not. At least it's not documented (will check the sources). That help string (in the ticket) is from the built-in documentation: $ rebar help co From wallentin.dahlberg@REDACTED Sat Mar 5 15:38:23 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sat, 5 Mar 2016 15:38:23 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: I think those functions are nice things to add to the lists module. No reason to hide them in some weird extra lib module. The names though. Naming isn't easy. I actually had to look up the word "intercalate" and now I know it roughly means "insert". Perhaps it really best describes what the functions does but .. can't we find anything shorter? 2016-03-05 15:22 GMT+01:00 Garrett Smith : > On Sat, Mar 5, 2016 at 4:04 AM Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> So to catch up: >> >> * On Richard's comments of other uses than for a string: >> >> The obvious use in Erlang is to produce iolist() types for binary data: >> intersperse($, [<<"a">>, <<"b">>, <<"c">>]) is relatively common in my >> code. The slightly less obvious use is to interleave a simplifier step in >> between every optimization step of a compiler, but I've only done that once >> in my life and I'm not sure it is that useful :) >> > > Yep, iolists is the application I have for this. > > >> * On the name choice of 'intersperse/intercalate': >> >> I think the semantics I'm aiming for is so close to the >> OCaml(Core)/Haskell implementations I'm willing to reuse that name. Another >> valid name is 'join' which is common in many other languages. I don't think >> one is better than the other, but the FP roots in me says we should adopt >> FP-language naming. >> > > In looking over the naming conventions in Erlang, one might assume that > the design pattern is for diversity rather than uniformity. That's a nice > way to look at it I think. > > Personally I'd vote to converge on some standards rather than diverge - > and taking cues from Erlang, and not the many many other languages, will > help there. The clear analog is string:join. It's also short - saves > valuable typing cycles! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Sat Mar 5 16:02:29 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 5 Mar 2016 15:02:29 +0000 Subject: [erlang-questions] [ANN] soap library for Erlang In-Reply-To: References: <365F7E649CC1CE498E3327E269E19C2CB010DE@hls2exmb02p.uk365office.co.uk> Message-ID: On 5 March 2016 at 11:15, Leo Liu wrote: > On 2016-03-04 07:28 +0000, Chandru wrote: > >> Can you make something like the following work in ODBC: > >> > >> odbc:sql_query(Ref, "select '??'") > >> > >> > > I'll pass it on internally to the developer working on this and get back > to > > you. > > That would solve a category of errors? for me when I am experimenting > with someone's SQL that may have some CJK chars in comments. So thanks > in advance. > > Leo > > Footnotes: > ? [[badmatch [error einval]] ([odbc odbc_send 2 ([file "odbc.erl"] [line > 833])] ...)] > > Yes, we found this didn't work and the fix was easy enough so it will be supported. So thanks both for raising it. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From eajam@REDACTED Sat Mar 5 22:58:10 2016 From: eajam@REDACTED (Alex Alvarez) Date: Sat, 5 Mar 2016 16:58:10 -0500 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: Sorry for my late response to this topic, but I do believe Erlang is actually a great language for the ML and statistics space. Take a basic feed-forward NN with back-propagation, for example. What you'd normally have in terms of mathematical computation is mainly addition and multiplication. You only need to put together a perceptron, which are only inputs (including a bias) multiplied by respective weights, you add them up and pass this value through a function like sigmoid or hyperbolic tangent and that's that. Back-propagation, as a way to adjust the weights during the training phase, doesn't require math-wise anything more complicated. You combine the perceptrons for the hidden and output layers and you got yourself a NN. In this configuration, deep learning will simply be two or more hidden layers, instead of one. The key thing to maximize the use of Erlang is certainly to distribute the load through processes, so each perception could be one individual process, for example. Definitely, not rocket science. Now, I concur that in some situations it might be advantageous to write a module, say for a perception, in C and make use of it within Erlang, but there is no reason why you couldn't start with Erlang and gradually move to that direction, if need be. Cheers, Alex On 02/10/2016 06:31 AM, Jesper Louis Andersen wrote: > > On Wed, Feb 10, 2016 at 10:34 AM, Samuel > wrote: > > I am not aware of existing ML or linear algrebra libraries in erlang > that can be used to quick start an ML project, but just piping into > tensorflow (or any other existing library/framework) isn't really > doing ML with erlang, is it? You can as well just use tensorflow > directly. > > > The question is if this is a practical problem which needs solving or > it is for research. If you are researching how to construct, say, SVM > or NNs, then surely Erlang is a good vehicle. But in practice, there > is a number of things which makes Erlang unsuitable: > > * ML is often CPU bound. You don't want a bytecode interpreter to be a > limiting factor here. Even if the interpreter in Erlang is > state-of-the-art and highly optimized, it is not far fetched that a > FP-intensive program will be roughly a factor of 30 faster if compiled > in a lower level language. > > * GPUs are popular in ML models for a reason: they speed up the FP > computations by a factor of 3000 or more. This in itself should hint > you that you need something else than Erlang. > > * Erlangs word overhead per process and terms means a lower-level > model can pack many more entities in memory. This affects caching > behavior. > > Training of the model is often off-line and using the model is online > in the system. How you train your model is less important. This is why > I'd just outsource this problem to the libraries built and tuned for > it. It is like solving LinAlg problems but forgetting everything about > existing LAPACK and ATLAS routines in Fortran. > > A model, in Erlang, which could be viable is to use Erlang to produce > programs for lower level consumption by compilation. But these are > problems for which languages such as Haskell and OCaml dominates for a > reason: their type systems makes it far easier to pull off. > > > > -- > J. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Mar 6 00:12:31 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sat, 5 Mar 2016 18:12:31 -0500 (EST) Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: <1457219551.32671065@apps.rackspace.com> Hello, I can't claim anything but the shallowest machine learning chops beyond cursory exploration of the literature and what my data scientist son has taught me. But it seems like the first and foremost consideration in determining whether or not to proceed with a machine learning project in Erlang is careful consideration of the problem you're trying to solve. My sense is that, as Alex points out, Erlang may work quite well for problems in the Machine Learning 101 domain; e.g. limited number of perceptron elements. How do we define limited? Exactly why it may be worthwhile to experiment. Certainly Gene Shor has done interesting work. Does anyone know of worthy follow-up? That said, big-data machine learning isn't the only game in town. This piece throws down a significant challenge: Neural modelling: Abstractions of the mind http://www.nature.com/nature/journal/v531/n7592_supp/full/531S16a.html All the best, LRP -----Original Message----- From: "Alex Alvarez" Sent: Saturday, March 5, 2016 4:58pm To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Machine Learning _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Sorry for my late response to this topic, but I do believe Erlang is actually a great language for the ML and statistics space. Take a basic feed-forward NN with back-propagation, for example. What you'd normally have in terms of mathematical computation is mainly addition and multiplication. You only need to put together a perceptron, which are only inputs (including a bias) multiplied by respective weights, you add them up and pass this value through a function like sigmoid or hyperbolic tangent and that's that. Back-propagation, as a way to adjust the weights during the training phase, doesn't require math-wise anything more complicated. You combine the perceptrons for the hidden and output layers and you got yourself a NN. In this configuration, deep learning will simply be two or more hidden layers, instead of one. The key thing to maximize the use of Erlang is certainly to distribute the load through processes, so each perception could be one individual process, for example. Definitely, not rocket science. Now, I concur that in some situations it might be advantageous to write a module, say for a perception, in C and make use of it within Erlang, but there is no reason why you couldn't start with Erlang and gradually move to that direction, if need be. Cheers, Alex On 02/10/2016 06:31 AM, Jesper Louis Andersen wrote: > > On Wed, Feb 10, 2016 at 10:34 AM, Samuel > wrote: > > I am not aware of existing ML or linear algrebra libraries in erlang > that can be used to quick start an ML project, but just piping into > tensorflow (or any other existing library/framework) isn't really > doing ML with erlang, is it? You can as well just use tensorflow > directly. > > > The question is if this is a practical problem which needs solving or > it is for research. If you are researching how to construct, say, SVM > or NNs, then surely Erlang is a good vehicle. But in practice, there > is a number of things which makes Erlang unsuitable: > > * ML is often CPU bound. You don't want a bytecode interpreter to be a > limiting factor here. Even if the interpreter in Erlang is > state-of-the-art and highly optimized, it is not far fetched that a > FP-intensive program will be roughly a factor of 30 faster if compiled > in a lower level language. > > * GPUs are popular in ML models for a reason: they speed up the FP > computations by a factor of 3000 or more. This in itself should hint > you that you need something else than Erlang. > > * Erlangs word overhead per process and terms means a lower-level > model can pack many more entities in memory. This affects caching > behavior. > > Training of the model is often off-line and using the model is online > in the system. How you train your model is less important. This is why > I'd just outsource this problem to the libraries built and tuned for > it. It is like solving LinAlg problems but forgetting everything about > existing LAPACK and ATLAS routines in Fortran. > > A model, in Erlang, which could be viable is to use Erlang to produce > programs for lower level consumption by compilation. But these are > problems for which languages such as Haskell and OCaml dominates for a > reason: their type systems makes it far easier to pull off. > > > > -- > J. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rvirding@REDACTED Sun Mar 6 04:43:41 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 6 Mar 2016 04:43:41 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CFF6D2.20305@ericsson.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <56CFF6D2.20305@ericsson.com> Message-ID: I have missed this discussion. Some comments though: - LFE now supports defining multiple modules in one file so you lose the coupling module name <-> file name. It is sometimes practical but I am not sure I like. - I wouldn't like a hierarchical module namespace which couples the module name to the directory structure. - I don't think I would like a hierarchical module namespace at all anyway. I agree with Joe that a name, in this case a module name, should only refer to one thing and each thing should only have one name. So having aliases as in Elixir may mean less characters to type but make things more difficult to understand. - I honestly don't see the problem with having to declare the name of the module, even if you only have one module per file. Robert On 26 February 2016 at 07:55, Bengt Kleberg wrote: > Greetings, > > It would be sufficient (for me) if Erlang provided dictionaries (name > spaces) with words (identifiers). > The modules/atoms/pids/? in one dictionary would have to be allowed to > look the same as in another, without being the same. And I must be able to > access them all from my code. > > This would be more useful (for me) than having parts of the module name > taken from the file system. > > > bengt > > > On 02/25/2016 10:31 AM, Joe Armstrong wrote: > >> On Wed, Feb 24, 2016 at 10:46 AM, Konstantin Vsochanov >> wrote: >> >>> On 2/23/16 01:30 , Richard A. O'Keefe wrote: >>> >>>> I do not recall ever seeing -module coming up here before. >>>> I certainly don't recall a long thread like this one. >>>> >>>> Maybe because people who stumble over -module() too shy to write to >>> erlang mail list? >>> >>> #!/bin/sh >>>> #Usage: edit module >>>> case "$1" in >>>> */*) echo "Module name may not contain /" >/dev/stderr; exit 1 ;; >>>> -*) echo "Module name may not start with -">/dev/stderr; exit 1 ;; >>>> *\'*) echo "Module name may not contain '" >/dev/stderr; exit 1 ;; >>>> *\\*) echo "Module name may not contain \\" >/dev/stderr; exit 1 ;; >>>> *) ;; >>>> esac >>>> file="$1.erl" # Beware: file system may mangle name! >>>> if [ ! -e "$file" ]; then >>>> date=`date +%Y-%m-%d` >>>> case `expr "$1" : '^[a-z][a-zA-Z0-9_]*$'` in >>>> 0) module="'$1'" ;; >>>> *) module="$1" ;; >>>> esac >>>> echo "%%% File : $file" >"$file" >>>> echo "%%% Author : $LOGNAME" >>"$file" >>>> echo "%%% Created: $date" >>"$file" >>>> echo "%%% Purpose: " >>"$file" >>>> echo "" >>"$file" >>>> echo "-module($module)." >>"$file" >>>> echo "" >>"$file" >>>> echo "-export([" >>"$file" >>>> echo " ])." >>"$file" >>>> echo "" >>"$file" >>>> fi >>>> exec ${EDITOR:-emacs} "$file" >>>> >>>> So here's how much burden -module adds to this programmer: >>>> instead of >>>> % emacs foobar.erl >>>> I have to type >>>> % edit foobar >>>> >>>> Hang on, that's LESS work. >>>> >>>> Thank you for the script, I think it will work for me. I personally >>> don't use emacs, but it doesn't matter. >>> >>> Reading all this long thread I must agree, the -module() is not the >>> worst problem in Erlang. >>> >>> Regarding modules, I think the flat space for module names is more >>> troubling. If you look at almost any Erlang library/application, you can >>> see, that file names are like >>> >>> myapp_file1.erl >>> myapp_file2.erl >>> myapp_subsystem1_file1.erl >>> ... >>> >>> So we manually implement hierarchic module system. It's a clear sign, >>> that something going wrong or is missing. >>> >> No - I fundamentally disagree - I think that all module names should be >> unique >> ie in a flat name space. >> >> Why? It's all about the meanings of words - when I use a word it's >> meaning should >> not depend upon context - it would be nice if this were the case. >> >> If two people use the same word it should have the same meaning and there >> should >> be one place to look up the word. >> >> We should agree on the place to lookup the word. So for example, in >> English >> I might say all the words I'm going to use are in the Chambers English >> dictionary. >> >> Dictionaries have flat structures - The Second Edition of the Oxford >> English Dictionary has 171,476 words, in alphabetical order so it's >> easy to find them. >> >> If all module names are unique then we can put all the modules in one >> directory >> so we'll know where to find them - so several very difficult problems go >> away. >> >> This solves all these problems: >> >> 1) I know the module name but I can't find it >> >> 2) I know the module name but a module with this name is in >> dozens of different directories >> >> 3) I have a new module and I don't know what directory to store it in >> >> Dictionaries are fantastic because we know where to find words - file >> systems >> are horrible because we don't know where to find things. >> >> I can imagine having different dictionaries, French, German etc - so I can >> imaging different large collections of modules - which is akin to >> having a two-level >> namespace. With rather few top-level names (like dictionaries) >> >> If you want to simulate stucture then by all means add underscores to the >> names >> this_sub_module is fine '_' has no semantics and is purely a >> linguistic convention (just like this.sub.module ) >> >> /Joe >> >> >> >> >> >> >> >> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Mar 6 05:03:32 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 6 Mar 2016 05:03:32 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D18081.7030307@ninenines.eu> References: <56D18081.7030307@ninenines.eu> Message-ID: One definite benefit in having to explicitly write every callback even if it is doing "default" stuff is that everything *IS* then explicit and you can directly SEE what happens. And come on, writing a 2 line default case for a few callbacks is really not that much to extra code. A major problem with having implicit "default" cases is that what people consider to be a reasonable default handling varies depending on what just that server is doing. There is no default that suits every server so I will probably have to write it out and anyway. Hiding callbacks behind defaults also means that you don't SEE everything that is there. This can be a major problem for newcomers. Robert On 27 February 2016 at 11:54, Lo?c Hoguin wrote: > On 02/26/2016 10:27 PM, Onorio Catenacci wrote: > >> While there may be other considerations you want to keep in mind, >> there's one, to my mind, major point that's been neglected in the >> discussion of using Elixir: macros. >> >> I have been led to understand by people far more familiar with Erlang >> than I am that in order to use certain behaviours from OTP one has to >> write boilerplate code for several callbacks whether one wants to modify >> the behavior from defaults or not. >> >> Elixir's macros allow the library to provide default implementations for >> these callback functions. Of course a developer can override the >> defaults if needed but not having to code what I don't need strikes me >> as a big advantage--especially if we're discussing a new developer who >> doesn't already know either Erlang or Elixir. >> > > I'm not even sure what makes you think you need macros to provide default > implementations? You don't. The gen_server for example has an optional > callback with a default implementation: format_error. That it doesn't > provide defaults for other callbacks is purely a design choice, and not a > limitation of the language. You could trivially extend gen_server to make > all callbacks optional, no need to have macros for that... > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Mar 6 05:13:20 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 6 Mar 2016 05:13:20 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: I think if you seriously want to push macros then LFE, or any other lisp for that matter, is a much better option. It is much more straight-forward and readable. It also gives you the option of creating new syntax which Elixir macros don't. Elixir has a slightly "lax" syntax which allows you to hide that everything is a function call but you can't create new syntax. Of course some people lisp doesn't actually have any syntax but I prefer to view it as being very simple and very consistent. :-) Robert On 26 February 2016 at 22:27, Onorio Catenacci wrote: > >> Message: 16 >> Date: Fri, 26 Feb 2016 18:28:05 +0000 >> From: Andrew Berman >> To: Erlang Questions >> Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? >> Message-ID: >> < >> CAEVpa76ftrrjTP56FPULTkDuNNqGDiXtZ-iPTEMtEKStRWjOQw@REDACTED> >> Content-Type: text/plain; charset="utf-8" >> >> Hey Fellow Erlangers, >> >> I was curious if any of you guys have switched or are contemplating using >> Elixir for your next project. I've been programming Erlang for a while >> now, >> but I'm about to start a new project and wanted to see where other Erlang >> devs stood on it. >> >> Thanks, >> >> Andrew >> > > Hi Andrew, > > While there may be other considerations you want to keep in mind, there's > one, to my mind, major point that's been neglected in the discussion of > using Elixir: macros. > > I have been led to understand by people far more familiar with Erlang than > I am that in order to use certain behaviours from OTP one has to write > boilerplate code for several callbacks whether one wants to modify the > behavior from defaults or not. > > Elixir's macros allow the library to provide default implementations for > these callback functions. Of course a developer can override the defaults > if needed but not having to code what I don't need strikes me as a big > advantage--especially if we're discussing a new developer who doesn't > already know either Erlang or Elixir. > > I say this with full awareness that Elixir's macros are a good thing and a > bad thing too because it seems that every noob Elixir developer on our > mailing list wants to use macros everywhere. That and every noob seems to > be convinced that the pipe forward operator (|>) just has to be used > _everywhere_. > > I can certainly understand the thinking of folks who are familiar with > Erlang. I feel the same way when people ask me why I never learned Java (I > do C# for the day job). Why would I bother? But for new developers I do > believe the fact that they can use OTP without having to worry about > creating a lot of boilerplate code is a significant advantage. > > By the way, I've never written a line of Ruby in my life and I still need > to get round to looking at Phoenix. Not everyone looking at Elixir is a > Ruby developer looking for a more scalable Ruby on Rails--although in > fairness there are a lot of them. > > FWIW. > > -- > Onorio > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Sun Mar 6 09:05:30 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sun, 6 Mar 2016 16:05:30 +0800 Subject: [erlang-questions] epmd logging with ubuntu rsyslogd Message-ID: Hi, I'm using erlang on Ubuntu 14.04.4 LTS (Trusty Tahr), which has rsyslog instead of syslog. I created "/etc/rsyslog.d/70-epmd.conf" with the following content: !epmd *.*/var/log/epmd.log restarted rsylogd, but epmd logging didn't work, and /var/log/epmd.log wasn't even created. How to get epmd logging working with Ubuntu rsyslogd? Thanks Khitai From khitai.pang@REDACTED Sun Mar 6 10:28:25 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sun, 6 Mar 2016 17:28:25 +0800 Subject: [erlang-questions] net_kernel:start/1 and net_kernel:connect_node/1 errors Message-ID: Hi, I'm having some problems with distributed erlang on Linux. I have two ubuntu hosts, and on each node the ERL_EPMD_PORT environment variable is set to 5779. Hostnames are properly set in /etc/hosts on both hosts. Erlang is started on the two hosts by $ erl -sname er1@REDACTED (er1@REDACTED)1> node(). er1@REDACTED $ erl -sname er2@REDACTED (er2@REDACTED)1> node(). er2@REDACTED and epmd is running on both hosts listening on 5779. Question 1: Why does connecting to er2@REDACTED using net_kernel:connect_node/1 fail? (er1@REDACTED)4> net_kernel:connect_node(er2@REDACTED). false It runs for a couple seconds and returns false. Is it because epmd is not listening on the default port? Question 2: If epmd is not running, when I run net_kernel:start([er1@REDACTED, shortnames]), it returns error. Why? If it's caused by epmd not running, how to make sure that epmd is started when you start an OTP release? 1> net_kernel:start([er1@REDACTED, shortnames]). {error, {{shutdown, {failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}, {child,undefined,net_sup_dynamic, {erl_distribution,start_link,[[er1@REDACTED,shortnames]]}, permanent,1000,supervisor, [erl_distribution]}}} Question 3: After starting erl with -sname, if you run netstat, you'll see a process listening on a random port, what is it? tcp 0 0 0.0.0.0:23333 0.0.0.0:* LISTEN 12650/beam.smp Question 4: If an OTP release is meant to run on multiple nodes, how to properly set node name and start distribution on each node automatically? I want to start running the release on all nodes by running one command on one of the nodes. Currently I'm planning to do it this way: 1) hardcode all participating hostnames in sys.conf, something like [{"er1", "host1"}, {"er2", "host2"}] 2) on each host, when the erlang application starts, it reads the config from sys.conf, finds its own name and use net_kernel:start/1 to set the node name and start distribution What is the best practice to do it? Thanks Khitai From roberto@REDACTED Sun Mar 6 12:33:03 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 6 Mar 2016 12:33:03 +0100 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch Message-ID: Dear All, Syn 1.0.0 has been released. This is mainly a version bump from 0.10.1, since it has successfully been running in production for months now. For those of you who don't know what Syn is: Syn (short for synonym) is a global process registry that has the following features: - Global (i.e. a process is uniquely identified with a Key across all the nodes of a cluster). - Any term can be used as Key. - Fast writes. - Automatically handles conflict resolution (such as net splits). - Configurable callbacks. - Processes are automatically monitored and removed from the registry if they die. You can find Syn here: https://github.com/ostinelli/syn Also, please note that Process Groups (which also enable PubSub mechanisms) have been implemented on a dedicated branch: https://github.com/ostinelli/syn/tree/pg AFAIK this branch is currently being tested in N2O [1], and it's there to collect feedback from anyone interested before it gets merged in. Thank you, r. [1] https://github.com/synrc/n2o -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Mar 6 12:23:33 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 06 Mar 2016 12:23:33 +0100 Subject: [erlang-questions] net_kernel:start/1 and net_kernel:connect_node/1 errors In-Reply-To: References: Message-ID: <9108a3loth14a5gq45avu38f.1457263413758@email.android.com> Hi, Did you check that both have same cookie ? "Envoy? depuis mon mobile " Eric ---- Khitai Pang a ?crit ---- >Hi, > >I'm having some problems with distributed erlang on Linux. > >I have two ubuntu hosts, and on each node the ERL_EPMD_PORT environment >variable is set to 5779. > >Hostnames are properly set in /etc/hosts on both hosts. Erlang is >started on the two hosts by > >$ erl -sname er1@REDACTED > >(er1@REDACTED)1> node(). >er1@REDACTED > >$ erl -sname er2@REDACTED > >(er2@REDACTED)1> node(). >er2@REDACTED > >and epmd is running on both hosts listening on 5779. > > >Question 1: Why does connecting to er2@REDACTED using >net_kernel:connect_node/1 fail? > >(er1@REDACTED)4> net_kernel:connect_node(er2@REDACTED). >false > >It runs for a couple seconds and returns false. > >Is it because epmd is not listening on the default port? > > >Question 2: If epmd is not running, when I run >net_kernel:start([er1@REDACTED, shortnames]), it returns error. Why? If >it's caused by epmd not running, how to make sure that epmd is started >when you start an OTP release? > >1> net_kernel:start([er1@REDACTED, shortnames]). >{error, > {{shutdown, >{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}, > {child,undefined,net_sup_dynamic, > {erl_distribution,start_link,[[er1@REDACTED,shortnames]]}, > permanent,1000,supervisor, > [erl_distribution]}}} > > >Question 3: After starting erl with -sname, if you run netstat, you'll >see a process listening on a random port, what is it? > >tcp 0 0 0.0.0.0:23333 0.0.0.0:* LISTEN >12650/beam.smp > > >Question 4: If an OTP release is meant to run on multiple nodes, how to >properly set node name and start distribution on each node >automatically? I want to start running the release on all nodes by >running one command on one of the nodes. > >Currently I'm planning to do it this way: > >1) hardcode all participating hostnames in sys.conf, something like >[{"er1", "host1"}, {"er2", "host2"}] >2) on each host, when the erlang application starts, it reads the config >from sys.conf, finds its own name and use net_kernel:start/1 to set the >node name and start distribution > >What is the best practice to do it? > > >Thanks >Khitai >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Mar 6 12:54:16 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sun, 6 Mar 2016 12:54:16 +0100 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch In-Reply-To: References: Message-ID: <56DC1A68.5020607@ninenines.eu> Cheers! What would you say would be the advantages of Syn over Gproc? And for process groups over cpg? On 03/06/2016 12:33 PM, Roberto Ostinelli wrote: > Dear All, > Syn 1.0.0 has been released. This is mainly a version bump from 0.10.1, > since it has successfully been running in production for months now. > > For those of you who don't know what Syn is: Syn (short for synonym) is > a global process registry that has the following features: > > * Global (i.e. a process is uniquely identified with a Key across all > the nodes of a cluster). > * Any term can be used as Key. > * Fast writes. > * Automatically handles conflict resolution (such as net splits). > * Configurable callbacks. > * Processes are automatically monitored and removed from the registry > if they die. > > You can find Syn here: > https://github.com/ostinelli/syn > > Also, please note that Process Groups (which also enable PubSub > mechanisms) have been implemented on a dedicated branch: > https://github.com/ostinelli/syn/tree/pg > > AFAIK this branch is currently being tested in N2O [1], and it's there > to collect feedback from anyone interested before it gets merged in. > > Thank you, > r. > > [1] https://github.com/synrc/n2o > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roberto@REDACTED Sun Mar 6 13:24:04 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 6 Mar 2016 13:24:04 +0100 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch In-Reply-To: <56DC1A68.5020607@ninenines.eu> References: <56DC1A68.5020607@ninenines.eu> Message-ID: On Sun, Mar 6, 2016 at 12:54 PM, Lo?c Hoguin wrote: > Cheers! > > What would you say would be the advantages of Syn over Gproc? > > And for process groups over cpg? Hello Lo?c! I wrote an article with comparisons between global, pg2, gproc and cpg which led me to write Syn here: http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ TL;DR Gproc is great! However, to my understanding it's main purpose is not to be distributed. I found instabilities when dealing with Gproc in a distributed environment, possibly because the distributed part is based on gen_leader. The README in Gproc?s Github page clearly depicts it as being an ?Extended process dictionary?; it just felt that the distributed part hasn?t been the primary focus in the development of this library. Ulf has said he wanted to use locks_leader, but the branch is still frozen to some years ago: http://erlang.org/pipermail/erlang-questions/2015-February/083176.html I fear this will take us down the rabbit hole, but here it goes for CPG: I've seen it first hand and I've had reports [1] that it might reach bottlenecks when using a single scope. The author acknowledges and recommends using multiple scopes, however even if I understand them, I don?t care about using them (I feel I shouldn't be dealing with extra complications to circumvent lib limitations). Hope this clears up :) Best, r. [1] For instance: http://erlang.org/pipermail/erlang-questions/2015-February/083196.html https://gist.github.com/AeroNotix/eaad66f121aaa199ca6e -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sun Mar 6 17:13:23 2016 From: g@REDACTED (Garrett Smith) Date: Sun, 06 Mar 2016 16:13:23 +0000 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch In-Reply-To: References: <56DC1A68.5020607@ninenines.eu> Message-ID: On Sun, Mar 6, 2016 at 6:24 AM Roberto Ostinelli wrote: > On Sun, Mar 6, 2016 at 12:54 PM, Lo?c Hoguin wrote: > >> Cheers! >> >> What would you say would be the advantages of Syn over Gproc? >> >> And for process groups over cpg? > > > Hello Lo?c! > I wrote an article with comparisons between global, pg2, gproc and cpg > which led me to write Syn here: > > http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ > Well done Robert! This is a great read for anyone who's not already convinced that "which library should I use" is a function of a) some basic understanding of your problem and b) experimentation with measurement. Garrett -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sun Mar 6 17:32:02 2016 From: g@REDACTED (Garrett Smith) Date: Sun, 06 Mar 2016 16:32:02 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> Message-ID: On Sat, Mar 5, 2016 at 10:04 PM Robert Virding wrote: > One definite benefit in having to explicitly write every callback even if > it is doing "default" stuff is that everything *IS* then explicit and you > can directly SEE what happens. And come on, writing a 2 line default case > for a few callbacks is really not that much to extra code. > > A major problem with having implicit "default" cases is that what people > consider to be a reasonable default handling varies depending on what just > that server is doing. There is no default that suits every server so I will > probably have to write it out and anyway. > > Hiding callbacks behind defaults also means that you don't SEE everything > that is there. This can be a major problem for newcomers. > A counter point to this is that by deferring increasingly complex cases by providing sensible defaults, a user can more quickly and correctly understand an API. A useful abstraction has a story line - and I think the simpler the better. Requiring the user to accommodate increasingly edge cases undermines the main story line. E.g. I like the pattern in Erlang functions that follows the getopt Tao of required params + options. Let me get to the gist of an operation with the fewest possible inputs and I can incrementally grow into the more complex cases. I think this is less about typing and more about clarity of intent. The challenge to getting this right is drawing that line between the essential interface and the incremental enhancements. Libraries fail when they outright hide important details and leave the user mystified with seemingly magical behavior. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Sun Mar 6 18:44:58 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 6 Mar 2016 18:44:58 +0100 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch In-Reply-To: References: <56DC1A68.5020607@ninenines.eu> Message-ID: <9FF2DC34-DC29-4AB6-B609-B8D5BA98E906@widetag.com> > On 06 mar 2016, at 17:13, Garrett Smith wrote: >> http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ > > Well done Robert! This is a great read for anyone who's not already convinced that "which library should I use" is a function of a) some basic understanding of your problem and b) experimentation with measurement. > > Garrett Thank you Garrett. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Mar 6 19:32:35 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 06 Mar 2016 10:32:35 -0800 Subject: [erlang-questions] [ANN] Syn 1.0.0 & PubSub branch In-Reply-To: References: <56DC1A68.5020607@ninenines.eu> Message-ID: <56DC77C3.2000304@gmail.com> On 03/06/2016 04:24 AM, Roberto Ostinelli wrote: > On Sun, Mar 6, 2016 at 12:54 PM, Lo?c Hoguin > wrote: > > Cheers! > > What would you say would be the advantages of Syn over Gproc? > > And for process groups over cpg? > > > Hello Lo?c! > I wrote an article with comparisons between global, pg2, gproc and cpg which led me to write Syn here: > http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ > > TL;DR > > Gproc is great! However, to my understanding it's main purpose is not to be distributed. I found instabilities when dealing with Gproc in a distributed environment, possibly because the distributed part is based on gen_leader. The README in Gproc?s Github page clearly depicts it as being an ?Extended process dictionary?; it just felt that the distributed part hasn?t been the primary focus in the development of this library. > > Ulf has said he wanted to use locks_leader, but the branch is still frozen to some years ago: > http://erlang.org/pipermail/erlang-questions/2015-February/083176.html > > I fear this will take us down the rabbit hole, but here it goes for CPG: I've seen it first hand and I've had reports [1] that it might reach bottlenecks when using a single scope. The author acknowledges and recommends using multiple scopes, however even if I understand them, I don?t care about using them (I feel I shouldn't be dealing with extra complications to circumvent lib limitations). The other thing you can do with CPG is use cached process lookup data, which avoids contention for the single scope process, with the cpg_data module. That is equivalent to a lazy destination refresh method (in CloudI) which can provide more scalability https://github.com/CloudI/CloudI/blob/develop/src/tests/request_rate/results/results_v1_5_1/results.txt#L24-L43 . > > Hope this clears up :) > > Best, > r. > > [1] For instance: > http://erlang.org/pipermail/erlang-questions/2015-February/083196.html > https://gist.github.com/AeroNotix/eaad66f121aaa199ca6e > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Sun Mar 6 20:06:14 2016 From: zkessin@REDACTED (Zachary Kessin) Date: Sun, 6 Mar 2016 21:06:14 +0200 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> Message-ID: I started a new project a while ago, and I am using Erlang + Webmachine as it is what I am familiar with. Elixir has some cool stuff going on, but as I was already using 1 language that I was not familiar with (Elm) I didn't want to add to the technical risk by adding a 2nd language when the benefits of Elixir over Erlang for me were not that huge. Of course it is possible that in the future some Elixir may be added to the project, but who knows. I for one would love to see an ML-family language on the beam! ? On Sun, Mar 6, 2016 at 6:32 PM, Garrett Smith wrote: > On Sat, Mar 5, 2016 at 10:04 PM Robert Virding wrote: > >> One definite benefit in having to explicitly write every callback even if >> it is doing "default" stuff is that everything *IS* then explicit and you >> can directly SEE what happens. And come on, writing a 2 line default case >> for a few callbacks is really not that much to extra code. >> >> A major problem with having implicit "default" cases is that what people >> consider to be a reasonable default handling varies depending on what just >> that server is doing. There is no default that suits every server so I will >> probably have to write it out and anyway. >> >> Hiding callbacks behind defaults also means that you don't SEE everything >> that is there. This can be a major problem for newcomers. >> > > A counter point to this is that by deferring increasingly complex cases by > providing sensible defaults, a user can more quickly and correctly > understand an API. A useful abstraction has a story line - and I think the > simpler the better. Requiring the user to accommodate increasingly edge > cases undermines the main story line. > > E.g. I like the pattern in Erlang functions that follows the getopt Tao of > required params + options. Let me get to the gist of an operation with the > fewest possible inputs and I can incrementally grow into the more complex > cases. > > I think this is less about typing and more about clarity of intent. > > The challenge to getting this right is drawing that line between the > essential interface and the incremental enhancements. Libraries fail when > they outright hide important details and leave the user mystified with > seemingly magical behavior. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Zach Kessin Your CRM Link Twitter: @zkessin Skype: zachkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From eajam@REDACTED Sun Mar 6 23:10:30 2016 From: eajam@REDACTED (Alex Alvarez) Date: Sun, 6 Mar 2016 17:10:30 -0500 Subject: [erlang-questions] Machine Learning In-Reply-To: <1457219551.32671065@apps.rackspace.com> References: <1457219551.32671065@apps.rackspace.com> Message-ID: I'd say the opposite. The more complicated the algorithm is, the more you might be able to get out of Erlang because it's a higher level language and it was explicitly built to run large number of processes. Again, for this and any other problem, the key thing is to visualize how to divide the problem into one or more types of units/functions that can be repeated in order to truly make the most out of the platform. My biggest question regarding Erlang in general is if it's currently making use of GPUs, say through OpenCL or other mechanisms? Sorry if this has already been covered some place else! Thanks, Alex On 03/05/2016 06:12 PM, lloyd@REDACTED wrote: > Hello, > > I can't claim anything but the shallowest machine learning chops beyond cursory exploration of the literature and what my data scientist son has taught me. But it seems like the first and foremost consideration in determining whether or not to proceed with a machine learning project in Erlang is careful consideration of the problem you're trying to solve. > > My sense is that, as Alex points out, Erlang may work quite well for problems in the Machine Learning 101 domain; e.g. limited number of perceptron elements. How do we define limited? Exactly why it may be worthwhile to experiment. Certainly Gene Shor has done interesting work. Does anyone know of worthy follow-up? > > That said, big-data machine learning isn't the only game in town. This piece throws down a significant challenge: > > Neural modelling: Abstractions of the mind > http://www.nature.com/nature/journal/v531/n7592_supp/full/531S16a.html > > All the best, > > LRP > > -----Original Message----- > From: "Alex Alvarez" > Sent: Saturday, March 5, 2016 4:58pm > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Machine Learning > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Sorry for my late response to this topic, but I do believe Erlang is > actually a great language for the ML and statistics space. Take a basic > feed-forward NN with back-propagation, for example. What you'd normally > have in terms of mathematical computation is mainly addition and > multiplication. You only need to put together a perceptron, which are > only inputs (including a bias) multiplied by respective weights, you add > them up and pass this value through a function like sigmoid or > hyperbolic tangent and that's that. Back-propagation, as a way to adjust > the weights during the training phase, doesn't require math-wise > anything more complicated. You combine the perceptrons for the hidden > and output layers and you got yourself a NN. In this configuration, > deep learning will simply be two or more hidden layers, instead of one. > The key thing to maximize the use of Erlang is certainly to distribute > the load through processes, so each perception could be one individual > process, for example. Definitely, not rocket science. Now, I concur > that in some situations it might be advantageous to write a module, say > for a perception, in C and make use of it within Erlang, but there is no > reason why you couldn't start with Erlang and gradually move to that > direction, if need be. > > Cheers, > Alex > > > On 02/10/2016 06:31 AM, Jesper Louis Andersen wrote: >> On Wed, Feb 10, 2016 at 10:34 AM, Samuel > > wrote: >> >> I am not aware of existing ML or linear algrebra libraries in erlang >> that can be used to quick start an ML project, but just piping into >> tensorflow (or any other existing library/framework) isn't really >> doing ML with erlang, is it? You can as well just use tensorflow >> directly. >> >> >> The question is if this is a practical problem which needs solving or >> it is for research. If you are researching how to construct, say, SVM >> or NNs, then surely Erlang is a good vehicle. But in practice, there >> is a number of things which makes Erlang unsuitable: >> >> * ML is often CPU bound. You don't want a bytecode interpreter to be a >> limiting factor here. Even if the interpreter in Erlang is >> state-of-the-art and highly optimized, it is not far fetched that a >> FP-intensive program will be roughly a factor of 30 faster if compiled >> in a lower level language. >> >> * GPUs are popular in ML models for a reason: they speed up the FP >> computations by a factor of 3000 or more. This in itself should hint >> you that you need something else than Erlang. >> >> * Erlangs word overhead per process and terms means a lower-level >> model can pack many more entities in memory. This affects caching >> behavior. >> >> Training of the model is often off-line and using the model is online >> in the system. How you train your model is less important. This is why >> I'd just outsource this problem to the libraries built and tuned for >> it. It is like solving LinAlg problems but forgetting everything about >> existing LAPACK and ATLAS routines in Fortran. >> >> A model, in Erlang, which could be viable is to use Erlang to produce >> programs for lower level consumption by compilation. But these are >> problems for which languages such as Haskell and OCaml dominates for a >> reason: their type systems makes it far easier to pull off. >> >> >> >> -- >> J. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Mon Mar 7 02:22:46 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 7 Mar 2016 09:22:46 +0800 Subject: [erlang-questions] net_kernel:start/1 and net_kernel:connect_node/1 errors In-Reply-To: <9108a3loth14a5gq45avu38f.1457263413758@email.android.com> References: <9108a3loth14a5gq45avu38f.1457263413758@email.android.com> Message-ID: Yes both nodes have the same cookie. Thanks Khitai On 2016/3/6 19:23, ?ric Pailleau wrote: > > Hi, > > Did you check that both have same cookie ? > > "Envoy? depuis mon mobile " Eric > > > > ---- Khitai Pang a ?crit ---- > > Hi, > > I'm having some problems with distributed erlang on Linux. > > I have two ubuntu hosts, and on each node the ERL_EPMD_PORT environment > variable is set to 5779. > > Hostnames are properly set in /etc/hosts on both hosts. Erlang is > started on the two hosts by > > $ erl -sname er1@REDACTED > > (er1@REDACTED)1> node(). > er1@REDACTED > > $ erl -sname er2@REDACTED > > (er2@REDACTED)1> node(). > er2@REDACTED > > and epmd is running on both hosts listening on 5779. > > > Question 1: Why does connecting to er2@REDACTED using > net_kernel:connect_node/1 fail? > > (er1@REDACTED)4> net_kernel:connect_node(er2@REDACTED). > false > > It runs for a couple seconds and returns false. > > Is it because epmd is not listening on the default port? > > > Question 2: If epmd is not running, when I run > net_kernel:start([er1@REDACTED, shortnames]), it returns error. Why? If > it's caused by epmd not running, how to make sure that epmd is started > when you start an OTP release? > > 1> net_kernel:start([er1@REDACTED, shortnames]). > {error, > {{shutdown, > {failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}, > {child,undefined,net_sup_dynamic, > {erl_distribution,start_link,[[er1@REDACTED,shortnames]]}, > permanent,1000,supervisor, > [erl_distribution]}}} > > > Question 3: After starting erl with -sname, if you run netstat, you'll > see a process listening on a random port, what is it? > > tcp 0 0 0.0.0.0:23333 0.0.0.0:* LISTEN > 12650/beam.smp > > > Question 4: If an OTP release is meant to run on multiple nodes, how to > properly set node name and start distribution on each node > automatically? I want to start running the release on all nodes by > running one command on one of the nodes. > > Currently I'm planning to do it this way: > > 1) hardcode all participating hostnames in sys.conf , > something like > [{"er1", "host1"}, {"er2", "host2"}] > 2) on each host, when the erlang application starts, it reads the config > from sys.conf , finds its own name and use > net_kernel:start/1 to set the > node name and start distribution > > What is the best practice to do it? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Mar 7 04:18:46 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 7 Mar 2016 16:18:46 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56D9AAE5.8080500@khandkar.net> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> <56D9AAE5.8080500@khandkar.net> Message-ID: <575a5f3e-8d71-9f72-9617-dead72e20d1c@cs.otago.ac.nz> On 5/03/16 4:33 am, Siraaj Khandkar wrote: > > The notion of sameness here is problematic, since you're now requiring > that the above-quoted "sequences of information" are sets, which is > unlikely the intended meaning. You just lost me. I was requiring that the sequences be SEQUENCES. How does anything I wrote require them to be sets? Here's what I was saying: going from [a,b,c] and [1,2,3] to [a,1,b,2,c,3] *IS* interleaving going from [a,b,c] and 0 to [a,0,b,0,c,0] is *NOT* interleaving. The question is not whether a sequence contains repeated elements but a question as to whether it *IS* a sequence in the first place, I am happy to call [a,b,c] [0,0,0] -> [a,0,b,0,c,0] interleaving. interleave([X|Xs], [Y|Ys]) -> [X,Y|interleave(Xs,Ys)]; interleave([], []) -> []. Didn't I give that very code? This DOESN'T care whether there are repeated elements in either sequence. It DOES care that it is given two lists, not a list and a single element. Again, what I'm saying is not that there is a problem with lists that *happen* to contain duplicate elements, but that not_interleave([X,Y|Zs], S) -> [X,S|not_interleave([Y|Zs], S)]; not_interleave(End, _) -> End. is not interleaving. We agree that "interleaving" does not "say anything about the contents of the two sequences", which is why I didn't say anything about the contents of the two sequences. We also agree that it "operate[s] on" "two sequences", not one sequence and one something else, which was my whole point. Sigh. From ok@REDACTED Mon Mar 7 04:50:22 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 7 Mar 2016 16:50:22 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> On 6/03/16 3:22 am, Garrett Smith wrote: > On Sat, Mar 5, 2016 at 4:04 AM Jesper Louis Andersen > > wrote: > > So to catch up: > > * On Richard's comments of other uses than for a string: > > The obvious use in Erlang is to produce iolist() types for binary > data: intersperse($, [<<"a">>, <<"b">>, <<"c">>]) is relatively > common in my code. The slightly less obvious use is to interleave > a simplifier step in between every optimization step of a > compiler, but I've only done that once in my life and I'm not sure > it is that useful :) > > > Yep, iolists is the application I have for this. This looks like a "string" use to me (in the sense of data type for representing text, not in the list of character code sense). I always worry about things like this because of the difficulty the receiver will have decoding it. Reverting to lists for a minute, suppose I do intersperse($,, ["a,b","c,,,d"]) and write the resulting iolist, or intercalate(",", ["a,b","c,,,d"]). How does the receiver decode "a,b,c,,,d" so as to recover the original input? For example, I might use an escape character. Let's say the escape character is #. I'd want to generate "a#,b,c#,#,#,d". Here is a paragraph from the documentation of my Smalltalk system. Joining is intrinsically problematic. Just pasting a bunch of things together is normally not an issue, but introducing a separator may mislead you into thinking that the output can be decoded. It is especially tempting when there is a related 'split' method. Consider $, join: #('a' 'b' 'c') ===> 'a,b,c' $, split: 'a,b,c' ===> an OrderedCollection('a' 'b' 'c') in Pharo. Doesn't it look as though these are inverses? But try $, join: #('a,b' 'c,,,d') ===> 'a,b,c,,,d' $, split: 'a,b,c,,,d' ===> an OrderedCollection('a' 'b' 'c' '' '' 'd') No, they are not inverses. True inverses would be something like $, join: #('a,b' 'c,,,d') escape: $# ===> 'a#,b,c#,#,#,d' $, split: 'a#,b,c#,#,#,d' escape: $# ===> anOrderedCollection('a,b' 'c,,,d'). But those methods do not exist in Pharo. Use splitting and joining methods with vigilant trepidation; make sure that your separator does *not* appear in the data you are joining. The use of binaries really doesn't change this: as long as what I'm going to do is to send a sequence of characters somewhere, I need a documented justification for sending something I can't decode. In your actual use, you may KNOW that the separator cannot appear in the list elements you're gluing together, so for you it may be safe. The problem with 'join' as a name is that it is hopelessly vague. Xs++Ys joins Xs and Ys. [{K,X,Y} || {K,X} <- Xs, {K,Y} <- Ys} also joins Xs and Ys. Some versions of join will let me do "a, b, and c" and some won't. And so it goes. From ok@REDACTED Mon Mar 7 05:14:17 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 7 Mar 2016 17:14:17 +1300 Subject: [erlang-questions] Machine Learning In-Reply-To: References: <1457219551.32671065@apps.rackspace.com> Message-ID: <2d892fb4-5cd9-161f-7c67-143f73aa10fc@cs.otago.ac.nz> On 7/03/16 11:10 am, Alex Alvarez wrote: > I'd say the opposite. The more complicated the algorithm is, the more > you might be able to get out of Erlang because it's a higher level > language and it was explicitly built to run large number of > processes. Again, for this and any other problem, the key thing is to > visualize how to divide the problem into one or more types of > units/functions that can be repeated in order to truly make the most > out of the platform. > > > My biggest question regarding Erlang in general is if it's currently > making use of GPUs, say through OpenCL or other mechanisms? Sorry if > this has already been covered some place else! AMD and NVIDIA are tripping over themselves to provide free libraries for their devices. I would prefer to do this kind of stuff in Haskell, largely because there's already support for access to number-crunching libraries, and more than one DSEL for working with GPUs. (The current OpenMP specification has support for 'offloading' some of your C or Fortran code to a GPU, and recent releases of GCC support that...) I really wouldn't dream of doing "big data" in Erlang. What I *might* do is to use Erlang for meta-programming. Specify the algorithm in Erlang, and map that down to OpenCL + C + routing code. If your problem is big enough for size to be an issue, it's big enough that an extra compilation stage may not be noticed. But even that I'd probably want to do in Haskell. When you get to the point where debugging is painful, you want to push *hard* on compile-time checks, and the amount of programming you can do with Haskell types is amazing. It's not a full dependently-typed language, but you *can* for example write a + b and have the compiler check that a and b are matrices with the same dimensions, via type checking. Of course, if you write your own DSL (or DSEL) in/using Erlang, you can do whatever typechecking you want. From bengt.kleberg@REDACTED Mon Mar 7 06:19:24 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 7 Mar 2016 06:19:24 +0100 Subject: [erlang-questions] net_kernel:start/1 and net_kernel:connect_node/1 errors In-Reply-To: References: Message-ID: <56DD0F5C.8060404@ericsson.com> Greetings, To start epmd , if it might not have been started automatically by "erl", I do os:cmd( "epmd" ). If you want to see what epmd knows, you can do "epmd -names" from the Unix CLI. If you want to run on multiple nodes, take a look at the slave module. Especially the bit about I/O happening on the master. bengt On 03/06/2016 10:28 AM, Khitai Pang wrote: > Hi, > > I'm having some problems with distributed erlang on Linux. > > I have two ubuntu hosts, and on each node the ERL_EPMD_PORT > environment variable is set to 5779. > > Hostnames are properly set in /etc/hosts on both hosts. Erlang is > started on the two hosts by > > $ erl -sname er1@REDACTED > > (er1@REDACTED)1> node(). > er1@REDACTED > > $ erl -sname er2@REDACTED > > (er2@REDACTED)1> node(). > er2@REDACTED > > and epmd is running on both hosts listening on 5779. > > > Question 1: Why does connecting to er2@REDACTED using > net_kernel:connect_node/1 fail? > > (er1@REDACTED)4> net_kernel:connect_node(er2@REDACTED). > false > > It runs for a couple seconds and returns false. > > Is it because epmd is not listening on the default port? > > > Question 2: If epmd is not running, when I run > net_kernel:start([er1@REDACTED, shortnames]), it returns error. Why? If > it's caused by epmd not running, how to make sure that epmd is started > when you start an OTP release? > > 1> net_kernel:start([er1@REDACTED, shortnames]). > {error, > {{shutdown, > {failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}, > {child,undefined,net_sup_dynamic, > {erl_distribution,start_link,[[er1@REDACTED,shortnames]]}, > permanent,1000,supervisor, > [erl_distribution]}}} > > > Question 3: After starting erl with -sname, if you run netstat, you'll > see a process listening on a random port, what is it? > > tcp 0 0 0.0.0.0:23333 0.0.0.0:* LISTEN > 12650/beam.smp > > > Question 4: If an OTP release is meant to run on multiple nodes, how > to properly set node name and start distribution on each node > automatically? I want to start running the release on all nodes by > running one command on one of the nodes. > > Currently I'm planning to do it this way: > > 1) hardcode all participating hostnames in sys.conf, something like > [{"er1", "host1"}, {"er2", "host2"}] > 2) on each host, when the erlang application starts, it reads the > config from sys.conf, finds its own name and use net_kernel:start/1 to > set the node name and start distribution > > What is the best practice to do it? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thoffmann@REDACTED Mon Mar 7 08:49:31 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Mon, 07 Mar 2016 08:49:31 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API Message-ID: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> Hi, I am working a HTTP API that has a version in it, like this: /api/1/... Should I create separate handler modules for each version? Or should I have code that branches out on the api version in one common handler module? If I dispatch to a version specific handler module I will not have to think about the version anymore. If I dispatch to a version generic handle module I can - potentially - reuse code that is common across api versions. I'm pretty sure there will be new api versions - this is not just a case of being a good citizen. The current version of the software is young, so I would not say it is in any way cast in stone yet. I'm inclined to do version specific handler modules because it makes the modules very clean in terms of contex. But I am not an expert on webmachine/cowboy, so I thought I would ask. Cheers, Torben -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From mikpelinux@REDACTED Mon Mar 7 10:18:22 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 7 Mar 2016 10:18:22 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> Message-ID: <22237.18270.703914.235613@gargle.gargle.HOWL> Zachary Kessin writes: > I for one would love to see an ML-family language on the beam! I'm working on a Standard ML (SML'97) compiler targeting the Erlang/OTP VM. I hope to have it finished enough to make it public sometime this year. From jesper.louis.andersen@REDACTED Mon Mar 7 12:59:34 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2016 12:59:34 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> Message-ID: On Mon, Mar 7, 2016 at 8:49 AM, Torben Hoffmann wrote: > If I dispatch to a version generic handle module I can - potentially - > reuse code that is common across api versions. > Can you map API version 1 into API version 2 in any way? If so, a way to handle the problem is simply to shim Version 1 and transform it to a Version 2 query and then run with the handler of Version 2. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Mon Mar 7 14:32:01 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Mon, 7 Mar 2016 08:32:01 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? Message-ID: On Sun, Mar 6, 2016 at 11:32 AM, Garrett Smith wrote: > On Sat, Mar 5, 2016 at 10:04 PM Robert Virding wrote: > >> One definite benefit in having to explicitly write every callback even if >> it is doing "default" stuff is that everything *IS* then explicit and you >> can directly SEE what happens. And come on, writing a 2 line default case >> for a few callbacks is really not that much to extra code. >> >> A major problem with having implicit "default" cases is that what people >> consider to be a reasonable default handling varies depending on what just >> that server is doing. There is no default that suits every server so I will >> probably have to write it out and anyway. >> >> Hiding callbacks behind defaults also means that you don't SEE everything >> that is there. This can be a major problem for newcomers. >> > > A counter point to this is that by deferring increasingly complex cases by > providing sensible defaults, a user can more quickly and correctly > understand an API. A useful abstraction has a story line - and I think the > simpler the better. Requiring the user to accommodate increasingly edge > cases undermines the main story line. > > E.g. I like the pattern in Erlang functions that follows the getopt Tao of > required params + options. Let me get to the gist of an operation with the > fewest possible inputs and I can incrementally grow into the more complex > cases. > > I think this is less about typing and more about clarity of intent. > > The challenge to getting this right is drawing that line between the > essential interface and the incremental enhancements. Libraries fail when > they outright hide important details and leave the user mystified with > seemingly magical behavior. > > Again, apologies--couldn't get away to respond till now: 1.) There's no question that explicit is better than implicit Robert. But on the other hand, if most of the explicit is going to be boilerplate code then it should be hidden by default. I don't know your experience but in mine I've met very few developers who bother to investigate compiler warnings. And I mean _default_ compiler warnings. This is because most of them say "Well it compiles so it's ok right?" Boilerplate code can encourage that tendency too--I simply copy/paste the boilerplate from another codebase and I don't worry about what it does. So, in that case I think it's ok to make an exception to the general rule that explicit is better than implicit. Explicit is better than implicit because it forces someone to actually _think_. Lots of boilerplate has the opposite effect wouldn't you agree? 2.) As I said in my original response, macros strike me as a very powerful tool but quite dangerous in the hands of a noob developer. I almost wish there were a way to disable macros from the compiler level on a setting that could be hidden from new developers. Yes they're extremely powerful (and yes I agree that LFE is even more powerful) but like so many other powerful mechanisms they're also really dangerous in the hands of the inexperienced. Or even those that learn only part of the story but don't get all of it. As Pope said "a little knowledge is a dangerous thing." 3.) I agree with Garrett. Language defaults are a lot more important than many developers seem to realize. Every since I started developing code I was always told to slap a const on immutable data and to pass arguments by value as much as it could be managed. This prevented accidental changes. Yet in all my experience of writing C and C++ I almost never saw people using #defaines for constants--let alone putting const modifiers on things or trying to figure out how to pass by value rather than pass by reference. This is part of the reason I think that FP is the future. The default in FP is immutable. The default is the good choice not the bad one. And that's also part of my point about boilerplate code versus macros. If the default on the code that needs to be supplied is relatively intelligent macros then it's one less thing for the developer to worry with--and most developers won't spend time fighting language defaults. This is part of what has always impressed me about Erlang too. The default is no shared memory. We all know that's a heck of a lot more scalable than what a lot of developers do with shared memory. An intelligent default is a great aid to productivity. -- Onorio -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Mon Mar 7 15:42:29 2016 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 7 Mar 2016 15:42:29 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <3BBAB72D-071E-41C7-B173-8127C0BB824F@gmail.com> Homoiconicity is way overrated when it comes to metaprogamming. There is nothing that Lisp does with parentheses that Elixir cannot with the 'do' token. And no Lisp is truly simple or consistent. Not LFE at least. lists:map #'list_to_integer/1 #*0 #o111 +1.0 #B(?) #(?) ; comments #.(?) There is nothing simple nor consistent about this. Regards. > Le 6 mars 2016 ? 05:13, Robert Virding a ?crit : > > I think if you seriously want to push macros then LFE, or any other lisp for that matter, is a much better option. It is much more straight-forward and readable. It also gives you the option of creating new syntax which Elixir macros don't. Elixir has a slightly "lax" syntax which allows you to hide that everything is a function call but you can't create new syntax. > > Of course some people lisp doesn't actually have any syntax but I prefer to view it as being very simple and very consistent. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Mar 7 16:08:06 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 07 Mar 2016 15:08:06 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On Sun, Mar 6, 2016 at 9:50 PM Richard A. O'Keefe wrote: > On 6/03/16 3:22 am, Garrett Smith wrote: > > > On Sat, Mar 5, 2016 at 4:04 AM Jesper Louis Andersen > > > > wrote: > > > > So to catch up: > > > > * On Richard's comments of other uses than for a string: > > > > The obvious use in Erlang is to produce iolist() types for binary > > data: intersperse($, [<<"a">>, <<"b">>, <<"c">>]) is relatively > > common in my code. The slightly less obvious use is to interleave > > a simplifier step in between every optimization step of a > > compiler, but I've only done that once in my life and I'm not sure > > it is that useful :) > > > > > > Yep, iolists is the application I have for this. > > This looks like a "string" use to me (in the sense of data type for > representing > text, not in the list of character code sense). > > I always worry about things like this because of the difficulty the > receiver > will have decoding it. Reverting to lists for a minute, suppose I do > intersperse($,, ["a,b","c,,,d"]) and write the resulting iolist, or > intercalate(",", ["a,b","c,,,d"]). > How does the receiver decode "a,b,c,,,d" so as to recover the original > input? > > For example, I might use an escape character. > Let's say the escape character is #. > I'd want to generate "a#,b,c#,#,#,d". > > Here is a paragraph from the documentation of my Smalltalk system. > > Joining is intrinsically problematic. Just pasting a bunch of things > together is normally not an issue, but introducing a separator may > mislead you into thinking that the output can be decoded. It is > especially tempting when there is a related 'split' method. Consider > $, join: #('a' 'b' 'c') ===> 'a,b,c' > $, split: 'a,b,c' ===> an OrderedCollection('a' 'b' 'c') > in Pharo. Doesn't it look as though these are inverses? But try > $, join: #('a,b' 'c,,,d') ===> 'a,b,c,,,d' > $, split: 'a,b,c,,,d' ===> an OrderedCollection('a' 'b' 'c' '' > '' 'd') > No, they are not inverses. True inverses would be something like > $, join: #('a,b' 'c,,,d') escape: $# > ===> 'a#,b,c#,#,#,d' > $, split: 'a#,b,c#,#,#,d' escape: $# > ===> anOrderedCollection('a,b' 'c,,,d'). > But those methods do not exist in Pharo. Use splitting and joining > methods with vigilant trepidation; make sure that your separator > does *not* appear in the data you are joining. > > > The use of binaries really doesn't change this: as long as what I'm going > to > do is to send a sequence of characters somewhere, I need a documented > justification for sending something I can't decode. > > In your actual use, you may KNOW that the separator cannot appear in > the list elements you're gluing together, so for you it may be safe. > I agree to use lists:join with vigilance and trepidation. It won't make it any less useful. > The problem with 'join' as a name is that it is hopelessly vague. > Xs++Ys joins Xs and Ys. [{K,X,Y} || {K,X} <- Xs, {K,Y} <- Ys} also > joins Xs and Ys. Some versions of join will let me do > "a, b, and c" and some won't. And so it goes. > It's vague, as is intercalculate, but as it's superficially doing what string "join" does with chars has some precedence within Erlang. I wouldn't call it hopeless. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Mar 7 16:27:30 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2016 16:27:30 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith wrote: > It's vague, as is intercalculate, but as it's superficially doing what > string "join" does with chars has some precedence within Erlang. I wouldn't > call it hopeless. I'm probably leaning away from using 'join' at this point, since 'join' already have type join :: Monad M => m (m a) -> m a so from an FP perspective, that name is highly confusing since it is in use in monadic context and is used to join monadic data into its own monadic context. For a list, join is essentially 'append': Prelude Control.Monad> join ["a", "b", "c"] "abc" But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. I mean, C++ also uses the word Functor, but they know jack shit about what that means mathematically. And almost every language knows jack shit about what join really is either :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Mar 7 16:53:09 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 07 Mar 2016 15:53:09 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On Mon, Mar 7, 2016 at 9:28 AM Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith wrote: > >> It's vague, as is intercalculate, but as it's superficially doing what >> string "join" does with chars has some precedence within Erlang. I wouldn't >> call it hopeless. > > > I'm probably leaning away from using 'join' at this point, since 'join' > already have type > > join :: Monad M => m (m a) -> m a > > so from an FP perspective, that name is highly confusing since it is in use > in monadic context and is used to join monadic data into its own monadic > context. For a list, join is essentially 'append': > > Prelude Control.Monad> join ["a", "b", "c"] > "abc" > > But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. > Sigh. Okay, so the future naming discussions will involve with word monad and monadic? You've seen the discussions around adoption and the disruptive influence of Elixir? For whatever reason FP pedantry is not a draw for me. When I need inspiration I look to Python. Maybe that's the wrong direction and we need to drive our community through more gates. > I mean, C++ also uses the word Functor, but they know jack shit about what > that means mathematically. And almost every language knows jack shit about > what join really is either :) > Oh, and functor. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Mon Mar 7 14:16:23 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Mon, 07 Mar 2016 14:16:23 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> Message-ID: <56dda60f.6610700a.73209.ffffcd05@mx.google.com> Jesper Louis Andersen writes: > [ text/plain ] > On Mon, Mar 7, 2016 at 8:49 AM, Torben Hoffmann wrote: > >> If I dispatch to a version generic handle module I can - potentially - >> reuse code that is common across api versions. >> > > Can you map API version 1 into API version 2 in any way? If so, a way to > handle the problem is simply to shim Version 1 and transform it to a > Version 2 query and then run with the handler of Version 2. > Well, I don't know what version 2 will look like yet :-) I will add a dispatch rule that forwards version 1 specifically to my current handler module. Then I can deal with the choice of approach for version numbers when it is time to do that new version. Future me is going to hate present me... but present me will document the considerations as comments. Cheers, Torben -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From essen@REDACTED Mon Mar 7 17:04:12 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 7 Mar 2016 17:04:12 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: <56DDA67C.90204@ninenines.eu> On 03/07/2016 04:53 PM, Garrett Smith wrote: > On Mon, Mar 7, 2016 at 9:28 AM Jesper Louis Andersen > > wrote: > > > On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith > wrote: > > It's vague, as is intercalculate, but as it's superficially > doing what string "join" does with chars has some precedence > within Erlang. I wouldn't call it hopeless. > > > I'm probably leaning away from using 'join' at this point, since > 'join' already have type > > join :: Monad M => m (m a) -> m a > > so from an FP perspective, that name is highly confusing since it is > in use in monadic context and is used to join monadic data into its > own monadic context. For a list, join is essentially 'append': > > Prelude Control.Monad> join ["a", "b", "c"] > "abc" > > But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. > > > Sigh. Okay, so the future naming discussions will involve with word > monad and monadic? > > You've seen the discussions around adoption and the disruptive influence > of Elixir? > > For whatever reason FP pedantry is not a draw for me. When I need > inspiration I look to Python. Maybe that's the wrong direction and we > need to drive our community through more gates. I am completely with you on that. If the function was called intercalculate I'd never find it and would continue writing my own. Precision is important, but it's even more important to think about who the users are (or who you want them to be) and to lose some of this precision to make things clearer for them. And as far as I can tell, people don't go to Erlang because of their mathematical background. Human languages are interesting like that, because the more you go for precision the less people understand what you mean. Specialized terms are only good for people with the same specialization. If you want the most people to get it, use terms that a fourth grader would understand. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jesper.louis.andersen@REDACTED Mon Mar 7 17:07:17 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2016 17:07:17 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: <56dda60f.6610700a.73209.ffffcd05@mx.google.com> References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> <56dda60f.6610700a.73209.ffffcd05@mx.google.com> Message-ID: On Mon, Mar 7, 2016 at 2:16 PM, Torben Hoffmann wrote: > Well, I don't know what version 2 will look like yet :-) Roy T. Fielding has some words to say about versioning APIs in the first place, which I think one should look at: http://www.infoq.com/articles/roy-fielding-on-versioning -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Mar 7 17:12:40 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 7 Mar 2016 17:12:40 +0100 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On Mon, Mar 7, 2016 at 4:53 PM, Garrett Smith wrote: > Sigh. Okay, so the future naming discussions will involve with word monad > and monadic? leaning /= decided. * I really like `join` for its context in other languages as the thing that "joins stuff with a separator". * I like intersperse as a name because it has an extremely precise specification where join is vague. One simple decision could be a majority point: the majority expect it to be join, and we are probably never going to see monadic joins in Erlang (they require typed worlds anyway for their full power). Besides, Haskell programmers are more likely to be able to adapt I think. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Mar 7 17:31:13 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 07 Mar 2016 16:31:13 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On Mon, Mar 7, 2016 at 10:13 AM Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > Besides, Haskell programmers are more likely to be able to adapt I think. > Some, perhaps. Though I could see others flocking from Erlang to Haskell because we can't keep our monads straight! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Mon Mar 7 17:59:55 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 7 Mar 2016 11:59:55 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <575a5f3e-8d71-9f72-9617-dead72e20d1c@cs.otago.ac.nz> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> <56D9AAE5.8080500@khandkar.net> <575a5f3e-8d71-9f72-9617-dead72e20d1c@cs.otago.ac.nz> Message-ID: <56DDB38B.8070309@khandkar.net> On 3/6/16 10:18 PM, Richard A. O'Keefe wrote: > > > On 5/03/16 4:33 am, Siraaj Khandkar wrote: >> >> The notion of sameness here is problematic, since you're now requiring >> that the above-quoted "sequences of information" are sets, which is >> unlikely the intended meaning. > > You just lost me. I was requiring that the sequences be SEQUENCES. > How does anything I wrote require them to be sets? > > Here's what I was saying: > going from [a,b,c] and [1,2,3] to [a,1,b,2,c,3] *IS* interleaving > going from [a,b,c] and 0 to [a,0,b,0,c,0] is *NOT* interleaving. Both of the above results are functions of 2 sequences, the difference is that 1st works with a pre-computed list, while the 2nd works with a an infinite stream of 0s. So the difference is in the sources of the sequences, not in how they're combined. This clearly calls for some distinction in API (i.e. how those sequences are passed), but not a different description of the meaning of the outcome (i.e. name). One (quick and rough) example of an API distinction could be: -spec interleave([A], {exactly, [A]} | {repeat, A}) -> [A]. Which can actually be made even more general, since repeating the same element is just a special case of cycling a single element list: -spec interleave([A], {exactly, [A]} | {cycle, [A]}) -> [A]. ... but I digress ... > > The question is not whether a sequence contains repeated elements > but a question as to whether it *IS* a sequence in the first place, > I am happy to call [a,b,c] [0,0,0] -> [a,0,b,0,c,0] interleaving. > Right, the main difference in our points of view is that I see this 0 as just an argument for a process which makes an infinite stream of 0s (i.e. another sequence), while the rule for combining the sequences remains exactly the same. Since, if you truly only have one 0, then the best you can do is : [1, 0, 2] or maybe [1, 0, 2, 3, 4, 5] to insert more 0s - you need more 0s, hence you need a sufficiently-long sequence of them; how that sufficiently-long sequence is computed is unrelated to how it is combined with the 1st sequence. What I'm naming is the rule for the combination. From vladdu55@REDACTED Mon Mar 7 20:27:37 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 Mar 2016 20:27:37 +0100 Subject: [erlang-questions] REST server Message-ID: Hi! I am an almost complete noob regarding this, so please excuse me if I am asking questions with obvious answers. I have an application that I'd like to expose via a REST api. From what I could find quickly, in the Erlang world it looks like the main options are Webmachine and Cowboy. Does anyone have any advice on which one fits better my requirements, as below? - the server must be embeddable in another application, i.e. possible to start/stop without a release - it must be possible to configure it to use a dynamic port value (multiple independent servers may run on the same machine) - the REST API is delegating to an Erlang API, so ease of implementation would be nice :-) - if it is possible to run multiple servers on the same Erlang runtime, then it would be a big plus best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Mon Mar 7 20:31:39 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Mon, 7 Mar 2016 16:31:39 -0300 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: <5A1C9FAD-E6FA-48EC-B3B0-FC16B59540B4@inakanetworks.com> As its author, it?s my duty :P to recommend you to take a look at sumo_rest on github or our blog . It might be more than what you need, but you can probably take advantage of the examples, individual libraries (like cowboy-trails and cowboy-swagger) or even the general architecture approach there. Cheers! > On Mar 7, 2016, at 16:27, Vlad Dumitrescu wrote: > > Hi! > > I am an almost complete noob regarding this, so please excuse me if I am asking questions with obvious answers. > > I have an application that I'd like to expose via a REST api. From what I could find quickly, in the Erlang world it looks like the main options are Webmachine and Cowboy. > > Does anyone have any advice on which one fits better my requirements, as below? > > - the server must be embeddable in another application, i.e. possible to start/stop without a release > - it must be possible to configure it to use a dynamic port value (multiple independent servers may run on the same machine) > - the REST API is delegating to an Erlang API, so ease of implementation would be nice :-) > - if it is possible to run multiple servers on the same Erlang runtime, then it would be a big plus > > best regards, > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Mar 7 20:44:23 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 7 Mar 2016 20:44:23 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: <56DDDA17.2060403@ninenines.eu> On 03/07/2016 08:27 PM, Vlad Dumitrescu wrote: > Hi! > > I am an almost complete noob regarding this, so please excuse me if I am > asking questions with obvious answers. > > I have an application that I'd like to expose via a REST api. From what > I could find quickly, in the Erlang world it looks like the main options > are Webmachine and Cowboy. > > Does anyone have any advice on which one fits better my requirements, as > below? > > - the server must be embeddable in another application, i.e. possible to > start/stop without a release No problem for that with Cowboy, either through a function call or provided child specs. > - it must be possible to configure it to use a dynamic port value > (multiple independent servers may run on the same machine) No problem, set {port, 0} option and then call ranch:get_port/1. > - the REST API is delegating to an Erlang API, so ease of implementation > would be nice :-) Cowboy is inspired by Webmachine so I suppose they're pretty close for this one. > - if it is possible to run multiple servers on the same Erlang runtime, > then it would be a big plus No problem, just declare more than one listener and you're good! -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From vladdu55@REDACTED Mon Mar 7 20:45:03 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 Mar 2016 20:45:03 +0100 Subject: [erlang-questions] REST server In-Reply-To: <5A1C9FAD-E6FA-48EC-B3B0-FC16B59540B4@inakanetworks.com> References: <5A1C9FAD-E6FA-48EC-B3B0-FC16B59540B4@inakanetworks.com> Message-ID: Thanks for pointing out sumo_rest. It is indeed a bit more than I need (I won't have any database backend in the conventional meaning), but looks worth checking out in more detail. regards, Vlad On Mon, Mar 7, 2016 at 8:31 PM, Brujo Benavides < fernando.benavides@REDACTED> wrote: > As its author, it?s my duty :P to recommend you to take a look at > sumo_rest on github or our blog > > . > It might be more than what you need, but you can probably take advantage > of the examples, individual libraries (like cowboy-trails and > cowboy-swagger) or even the general architecture approach there. > Cheers! > > On Mar 7, 2016, at 16:27, Vlad Dumitrescu wrote: > > Hi! > > I am an almost complete noob regarding this, so please excuse me if I am > asking questions with obvious answers. > > I have an application that I'd like to expose via a REST api. From what I > could find quickly, in the Erlang world it looks like the main options are > Webmachine and Cowboy. > > Does anyone have any advice on which one fits better my requirements, as > below? > > - the server must be embeddable in another application, i.e. possible to > start/stop without a release > - it must be possible to configure it to use a dynamic port value > (multiple independent servers may run on the same machine) > - the REST API is delegating to an Erlang API, so ease of implementation > would be nice :-) > - if it is possible to run multiple servers on the same Erlang runtime, > then it would be a big plus > > best regards, > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Mar 7 20:46:23 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 Mar 2016 20:46:23 +0100 Subject: [erlang-questions] REST server In-Reply-To: <56DDDA17.2060403@ninenines.eu> References: <56DDDA17.2060403@ninenines.eu> Message-ID: Thanks, Lo?c, I will then check out Cowboy in more detail, to see how it feels. regards, Vlad On Mon, Mar 7, 2016 at 8:44 PM, Lo?c Hoguin wrote: > On 03/07/2016 08:27 PM, Vlad Dumitrescu wrote: > >> Hi! >> >> I am an almost complete noob regarding this, so please excuse me if I am >> asking questions with obvious answers. >> >> I have an application that I'd like to expose via a REST api. From what >> I could find quickly, in the Erlang world it looks like the main options >> are Webmachine and Cowboy. >> >> Does anyone have any advice on which one fits better my requirements, as >> below? >> >> - the server must be embeddable in another application, i.e. possible to >> start/stop without a release >> > > No problem for that with Cowboy, either through a function call or > provided child specs. > > - it must be possible to configure it to use a dynamic port value >> (multiple independent servers may run on the same machine) >> > > No problem, set {port, 0} option and then call ranch:get_port/1. > > - the REST API is delegating to an Erlang API, so ease of implementation >> would be nice :-) >> > > Cowboy is inspired by Webmachine so I suppose they're pretty close for > this one. > > - if it is possible to run multiple servers on the same Erlang runtime, >> then it would be a big plus >> > > No problem, just declare more than one listener and you're good! > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Mon Mar 7 20:47:49 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 7 Mar 2016 14:47:49 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: On Mon, Mar 7, 2016 at 2:27 PM, Vlad Dumitrescu wrote: > Hi! > > I am an almost complete noob regarding this, so please excuse me if I am > asking questions with obvious answers. > > I have an application that I'd like to expose via a REST api. From what I > could find quickly, in the Erlang world it looks like the main options are > Webmachine and Cowboy. > And Yaws. See http://yaws.hyber.org/ . > Does anyone have any advice on which one fits better my requirements, as > below? > > - the server must be embeddable in another application, i.e. possible to > start/stop without a release > - it must be possible to configure it to use a dynamic port value > (multiple independent servers may run on the same machine) > - the REST API is delegating to an Erlang API, so ease of implementation > would be nice :-) > - if it is possible to run multiple servers on the same Erlang runtime, > then it would be a big plus > Yaws can do all this and more. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From trenthampton@REDACTED Mon Mar 7 22:34:46 2016 From: trenthampton@REDACTED (Trent Hampton) Date: Mon, 7 Mar 2016 14:34:46 -0700 Subject: [erlang-questions] escript node name resolution Message-ID: Background: - erts 17 - rebar3 (beta 4) - We are building releases using rebar3 and relx - In vm.args we are using the short name -sname my_app - centos 6.3 Problem: - When we call the generated shell script with a ping; "my_app ping", it calls the nodetool ping which calls {true, pong} = {net_kernel:hidden_connect_node( 'my_app@REDACTED '),net_adm:ping('my_app@REDACTED')}. The tuple returned is usually either {false, pong}, {true, pang}, or {false, pang}. While this call does not fail every single time it does fail almost every time. If I attach to the node and run the commands from the erlang cmd prompt, it consistently succeeds. (my_app@REDACTED)242> {true, pong} = {net_kernel:hidden_connect_node( 'my_app@REDACTED'), net_adm:ping('my_app@REDACTED')}. {true,pong} The interesting thing is that when I add an alias to the /etc/hosts file for my_app, the escript consistently succeeds. Is the problem with the network configuration or with the erlang application configuration? cat /etc/hosts 10.0.5.21 my_app Any ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkbucc@REDACTED Tue Mar 8 00:30:51 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Mon, 7 Mar 2016 18:30:51 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: I'm curious to hear what do more experienced erlang'ers think of mochiweb? That's where I ended up after googling and reading on my own. On Mon, Mar 7, 2016 at 2:47 PM, Steve Vinoski wrote: > > > On Mon, Mar 7, 2016 at 2:27 PM, Vlad Dumitrescu > wrote: > >> Hi! >> >> I am an almost complete noob regarding this, so please excuse me if I am >> asking questions with obvious answers. >> >> I have an application that I'd like to expose via a REST api. From what I >> could find quickly, in the Erlang world it looks like the main options are >> Webmachine and Cowboy. >> > > And Yaws. See http://yaws.hyber.org/ . > > >> Does anyone have any advice on which one fits better my requirements, as >> below? >> >> - the server must be embeddable in another application, i.e. possible to >> start/stop without a release >> - it must be possible to configure it to use a dynamic port value >> (multiple independent servers may run on the same machine) >> - the REST API is delegating to an Erlang API, so ease of implementation >> would be nice :-) >> - if it is possible to run multiple servers on the same Erlang runtime, >> then it would be a big plus >> > > Yaws can do all this and more. > > --steve > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Mar 8 00:55:31 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 8 Mar 2016 12:55:31 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: On 8/03/16 4:53 am, Garrett Smith wrote: > > For whatever reason FP pedantry is not a draw for me. When I need > inspiration I look to Python. Maybe that's the wrong direction and we > need to drive our community through more gates. I don't know what "drive our community through more gates" means, but copying Python is *definitely* the wrong direction. Heck, even *Python* (3) hasn't followed Python (2) When it comes to cultural compatibility with Erlang, Python is better than COBOL, PL/I, or RPG. But that's about as far as I'd go. For what it's worth, in Swift there is no 'join' function; instead there are JoinGenerator and JoinSequence *types*. Monads aren't pedantry: they're (an instance of) *power*. If you can view something as a Monad, then you get a range of useful operations for free. (And this applies to other things like Monoid as well.) It's not, I think, unfair to say that Haskell : Erlang :: Category Theory : Naive Set Theory. I'm a set theory person myself; I have been trying for years to get further than chapter 3 of any category theory textbook and failing. (Not unlike the way I get the generalisation of vector spaces to tensor algebras, but then Clifford algebras trigger math-acrophobia.) I am very happy with Haskell as a typed functional language, but the Haskell community does tend to push abstraction to levels where my ears bleed. But I understand why they are doing it, and it's not pedantry: it's the passionate pursuit of programming power from precise analogies. It's the entirely reasonable desire to take a piece of code that might have worked in Lisp or Erlang and say "WHY does it work? How can we KNOW before we call it that it's going to work?" It's the lazy programmer's wish to write something just once crossed with the software engineer's wish that it should be *safe* to do so. From kennethlakin@REDACTED Tue Mar 8 01:16:20 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Mon, 7 Mar 2016 16:16:20 -0800 Subject: [erlang-questions] escript node name resolution In-Reply-To: References: Message-ID: <56DE19D4.7060506@gmail.com> On 03/07/2016 01:34 PM, Trent Hampton wrote: > Problem: > - When we call the generated shell script with a ping; "my_app ping", it > calls the nodetool ping which calls > > {true, pong} = {net_kernel:hidden_connect_node( > 'my_app@REDACTED'),net_adm:ping('my_app@REDACTED')}. > > The tuple returned is usually either {false, pong}, {true, pang}, or > {false, pang}. While this call does not fail every single time it does > fail almost every time. What happens if you add in a timer:sleep(timer:seconds(6)) to the beginning of your erlang program? If that makes it reliable, then you're maybe being bitten by something that another fellow ran in to: A while back, I tracked down what I think was the root cause of a five-second delay from the startup of a node until you could resolve names to some code in the inet_db module. The details are in this message: http://erlang.org/pipermail/erlang-questions/2015-November/086806.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From ok@REDACTED Tue Mar 8 01:24:16 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 8 Mar 2016 13:24:16 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56DDB38B.8070309@khandkar.net> References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> <56D9AAE5.8080500@khandkar.net> <575a5f3e-8d71-9f72-9617-dead72e20d1c@cs.otago.ac.nz> <56DDB38B.8070309@khandkar.net> Message-ID: On 8/03/16 5:59 am, Siraaj Khandkar wrote: > Both of the above results are functions of 2 sequences, the difference > is that 1st works with a pre-computed list, while the 2nd works with a > an infinite stream of 0s. (A) No. 0 is not an infinite stream of 0s. (B) Erlang doesn't have infinite lists. (C) Haskell *does* have infinite lists, but you still have to write cycle [0] to get one. (D) I was discussing *specific* code definitions: intersperse(Sep, [X|Xs]) -> [X | intersperse_(Xs)]; intersperse(_, []) -> []. intersperse_(Sep, [X|Xs]) -> [Sep,X | intersperse_(Xs)]; intersperse_(_, []) -> []. interleave([X|Xs], [Y|Ys]) -> [X,Y | interleave(Xs, Ys)]; interleave([], []) -> []. which have quite distinct interfaces. interleave/2 takes two list(T) sequences; intersperse/2 takes a T and a list(T). Someone raised the issue of finding functions. It's actually quite hard in most programming languages to guess the names of most built-in functions from their meaning. For example, who would have guess that "read a line from standard input" was called gets() in C? Haskell has a web interface where you can give the *type* of a function and get a list of well-known functions with that type. Squeak has (or had, I haven't updated in a while) an IDE interface where you can give an input-output *example* of a function and get a list of well-known functions that give that output for that input. It might be a good idea to do that for at least the Erlang lists module, so that you could do function_finder:find(",", ["a","b"], "a,b") ==> lists:intercalate/2 and function_finder:find([d,r,e,a,d], [a,d,d,e,r]) ==> lists:sort/1 From heinz@REDACTED Tue Mar 8 01:48:39 2016 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Mon, 7 Mar 2016 19:48:39 -0500 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> Message-ID: <90EC840A-CF33-42D5-8875-3E8A6F1651D4@licenser.net> In FiFo?s API I am dispatching in the code, for me between versions there is a lot of similarity (as in 95% of the code stays the same but there is 5% breakage) so with dispatching in the code I can prevent douplication form code. The downside here is that it?s easy to mess something up or forget to remove code when a version is deprecated. > On Mar 7, 2016, at 2:49, Torben Hoffmann wrote: > > Hi, > > I am working a HTTP API that has a version in it, like this: > /api/1/... > > Should I create separate handler modules for each version? > Or should I have code that branches out on the api version in one common > handler module? > > If I dispatch to a version specific handler module I will not have to > think about the version anymore. > > If I dispatch to a version generic handle module I can - potentially - > reuse code that is common across api versions. > > I'm pretty sure there will be new api versions - this is not just a case > of being a good citizen. The current version of the software is young, > so I would not say it is in any way cast in stone yet. > > I'm inclined to do version specific handler modules because it makes the > modules very clean in terms of contex. But I am not an expert on > webmachine/cowboy, so I thought I would ask. > > Cheers, > Torben > -- > Torben Hoffmann > Architect, basho.com > M: +45 25 14 05 38 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From on-your-mark@REDACTED Tue Mar 8 07:47:11 2016 From: on-your-mark@REDACTED (YuanZhiqian) Date: Tue, 8 Mar 2016 14:47:11 +0800 Subject: [erlang-questions] gen_server:cast from escript Message-ID: Hi guys, I was trying to call gen_server:cast from a escript file, everything that used to work well in erl shell won't work in escript, I have no idea what made the difference, here is the code, could anyone help me to find a clue? #!/usr/bin/env escript %%! -name bk@REDACTED -setcookie budget_keeper main(Argv) -> {Node, File} = case Argv of [] -> {'budget_keeper@REDACTED', "cache"}; [F] -> {'budget_keeper@REDACTED', F}; [N, F] -> {N, F} end, io:format("~p ~p ~p ~p~n", [node(), erlang:get_cookie(), Node, File]), gen_server:cast({bk_main, Node}, {dump_data, File}). As shown above, the target process is called "bk_main" which is on the node 'budget_keeper@REDACTED', I have run the same code in erl shell, and bk_main can get the notice, but nothing happened when I ran the same code in this script. Best regardsZhiqian -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Mar 8 08:09:42 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 08 Mar 2016 16:09:42 +0900 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: <1625329.6E7TUCNP1e@burrito> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: > I'm curious to hear what do more experienced erlang'ers think of mochiweb? > That's where I ended up after googling and reading on my own. Its a toolkit from which you can build a server, but its not a server in the way something like YAWS is. Quickest from zero to doing things is probably YAWS or Cowboy. Most of my experience has been client-side things using mochi libs or doing simplish web faces with YAWS for systems that have much richer native clients already -- so my web experience in this regard is (deliberately) limited. -Craig From chandrashekhar.mullaparthi@REDACTED Tue Mar 8 08:24:41 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 8 Mar 2016 07:24:41 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <56DDA67C.90204@ninenines.eu> References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> Message-ID: On 7 March 2016 at 16:04, Lo?c Hoguin wrote: > On 03/07/2016 04:53 PM, Garrett Smith wrote: > >> On Mon, Mar 7, 2016 at 9:28 AM Jesper Louis Andersen >> > > wrote: >> >> >> On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith > > wrote: >> >> It's vague, as is intercalculate, but as it's superficially >> doing what string "join" does with chars has some precedence >> within Erlang. I wouldn't call it hopeless. >> >> >> I'm probably leaning away from using 'join' at this point, since >> 'join' already have type >> >> join :: Monad M => m (m a) -> m a >> >> so from an FP perspective, that name is highly confusing since it is >> in use in monadic context and is used to join monadic data into its >> own monadic context. For a list, join is essentially 'append': >> >> Prelude Control.Monad> join ["a", "b", "c"] >> "abc" >> >> But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. >> >> >> Sigh. Okay, so the future naming discussions will involve with word >> monad and monadic? >> >> You've seen the discussions around adoption and the disruptive influence >> of Elixir? >> >> For whatever reason FP pedantry is not a draw for me. When I need >> inspiration I look to Python. Maybe that's the wrong direction and we >> need to drive our community through more gates. >> > > I am completely with you on that. If the function was called > intercalculate I'd never find it and would continue writing my own. > > I second this. I'm part of the vast number of unwashed masses who've never heard of the term intercalculate, and if I came across it in a developer's code would think that they were being a bit too clever. Like many others I have written this piece of code several times and invariably named it 'concat_with_separator' - a mouthful but it conveys (at least to me) what exactly the function is doing. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfrench@REDACTED Tue Mar 8 08:42:35 2016 From: mfrench@REDACTED (Mike French) Date: Tue, 8 Mar 2016 07:42:35 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu>, Message-ID: <1457422929204.38001@tahakom.com> I think the original suggestion was not 'intercalculate', but 'intercalate', which may be accurate, but is very obscure, even for native English speakers. 'Join' is both vague and specific, since it already has many specialized uses in algebra (e.g. lattices have 'meet' and 'join'), which Haskell has picked up. I don't have a dog in this fight, but tentatively suggest 'delimit', which is ugly, but comprehensible based on the common usage of 'delimiter'. Mike ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Chandru Sent: Tuesday, March 8, 2016 10:24 To: Lo?c Hoguin Cc: Erlang (E-mail) Subject: Re: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 On 7 March 2016 at 16:04, Lo?c Hoguin > wrote: On 03/07/2016 04:53 PM, Garrett Smith wrote: On Mon, Mar 7, 2016 at 9:28 AM Jesper Louis Andersen >> wrote: On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith >> wrote: It's vague, as is intercalculate, but as it's superficially doing what string "join" does with chars has some precedence within Erlang. I wouldn't call it hopeless. I'm probably leaning away from using 'join' at this point, since 'join' already have type join :: Monad M => m (m a) -> m a so from an FP perspective, that name is highly confusing since it is in use in monadic context and is used to join monadic data into its own monadic context. For a list, join is essentially 'append': Prelude Control.Monad> join ["a", "b", "c"] "abc" But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. Sigh. Okay, so the future naming discussions will involve with word monad and monadic? You've seen the discussions around adoption and the disruptive influence of Elixir? For whatever reason FP pedantry is not a draw for me. When I need inspiration I look to Python. Maybe that's the wrong direction and we need to drive our community through more gates. I am completely with you on that. If the function was called intercalculate I'd never find it and would continue writing my own. I second this. I'm part of the vast number of unwashed masses who've never heard of the term intercalculate, and if I came across it in a developer's code would think that they were being a bit too clever. Like many others I have written this piece of code several times and invariably named it 'concat_with_separator' - a mouthful but it conveys (at least to me) what exactly the function is doing. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfrench@REDACTED Tue Mar 8 08:46:33 2016 From: mfrench@REDACTED (Mike French) Date: Tue, 8 Mar 2016 07:46:33 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> , Message-ID: <1457423166562.68443@tahakom.com> For me, Clifford Algebras are all about the physics, not the math. Haskell : Erlang :: Clifford Algebra : Gibbs' vector notation So the usual Gibbs vector analysis stuff is a pile of loosely typed expressions that work most of the time if you are careful, but Clifford Algebra is a beautiful strongly typed hierarchy, which is usually correct by construction, but rarely used in the real world. I would honestly implore everyone to read some of David Hestenes's work on Geometric Algebra for the sheer aesthetic pleasure of seeing how something can be beautiful, powerful and true (in the physics sense of gaining useful insight) all at the same time. Regards, Mike ________________________________________ From: erlang-questions-bounces@REDACTED on behalf of Richard A. O'Keefe Sent: Tuesday, March 8, 2016 02:55 To: Garrett Smith; Jesper Louis Andersen Cc: Erlang (E-mail) Subject: Re: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 On 8/03/16 4:53 am, Garrett Smith wrote: > > For whatever reason FP pedantry is not a draw for me. When I need > inspiration I look to Python. Maybe that's the wrong direction and we > need to drive our community through more gates. I don't know what "drive our community through more gates" means, but copying Python is *definitely* the wrong direction. Heck, even *Python* (3) hasn't followed Python (2) When it comes to cultural compatibility with Erlang, Python is better than COBOL, PL/I, or RPG. But that's about as far as I'd go. For what it's worth, in Swift there is no 'join' function; instead there are JoinGenerator and JoinSequence *types*. Monads aren't pedantry: they're (an instance of) *power*. If you can view something as a Monad, then you get a range of useful operations for free. (And this applies to other things like Monoid as well.) It's not, I think, unfair to say that Haskell : Erlang :: Category Theory : Naive Set Theory. I'm a set theory person myself; I have been trying for years to get further than chapter 3 of any category theory textbook and failing. (Not unlike the way I get the generalisation of vector spaces to tensor algebras, but then Clifford algebras trigger math-acrophobia.) I am very happy with Haskell as a typed functional language, but the Haskell community does tend to push abstraction to levels where my ears bleed. But I understand why they are doing it, and it's not pedantry: it's the passionate pursuit of programming power from precise analogies. It's the entirely reasonable desire to take a piece of code that might have worked in Lisp or Erlang and say "WHY does it work? How can we KNOW before we call it that it's going to work?" It's the lazy programmer's wish to write something just once crossed with the software engineer's wish that it should be *safe* to do so. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Tue Mar 8 09:21:21 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 8 Mar 2016 08:21:21 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <1457422929204.38001@tahakom.com> References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> <1457422929204.38001@tahakom.com> Message-ID: > ... 'intercalculate' ... 'intercalate' ... > Let's call the whole thing off. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Tue Mar 8 11:40:23 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Tue, 08 Mar 2016 11:40:23 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> <56dda60f.6610700a.73209.ffffcd05@mx.google.com> Message-ID: <56deac90.71268c0a.d2b8f.3781@mx.google.com> Jesper Louis Andersen writes: > [ text/plain ] > On Mon, Mar 7, 2016 at 2:16 PM, Torben Hoffmann wrote: > >> Well, I don't know what version 2 will look like yet :-) > > > Roy T. Fielding has some words to say about versioning APIs in the first > place, which I think one should look at: > > http://www.infoq.com/articles/roy-fielding-on-versioning > > -- > J. Thx, Jesper. That is a good article. Not sure if it is the right time for this though... Cheers, Torben -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From thoffmann@REDACTED Tue Mar 8 11:42:19 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Tue, 08 Mar 2016 11:42:19 +0100 Subject: [erlang-questions] Code structure for webmachine/cowboy with versioned API In-Reply-To: <90EC840A-CF33-42D5-8875-3E8A6F1651D4@licenser.net> References: <56dd3292.a328700a.cf66b.ffffb454@mx.google.com> <90EC840A-CF33-42D5-8875-3E8A6F1651D4@licenser.net> Message-ID: <56deac93.88fe8c0a.2faef.375b@mx.google.com> Heinz Nikolaus Gies writes: > [ text/plain ] > In FiFo?s API I am dispatching in the code, for me between versions > there is a lot of similarity (as in 95% of the code stays the same but > there is 5% breakage) so with dispatching in the code I can prevent > douplication form code. The downside here is that it?s easy to mess > something up or forget to remove code when a version is deprecated. That sounds very reasonable. For now I have punted the issue to future me, so the amount of breakage will decide for me what course to take. Thanks, Torben >> On Mar 7, 2016, at 2:49, Torben Hoffmann wrote: >> >> Hi, >> >> I am working a HTTP API that has a version in it, like this: >> /api/1/... >> >> Should I create separate handler modules for each version? >> Or should I have code that branches out on the api version in one common >> handler module? >> >> If I dispatch to a version specific handler module I will not have to >> think about the version anymore. >> >> If I dispatch to a version generic handle module I can - potentially - >> reuse code that is common across api versions. >> >> I'm pretty sure there will be new api versions - this is not just a case >> of being a good citizen. The current version of the software is young, >> so I would not say it is in any way cast in stone yet. >> >> I'm inclined to do version specific handler modules because it makes the >> modules very clean in terms of contex. But I am not an expert on >> webmachine/cowboy, so I thought I would ask. >> >> Cheers, >> Torben >> -- >> Torben Hoffmann >> Architect, basho.com >> M: +45 25 14 05 38 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > [ signature.asc: application/pgp-signature ] -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From roberto@REDACTED Tue Mar 8 11:52:42 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 8 Mar 2016 11:52:42 +0100 Subject: [erlang-questions] [ANN] PGPool 1.0.0 Message-ID: Dear All, PGPool 1.0.0 has been released. This is mainly a version bump from 0.9.0, since it has successfully been running in production for months now. For those of you who don't know what PGPool is: PGPool is a PosgreSQL client that automatically uses connection pools and handles reconnections in case of errors You can find PGPool here: https://github.com/ostinelli/pgpool Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Tue Mar 8 12:07:17 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Tue, 08 Mar 2016 12:07:17 +0100 Subject: [erlang-questions] Webmachine content_types_accepted w/ POST Message-ID: <56deb26f.c6ab8c0a.5e8c0.3ae9@mx.google.com> Hi, I have a URI where I accept a POST. It has to support one or multiple entities for creation as JSON in the body. So post_is_create/2 should return false. Hence I have to implement process_post/2 to deal with the POST. So far, so good. But I have to write content_types_accepted(RD, Ctx) -> {[{"application/json", bogus_handler_name}], RD, Ctx}. for the JSON body to be accepted. I can write whatever as the handler function, because it will not be called by Webmachine. I only allow JSON on that POST, so I can control it and return an empty list of accepted types for other URI/method combos. It seems a little dirty to me. I can live with it, just wanted to hear what others do. Cheers, Torben -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From essen@REDACTED Tue Mar 8 13:09:30 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 8 Mar 2016 13:09:30 +0100 Subject: [erlang-questions] Webmachine content_types_accepted w/ POST In-Reply-To: <56deb26f.c6ab8c0a.5e8c0.3ae9@mx.google.com> References: <56deb26f.c6ab8c0a.5e8c0.3ae9@mx.google.com> Message-ID: <56DEC0FA.4030600@ninenines.eu> On 03/08/2016 12:07 PM, Torben Hoffmann wrote: > Hi, > > I have a URI where I accept a POST. > > It has to support one or multiple entities for creation as JSON in the > body. > So post_is_create/2 should return false. > Hence I have to implement process_post/2 to deal with the POST. > So far, so good. > > But I have to write > > content_types_accepted(RD, Ctx) -> > {[{"application/json", bogus_handler_name}], RD, Ctx}. > > for the JSON body to be accepted. Cowboy REST has the same behavior. I recommend putting undefined personally. I suppose '' would also work. It might be worth having a special return value known to Webmachine/Cowboy that would never get dispatched, in case the developer leaves GET enabled. > I can write whatever as the handler function, because it will not be > called by Webmachine. > > I only allow JSON on that POST, so I can control it and return an empty > list of accepted types for other URI/method combos. > > It seems a little dirty to me. > I can live with it, just wanted to hear what others do. > > Cheers, > Torben > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From bengt.kleberg@REDACTED Tue Mar 8 13:33:54 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 8 Mar 2016 13:33:54 +0100 Subject: [erlang-questions] link: Software dev 101: 'The best time to understand how your system works is when it is dying' Message-ID: <56DEC6B2.1020303@ericsson.com> Greetings, Positive mentioning of Erlang: http://www.theregister.co.uk/2016/03/08/software_dev_tips_from_qcon/ Bengt From siraaj@REDACTED Tue Mar 8 15:02:42 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 8 Mar 2016 09:02:42 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <56D71CB9.2010209@khandkar.net> <0a631b61-a7b6-38d9-ab2f-451b7c2e107c@cs.otago.ac.nz> <56D7922B.1090102@khandkar.net> <693bc3e4-a620-e88d-8afa-b50f8f68a06e@cs.otago.ac.nz> <56D9AAE5.8080500@khandkar.net> <575a5f3e-8d71-9f72-9617-dead72e20d1c@cs.otago.ac.nz> <56DDB38B.8070309@khandkar.net> Message-ID: <56DEDB82.3040005@khandkar.net> On 3/7/16 7:24 PM, Richard A. O'Keefe wrote: > On 8/03/16 5:59 am, Siraaj Khandkar wrote: > >> Both of the above results are functions of 2 sequences, the difference >> is that 1st works with a pre-computed list, while the 2nd works with a >> an infinite stream of 0s. > > (A) No. 0 is not an infinite stream of 0s. Then, given a 0 and [1, 2, 3], you can't do much better than [1, 0, 2, 3] > (B) Erlang doesn't have infinite lists. There's no primitive named "sequence" in Erlang either. > (C) Haskell *does* have infinite lists, but you still have to > write cycle [0] to get one. > (D) I was discussing *specific* code definitions: > > intersperse(Sep, [X|Xs]) -> [X | intersperse_(Xs)]; > intersperse(_, []) -> []. > > intersperse_(Sep, [X|Xs]) -> [Sep,X | intersperse_(Xs)]; > intersperse_(_, []) -> []. The above function clearly constructs the second sequence (of Seps) on the fly. Please note the obvious similarity to: -spec seq(A) :: nil | {cons, A, seq(A)}. > > interleave([X|Xs], [Y|Ys]) -> [X,Y | interleave(Xs, Ys)]; > interleave([], []) -> []. > > which have quite distinct interfaces. interleave/2 takes two > list(T) sequences; intersperse/2 takes a T and a list(T). > > Someone raised the issue of finding functions. > It's actually quite hard in most programming languages to guess > the names of most built-in functions from their meaning. > For example, who would have guess that "read a line from > standard input" was called gets() in C? > > Haskell has a web interface where you can give the *type* of > a function and get a list of well-known functions with that > type. Squeak has (or had, I haven't updated in a while) an > IDE interface where you can give an input-output *example* > of a function and get a list of well-known functions that give > that output for that input. It might be a good idea to do > that for at least the Erlang lists module, so that you could > do function_finder:find(",", ["a","b"], "a,b") > ==> lists:intercalate/2 > and function_finder:find([d,r,e,a,d], [a,d,d,e,r]) > ==> lists:sort/1 These are, indeed, great ideas! There's been an effort to do similar for OCaml: http://ocamloscope.herokuapp.com/ Which, I have to admit, I have not felt the urge to use in practice. From siraaj@REDACTED Tue Mar 8 15:12:13 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 8 Mar 2016 09:12:13 -0500 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <1457423166562.68443@tahakom.com> References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <1457423166562.68443@tahakom.com> Message-ID: <56DEDDBD.9000304@khandkar.net> There's really no necessity to appeal to technical algebraic terminology (at least in this case :)). In common, everyday usage, "join" means to connect multiple components into one (usually using some sort of a connector part), so, for strings - its common usage is just fine, while for sequences/lists - the closest match seems to be fold/reduce and friends. On 3/8/16 2:46 AM, Mike French wrote: > For me, Clifford Algebras are all about the physics, not the math. > > Haskell : Erlang :: Clifford Algebra : Gibbs' vector notation > > So the usual Gibbs vector analysis stuff is a pile of loosely typed expressions that work most of the time if you are careful, but Clifford Algebra is a beautiful strongly typed hierarchy, which is usually correct by construction, but rarely used in the real world. > > I would honestly implore everyone to read some of David Hestenes's work on Geometric Algebra for the sheer aesthetic pleasure of seeing how something can be beautiful, powerful and true (in the physics sense of gaining useful insight) all at the same time. > > Regards, > Mike > ________________________________________ > From: erlang-questions-bounces@REDACTED on behalf of Richard A. O'Keefe > Sent: Tuesday, March 8, 2016 02:55 > To: Garrett Smith; Jesper Louis Andersen > Cc: Erlang (E-mail) > Subject: Re: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 > > On 8/03/16 4:53 am, Garrett Smith wrote: >> >> For whatever reason FP pedantry is not a draw for me. When I need >> inspiration I look to Python. Maybe that's the wrong direction and we >> need to drive our community through more gates. > > I don't know what "drive our community through more gates" means, > but copying Python is *definitely* the wrong direction. Heck, even > *Python* (3) hasn't followed Python (2) When it comes to cultural > compatibility with Erlang, Python is better than COBOL, PL/I, or RPG. > But that's about as far as I'd go. > > For what it's worth, in Swift there is no 'join' function; instead there > are JoinGenerator and JoinSequence *types*. > > Monads aren't pedantry: they're (an instance of) *power*. If you can > view something as a Monad, then you get a range of useful operations > for free. (And this applies to other things like Monoid as well.) It's > not, I think, unfair to say that > Haskell : Erlang :: Category Theory : Naive Set Theory. > I'm a set theory person myself; I have been trying for years to get > further than chapter 3 of any category theory textbook and failing. > (Not unlike the way I get the generalisation of vector spaces to > tensor algebras, but then Clifford algebras trigger math-acrophobia.) > I am very happy with Haskell as a typed functional language, but the > Haskell community does tend to push abstraction to levels where my > ears bleed. But I understand why they are doing it, and it's not > pedantry: it's the passionate pursuit of programming power from > precise analogies. It's the entirely reasonable desire to take a > piece of code that might have worked in Lisp or Erlang and say > "WHY does it work? How can we KNOW before we call it that it's > going to work?" It's the lazy programmer's wish to write something > just once crossed with the software engineer's wish that it should be > *safe* to do so. > From eajam@REDACTED Tue Mar 8 17:30:16 2016 From: eajam@REDACTED (Alex Alvarez) Date: Tue, 8 Mar 2016 11:30:16 -0500 Subject: [erlang-questions] REST server In-Reply-To: <1625329.6E7TUCNP1e@burrito> References: <1625329.6E7TUCNP1e@burrito> Message-ID: For REST, my initial recommendation is Yaws (http://yaws.hyber.org/), as it's very easy to use and pretty solid solution, but the documentation for it is simply terrible and I mean really, really terrible. Hence, you probably would have an easier time with Cowboy. Cheers, Alex On 03/08/2016 02:09 AM, zxq9 wrote: > On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >> I'm curious to hear what do more experienced erlang'ers think of mochiweb? >> That's where I ended up after googling and reading on my own. > Its a toolkit from which you can build a server, but its not a server > in the way something like YAWS is. Quickest from zero to doing things > is probably YAWS or Cowboy. > > Most of my experience has been client-side things using mochi libs or > doing simplish web faces with YAWS for systems that have much richer > native clients already -- so my web experience in this regard is > (deliberately) limited. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Mar 8 17:56:13 2016 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 8 Mar 2016 08:56:13 -0800 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> Message-ID: I'd call it punctuate (to mark or divide with punctuation marks) - since this is what I'd use this of, so punctuate(Claues, ';') means a a ';' between clauses. We could make a new library called stacks which is all of lists renamed with new things in and phase out lists. Lists are actually stacks - explaining how to reverse a stack to beginners is far easier than a list - I just get them to think of a stack of plates in a restaurant, one empty one full .... Just because lisp go it wrong doesn't mean to say that everybody else should follow /Joe On Mon, Mar 7, 2016 at 11:24 PM, Chandru wrote: > On 7 March 2016 at 16:04, Lo?c Hoguin wrote: >> >> On 03/07/2016 04:53 PM, Garrett Smith wrote: >>> >>> On Mon, Mar 7, 2016 at 9:28 AM Jesper Louis Andersen >>> >> > wrote: >>> >>> >>> On Mon, Mar 7, 2016 at 4:08 PM, Garrett Smith >> > wrote: >>> >>> It's vague, as is intercalculate, but as it's superficially >>> doing what string "join" does with chars has some precedence >>> within Erlang. I wouldn't call it hopeless. >>> >>> >>> I'm probably leaning away from using 'join' at this point, since >>> 'join' already have type >>> >>> join :: Monad M => m (m a) -> m a >>> >>> so from an FP perspective, that name is highly confusing since it is >>> in use in monadic context and is used to join monadic data into its >>> own monadic context. For a list, join is essentially 'append': >>> >>> Prelude Control.Monad> join ["a", "b", "c"] >>> "abc" >>> >>> But join is monadic, so `join $ Just Nothing` evaluates to `Nothing`. >>> >>> >>> Sigh. Okay, so the future naming discussions will involve with word >>> monad and monadic? >>> >>> You've seen the discussions around adoption and the disruptive influence >>> of Elixir? >>> >>> For whatever reason FP pedantry is not a draw for me. When I need >>> inspiration I look to Python. Maybe that's the wrong direction and we >>> need to drive our community through more gates. >> >> >> I am completely with you on that. If the function was called >> intercalculate I'd never find it and would continue writing my own. >> > > I second this. I'm part of the vast number of unwashed masses who've never > heard of the term intercalculate, and if I came across it in a developer's > code would think that they were being a bit too clever. Like many others I > have written this piece of code several times and invariably named it > 'concat_with_separator' - a mouthful but it conveys (at least to me) what > exactly the function is doing. > > cheers, > Chandru > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vinoski@REDACTED Tue Mar 8 18:10:19 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 8 Mar 2016 12:10:19 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez wrote: > For REST, my initial recommendation is Yaws (http://yaws.hyber.org/), as > it's very easy to use and pretty solid solution, but the documentation for > it is simply terrible and I mean really, really terrible. Hence, you > probably would have an easier time with Cowboy. > If you have suggestions for documentation improvements please let me know -- I'd like to know details of why you think the docs are "really, really terrible" given that I think they're pretty reasonable, and I know for a fact we keep them up to date. Pull requests are also welcomed. --steve > > Cheers, > Alex > > > On 03/08/2016 02:09 AM, zxq9 wrote: > > On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: > > I'm curious to hear what do more experienced erlang'ers think of mochiweb? > That's where I ended up after googling and reading on my own. > > Its a toolkit from which you can build a server, but its not a server > in the way something like YAWS is. Quickest from zero to doing things > is probably YAWS or Cowboy. > > Most of my experience has been client-side things using mochi libs or > doing simplish web faces with YAWS for systems that have much richer > native clients already -- so my web experience in this regard is > (deliberately) limited. > > -Craig > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungkkr@REDACTED Tue Mar 8 18:12:41 2016 From: jungkkr@REDACTED (Jung Kim) Date: Wed, 9 Mar 2016 01:12:41 +0800 Subject: [erlang-questions] How does Erlang schedule process? Message-ID: I am very new to Erlang, and just read that Erlang style process is done with something similar to coroutine + preemptive scheduling[1]. So I am interested to know how Erlang schedule process without the user inserting code like coroutine.yield() in the function created. Is there any document detail about this (I find some discussion like [2] but that doesn't give me a clear blueprint)? Or which part of source code should I look into? I appreciate any suggestions. [1]. https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html [2]. http://erlang.org/pipermail/erlang-questions/2001-April/003132.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Mar 8 19:03:45 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 8 Mar 2016 19:03:45 +0100 Subject: [erlang-questions] How does Erlang schedule process? In-Reply-To: References: Message-ID: On Tue, Mar 8, 2016 at 6:12 PM, Jung Kim wrote: > So I am interested to know how Erlang schedule process without the user > inserting code like coroutine.yield() in the function created. Two observations: * Every loop has to use a tail-call. * The interpreter automatically inserts a yield at every 2000 function calls. Of course, if you end up waiting for a message and none has arrived, you are put on a sleep queue until such a message arrives. More here: http://jlouisramblings.blogspot.dk/2013/01/how-erlang-does-scheduling.html .. it is still somewhat up-to-date with the current state-of-the-art. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Tue Mar 8 19:06:17 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Tue, 8 Mar 2016 19:06:17 +0100 Subject: [erlang-questions] How does Erlang schedule process? In-Reply-To: References: Message-ID: <56DF1499.6050207@ericsson.com> This thesis gives some background and orientation: Characterizing the Scalability of Erlang VM on Many-core Processors (Jianrong Zhang) http://www.diva-portal.org/smash/get/diva2:392243/FULLTEXT01.pdf It's from 2011 so the information is not up to date on but should be good enough. For details, look at https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.c#L9368-L9972 and in beam_emu.c. Start with the thesis. On 2016-03-08 18:12, Jung Kim wrote: > I am very new to Erlang, and just read that Erlang style process is > done with something similar to coroutine + preemptive scheduling[1]. > So I am interested to know how Erlang schedule process without the > user inserting code like coroutine.yield() in the function created. Is > there any document detail about this (I find some discussion like [2] > but that doesn't give me a clear blueprint)? Or which part of source > code should I look into? > > I appreciate any suggestions. > > [1]. > https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/migration/2016/02/09/erlang-scheduler-details.html > [2]. http://erlang.org/pipermail/erlang-questions/2001-April/003132.html > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Tue Mar 8 19:31:52 2016 From: garry@REDACTED (Garry Hodgson) Date: Tue, 8 Mar 2016 13:31:52 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: <56DDDA17.2060403@ninenines.eu> Message-ID: <56DF1A98.8070104@research.att.com> I can't speak to ease of building user interfaces, as we're building web security components. We started off using Webmachine, and were pretty happy with it. But some of our REST services were sending very large messages, and the memory consumption of Webmachine, which uses strings, became a big problem. Cowboy uses binaries instead, and was a much more comfortable fit. We also need to implement various binary protocols, and being able to do so using Cowboy's underlying Ranch acceptor pools was a big win. Converting Webmachine apps to Cowboy apps was pretty straightforward, and Cowboy feels like a more mature and feature rich Webmachine. As you evaluate your options, I'd consider these as two implementations of the Webmachine model, and go with the Cowboy implementation if you choose that route. On 3/7/16 2:46 PM, Vlad Dumitrescu wrote: > Thanks, Lo?c, I will then check out Cowboy in more detail, to see how > it feels. > > regards, > Vlad > > > On Mon, Mar 7, 2016 at 8:44 PM, Lo?c Hoguin > wrote: > > On 03/07/2016 08:27 PM, Vlad Dumitrescu wrote: > > Hi! > > I am an almost complete noob regarding this, so please excuse > me if I am > asking questions with obvious answers. > > I have an application that I'd like to expose via a REST api. > From what > I could find quickly, in the Erlang world it looks like the > main options > are Webmachine and Cowboy. > > Does anyone have any advice on which one fits better my > requirements, as > below? > > - the server must be embeddable in another application, i.e. > possible to > start/stop without a release > > > No problem for that with Cowboy, either through a function call or > provided child specs. > > - it must be possible to configure it to use a dynamic port value > (multiple independent servers may run on the same machine) > > > No problem, set {port, 0} option and then call ranch:get_port/1. > > - the REST API is delegating to an Erlang API, so ease of > implementation > would be nice :-) > > > Cowboy is inspired by Webmachine so I suppose they're pretty close > for this one. > > - if it is possible to run multiple servers on the same Erlang > runtime, > then it would be a big plus > > > No problem, just declare more than one listener and you're good! > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Mar 8 21:01:28 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 8 Mar 2016 21:01:28 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: Hi, Where is yaws documentation relative to how to implement a REST API? I can't see anything referring to that specifically, so I suppose it would involve appmods? best regards, Vlad On Tue, Mar 8, 2016 at 6:10 PM, Steve Vinoski wrote: > > > On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez wrote: > >> For REST, my initial recommendation is Yaws (http://yaws.hyber.org/), as >> it's very easy to use and pretty solid solution, but the documentation for >> it is simply terrible and I mean really, really terrible. Hence, you >> probably would have an easier time with Cowboy. >> > > If you have suggestions for documentation improvements please let me know > -- I'd like to know details of why you think the docs are "really, really > terrible" given that I think they're pretty reasonable, and I know for a > fact we keep them up to date. Pull requests are also welcomed. > > --steve > > >> >> Cheers, >> Alex >> >> >> On 03/08/2016 02:09 AM, zxq9 wrote: >> >> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >> >> I'm curious to hear what do more experienced erlang'ers think of mochiweb? >> That's where I ended up after googling and reading on my own. >> >> Its a toolkit from which you can build a server, but its not a server >> in the way something like YAWS is. Quickest from zero to doing things >> is probably YAWS or Cowboy. >> >> Most of my experience has been client-side things using mochi libs or >> doing simplish web faces with YAWS for systems that have much richer >> native clients already -- so my web experience in this regard is >> (deliberately) limited. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Mar 8 21:11:39 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 8 Mar 2016 21:11:39 +0100 Subject: [erlang-questions] REST server In-Reply-To: <56DF1A98.8070104@research.att.com> References: <56DDDA17.2060403@ninenines.eu> <56DF1A98.8070104@research.att.com> Message-ID: Cool, thanks! /Vlad On Tue, Mar 8, 2016 at 7:31 PM, Garry Hodgson wrote: > I can't speak to ease of building user interfaces, as we're building > web security components. We started off using Webmachine, and > were pretty happy with it. But some of our REST services were sending > very large messages, and the memory consumption of Webmachine, > which uses strings, became a big problem. Cowboy uses binaries > instead, and was a much more comfortable fit. We also need to > implement various binary protocols, and being able to do so using > Cowboy's underlying Ranch acceptor pools was a big win. > > Converting Webmachine apps to Cowboy apps was pretty straightforward, > and Cowboy feels like a more mature and feature rich Webmachine. As > you evaluate your options, I'd consider these as two implementations > of the Webmachine model, and go with the Cowboy implementation if > you choose that route. > > > > > > On 3/7/16 2:46 PM, Vlad Dumitrescu wrote: > > Thanks, Lo?c, I will then check out Cowboy in more detail, to see how it > feels. > > regards, > Vlad > > > On Mon, Mar 7, 2016 at 8:44 PM, Lo?c Hoguin wrote: > >> On 03/07/2016 08:27 PM, Vlad Dumitrescu wrote: >> >>> Hi! >>> >>> I am an almost complete noob regarding this, so please excuse me if I am >>> asking questions with obvious answers. >>> >>> I have an application that I'd like to expose via a REST api. From what >>> I could find quickly, in the Erlang world it looks like the main options >>> are Webmachine and Cowboy. >>> >>> Does anyone have any advice on which one fits better my requirements, as >>> below? >>> >>> - the server must be embeddable in another application, i.e. possible to >>> start/stop without a release >>> >> >> No problem for that with Cowboy, either through a function call or >> provided child specs. >> >> - it must be possible to configure it to use a dynamic port value >>> (multiple independent servers may run on the same machine) >>> >> >> No problem, set {port, 0} option and then call ranch:get_port/1. >> >> - the REST API is delegating to an Erlang API, so ease of implementation >>> would be nice :-) >>> >> >> Cowboy is inspired by Webmachine so I suppose they're pretty close for >> this one. >> >> - if it is possible to run multiple servers on the same Erlang runtime, >>> then it would be a big plus >>> >> >> No problem, just declare more than one listener and you're good! >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Tue Mar 8 21:15:42 2016 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Tue, 8 Mar 2016 22:15:42 +0200 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: I used this as a guide for implementing REST services with Yaws: http://www.infoq.com/articles/vinoski-erlang-rest I like the flexibility of this approach. On the other side, each service takes only 5-20 lines of code to implement it. Karolis On Tue, Mar 8, 2016 at 10:01 PM, Vlad Dumitrescu wrote: > Hi, > > Where is yaws documentation relative to how to implement a REST API? I can't > see anything referring to that specifically, so I suppose it would involve > appmods? > > best regards, > Vlad > > > On Tue, Mar 8, 2016 at 6:10 PM, Steve Vinoski wrote: >> >> >> >> On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez wrote: >>> >>> For REST, my initial recommendation is Yaws (http://yaws.hyber.org/), as >>> it's very easy to use and pretty solid solution, but the documentation for >>> it is simply terrible and I mean really, really terrible. Hence, you >>> probably would have an easier time with Cowboy. >> >> >> If you have suggestions for documentation improvements please let me know >> -- I'd like to know details of why you think the docs are "really, really >> terrible" given that I think they're pretty reasonable, and I know for a >> fact we keep them up to date. Pull requests are also welcomed. >> >> --steve >> >>> >>> >>> Cheers, >>> Alex >>> >>> >>> On 03/08/2016 02:09 AM, zxq9 wrote: >>> >>> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >>> >>> I'm curious to hear what do more experienced erlang'ers think of >>> mochiweb? >>> That's where I ended up after googling and reading on my own. >>> >>> Its a toolkit from which you can build a server, but its not a server >>> in the way something like YAWS is. Quickest from zero to doing things >>> is probably YAWS or Cowboy. >>> >>> Most of my experience has been client-side things using mochi libs or >>> doing simplish web faces with YAWS for systems that have much richer >>> native clients already -- so my web experience in this regard is >>> (deliberately) limited. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eajam@REDACTED Tue Mar 8 22:16:17 2016 From: eajam@REDACTED (Alex Alvarez) Date: Tue, 8 Mar 2016 16:16:17 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: Note that I very much like Yaws. I've been using it for a number of years and I've never ran into any operational issues with it. Thus, I'm very thankful to Claes and all contributors to the project. And yes, it's very easy to complain while you don't do anything about it, but since I was asked... The issues dealing with documentation I've run into could be summarized in Yaws having many great features available, but in many cases the documentation doesn't go far enough explaining how to go about using them. That probably sums it up. The website has bits and pieces for you to figure out, which is fine if you have the time, but it's not exactly an example of good documentation in 2016. Your best overall bet is the manual (yaws.pdf) that tries to be complete, but most explanations are simple and limited. Many chapters are simply made up of one page, which is not exactly thorough. I mean, take a look and judge for yourself. I wish I would have written this a few years ago when I was writing this web app, as I would have had better examples of the issues I ran into. On 03/08/2016 12:10 PM, Steve Vinoski wrote: > > > On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez > wrote: > > For REST, my initial recommendation is Yaws > (http://yaws.hyber.org/), as it's very easy to use and pretty > solid solution, but the documentation for it is simply terrible > and I mean really, really terrible. Hence, you probably would > have an easier time with Cowboy. > > > If you have suggestions for documentation improvements please let me > know -- I'd like to know details of why you think the docs are > "really, really terrible" given that I think they're pretty > reasonable, and I know for a fact we keep them up to date. Pull > requests are also welcomed. > > --steve > > > Cheers, > Alex > > > On 03/08/2016 02:09 AM, zxq9 wrote: >> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >>> I'm curious to hear what do more experienced erlang'ers think of mochiweb? >>> That's where I ended up after googling and reading on my own. >> Its a toolkit from which you can build a server, but its not a server >> in the way something like YAWS is. Quickest from zero to doing things >> is probably YAWS or Cowboy. >> >> Most of my experience has been client-side things using mochi libs or >> doing simplish web faces with YAWS for systems that have much richer >> native clients already -- so my web experience in this regard is >> (deliberately) limited. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Mar 8 22:22:04 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 8 Mar 2016 22:22:04 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: Re yaws docs: Could the contents of the article Karolis pointed to be included in the docs? Or at least a link to it... /Vlad On Tue, Mar 8, 2016 at 10:16 PM, Alex Alvarez wrote: > Note that I very much like Yaws. I've been using it for a number of years > and I've never ran into any operational issues with it. Thus, I'm very > thankful to Claes and all contributors to the project. And yes, it's very > easy to complain while you don't do anything about it, but since I was > asked... The issues dealing with documentation I've run into could be > summarized in Yaws having many great features available, but in many cases > the documentation doesn't go far enough explaining how to go about using > them. That probably sums it up. The website has bits and pieces for you > to figure out, which is fine if you have the time, but it's not exactly an > example of good documentation in 2016. Your best overall bet is the manual > (yaws.pdf) that tries to be complete, but most explanations are simple and > limited. Many chapters are simply made up of one page, which is not > exactly thorough. I mean, take a look and judge for yourself. I wish I > would have written this a few years ago when I was writing this web app, as > I would have had better examples of the issues I ran into. > > > > On 03/08/2016 12:10 PM, Steve Vinoski wrote: > > > > On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez wrote: > >> For REST, my initial recommendation is Yaws ( >> http://yaws.hyber.org/), as it's very easy to use and pretty solid >> solution, but the documentation for it is simply terrible and I mean >> really, really terrible. Hence, you probably would have an easier time >> with Cowboy. >> > > If you have suggestions for documentation improvements please let me know > -- I'd like to know details of why you think the docs are "really, really > terrible" given that I think they're pretty reasonable, and I know for a > fact we keep them up to date. Pull requests are also welcomed. > > --steve > > >> >> Cheers, >> Alex >> >> >> On 03/08/2016 02:09 AM, zxq9 wrote: >> >> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >> >> I'm curious to hear what do more experienced erlang'ers think of mochiweb? >> That's where I ended up after googling and reading on my own. >> >> Its a toolkit from which you can build a server, but its not a server >> in the way something like YAWS is. Quickest from zero to doing things >> is probably YAWS or Cowboy. >> >> Most of my experience has been client-side things using mochi libs or >> doing simplish web faces with YAWS for systems that have much richer >> native clients already -- so my web experience in this regard is >> (deliberately) limited. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Tue Mar 8 22:37:07 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 8 Mar 2016 16:37:07 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: On Tue, Mar 8, 2016 at 4:22 PM, Vlad Dumitrescu wrote: > Re yaws docs: Could the contents of the article Karolis pointed to be > included in the docs? Or at least a link to it... > /Vlad > There are links to all known Yaws articles and books here: http://yaws.hyber.org/articles.yaws --steve > > > On Tue, Mar 8, 2016 at 10:16 PM, Alex Alvarez wrote: > >> Note that I very much like Yaws. I've been using it for a number of >> years and I've never ran into any operational issues with it. Thus, I'm >> very thankful to Claes and all contributors to the project. And yes, it's >> very easy to complain while you don't do anything about it, but since I was >> asked... The issues dealing with documentation I've run into could be >> summarized in Yaws having many great features available, but in many cases >> the documentation doesn't go far enough explaining how to go about using >> them. That probably sums it up. The website has bits and pieces for you >> to figure out, which is fine if you have the time, but it's not exactly an >> example of good documentation in 2016. Your best overall bet is the manual >> (yaws.pdf) that tries to be complete, but most explanations are simple and >> limited. Many chapters are simply made up of one page, which is not >> exactly thorough. I mean, take a look and judge for yourself. I wish I >> would have written this a few years ago when I was writing this web app, as >> I would have had better examples of the issues I ran into. >> >> >> >> On 03/08/2016 12:10 PM, Steve Vinoski wrote: >> >> >> >> On Tue, Mar 8, 2016 at 11:30 AM, Alex Alvarez wrote: >> >>> For REST, my initial recommendation is Yaws ( >>> http://yaws.hyber.org/), as it's very easy to use and pretty solid >>> solution, but the documentation for it is simply terrible and I mean >>> really, really terrible. Hence, you probably would have an easier time >>> with Cowboy. >>> >> >> If you have suggestions for documentation improvements please let me know >> -- I'd like to know details of why you think the docs are "really, really >> terrible" given that I think they're pretty reasonable, and I know for a >> fact we keep them up to date. Pull requests are also welcomed. >> >> --steve >> >> >>> >>> Cheers, >>> Alex >>> >>> >>> On 03/08/2016 02:09 AM, zxq9 wrote: >>> >>> On Monday 07 March 2016 18:30:51 Mark Bucciarelli wrote: >>> >>> I'm curious to hear what do more experienced erlang'ers think of mochiweb? >>> That's where I ended up after googling and reading on my own. >>> >>> Its a toolkit from which you can build a server, but its not a server >>> in the way something like YAWS is. Quickest from zero to doing things >>> is probably YAWS or Cowboy. >>> >>> Most of my experience has been client-side things using mochi libs or >>> doing simplish web faces with YAWS for systems that have much richer >>> native clients already -- so my web experience in this regard is >>> (deliberately) limited. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Mar 8 22:41:06 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 8 Mar 2016 22:41:06 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: On Tue, Mar 8, 2016 at 10:37 PM, Steve Vinoski wrote: > On Tue, Mar 8, 2016 at 4:22 PM, Vlad Dumitrescu > wrote: > >> Re yaws docs: Could the contents of the article Karolis pointed to be >> included in the docs? Or at least a link to it... >> /Vlad >> > > There are links to all known Yaws articles and books here: > > http://yaws.hyber.org/articles.yaws > Oh, yes, missed that... For the InfoQ article, I wouldn't have guessed it's about REST and not "regular" usage. /Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Mar 9 00:18:44 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 9 Mar 2016 12:18:44 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> Message-ID: <92d3e03a-cc17-7717-e58c-76968d76e217@cs.otago.ac.nz> On 8/03/16 8:24 pm, Chandru wrote: > On 7 March 2016 at 16:04, Lo?c Hoguin > wrote: > > > I am completely with you on that. If the function was called > intercalculate I'd never find it and would continue writing my own. > > > I second this. I'm part of the vast number of unwashed masses who've > never heard of the term intercalculate, Please, if you are going to pour scorn on a word, at least COPY the thing correctly. The proposal is not "intercalCULate" but "intercalate", and it's a real English word with a history going back at least 400 years in English, 500 years in French, and 2500 years in Latin. I think it's a bad name not because nobody knows what it means (I'd expect any skilled native speaker of English to have met it several times) but because it means to insert ONE (unusual) thing into a sequence of (ordinary) things, as in one leap day or one leap month into a year. > and if I came across it in a developer's code would think that they > were being a bit too clever. Like many others I have written this > piece of code several times and invariably named it > 'concat_with_separator' - a mouthful but it conveys (at least to me) > what exactly the function is doing. Since 1997 (if not before) the SML Basis Library has included fun concatWith _ [] = "" | concatWith separator (string :: strings) = concat (string :: map (fn x => separator^x) strings) (*) I am using that as a specification. It would be a horrible implementation. Since 2015 that library includes fun concatWithMap separator f data = concatWith separator (map f data) except that the point of concatWithMap is to fuse the calculations. concat{,With}{,Map} are available on all vector-like types in the SML Basis. Would those names do? From ok@REDACTED Wed Mar 9 00:45:11 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 9 Mar 2016 12:45:11 +1300 Subject: [erlang-questions] How does Erlang schedule process? In-Reply-To: References: Message-ID: <1c5aa4b0-822d-fa6c-f3ee-534ef9885c2e@cs.otago.ac.nz> On 9/03/16 6:12 am, Jung Kim wrote: > I am very new to Erlang, and just read that Erlang style process is > done with something similar to coroutine + preemptive scheduling[1]. That could be said of ANY programming language that offers threads, including C, C++, Java, &c. It's really not very helpful, because more people these days know about threads than know about coroutines. In fact when I *want* coroutines in most programming languages these days, I use threads instead. > So I am interested to know how Erlang schedule process without the > user inserting code like coroutine.yield() in the function created. Do you understand how UNIX schedules processes without the user calling the POSIX sched_yield() function? In effect, the operating system uses timer interrupts to forcibly insert sched_yield() calls, or switches to doing something else whenever a (UNIX) process needs to wait. Those are the two basic techniques used by any process/thread implementation: if a process/thread starts an operation and needs to wait for completion, that process/thread is put into a wait state and something that's ready to run is executed in its place, OR a 'yield' is inserted when a timer goes off or an event counter (like Erlang's "reduction count") reaches a limit. > Is there any document detail about this (I find some discussion like > [2] but that doesn't give me a clear blueprint)? Or which part of > source code should I look into? How much detail do you need? Do you understand how this kind of thing works in UNIX or Windows? If you don't, try reading a good book about operating systems first. From vinoski@REDACTED Wed Mar 9 00:47:09 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 8 Mar 2016 18:47:09 -0500 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: On Tue, Mar 8, 2016 at 4:41 PM, Vlad Dumitrescu wrote: > > On Tue, Mar 8, 2016 at 10:37 PM, Steve Vinoski wrote: > >> On Tue, Mar 8, 2016 at 4:22 PM, Vlad Dumitrescu >> wrote: >> >>> Re yaws docs: Could the contents of the article Karolis pointed to be >>> included in the docs? Or at least a link to it... >>> /Vlad >>> >> >> There are links to all known Yaws articles and books here: >> >> http://yaws.hyber.org/articles.yaws >> > > Oh, yes, missed that... For the InfoQ article, I wouldn't have guessed > it's about REST and not "regular" usage. > I wrote it in 2008, 8 years after Roy Fielding wrote his thesis that defines REST. In the intervening ~16 years, REST has not changed, but the number of people who misuse the term and apply it incorrectly has grown tremendously. No web server can make your application RESTful. Some can help, sure, but if you want to write a system that's actually RESTful, as defined by Roy's thesis, then you need to read and understand that thesis. Even implementing Alan Dean's HTTP flowchart as Webmachine does isn't enough to make your application RESTful. There is no magic. My article simply shows how to use Yaws to help with organizing your code and show you how to get at the information you need to help make your application RESTful. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Mar 9 00:56:46 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 9 Mar 2016 00:56:46 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: And for the lazy: https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf It is definitely worth a read if you want to understand what he was trying to do in the first place. We are getting there, albeit too slowly for it to be fun. On Wed, Mar 9, 2016 at 12:47 AM, Steve Vinoski wrote: > > > On Tue, Mar 8, 2016 at 4:41 PM, Vlad Dumitrescu > wrote: > >> >> On Tue, Mar 8, 2016 at 10:37 PM, Steve Vinoski wrote: >> >>> On Tue, Mar 8, 2016 at 4:22 PM, Vlad Dumitrescu >>> wrote: >>> >>>> Re yaws docs: Could the contents of the article Karolis pointed to be >>>> included in the docs? Or at least a link to it... >>>> /Vlad >>>> >>> >>> There are links to all known Yaws articles and books here: >>> >>> http://yaws.hyber.org/articles.yaws >>> >> >> Oh, yes, missed that... For the InfoQ article, I wouldn't have guessed >> it's about REST and not "regular" usage. >> > > I wrote it in 2008, 8 years after Roy Fielding wrote his thesis that > defines REST. In the intervening ~16 years, REST has not changed, but the > number of people who misuse the term and apply it incorrectly has grown > tremendously. No web server can make your application RESTful. Some can > help, sure, but if you want to write a system that's actually RESTful, as > defined by Roy's thesis, then you need to read and understand that thesis. > Even implementing Alan Dean's HTTP flowchart as Webmachine does isn't > enough to make your application RESTful. There is no magic. My article > simply shows how to use Yaws to help with organizing your code and show you > how to get at the information you need to help make your application > RESTful. > > --steve > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Wed Mar 9 01:05:21 2016 From: raould@REDACTED (Raoul Duke) Date: Tue, 8 Mar 2016 16:05:21 -0800 Subject: [erlang-questions] How does Erlang schedule process? In-Reply-To: <1c5aa4b0-822d-fa6c-f3ee-534ef9885c2e@cs.otago.ac.nz> References: <1c5aa4b0-822d-fa6c-f3ee-534ef9885c2e@cs.otago.ac.nz> Message-ID: https://duckduckgo.com/?q=+Erlang%27s+%22reduction+count%22+scheduler&t=ffsb From vladdu55@REDACTED Wed Mar 9 08:03:12 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 9 Mar 2016 08:03:12 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: On Wed, Mar 9, 2016 at 12:47 AM, Steve Vinoski wrote: > I wrote it in 2008, 8 years after Roy Fielding wrote his thesis that > defines REST. In the intervening ~16 years, REST has not changed, but the > number of people who misuse the term and apply it incorrectly has grown > tremendously. No web server can make your application RESTful. Some can > help, sure, but if you want to write a system that's actually RESTful, as > defined by Roy's thesis, then you need to read and understand that thesis. > Even implementing Alan Dean's HTTP flowchart as Webmachine does isn't > enough to make your application RESTful. There is no magic. My article > simply shows how to use Yaws to help with organizing your code and show you > how to get at the information you need to help make your application > RESTful. All true, of course. I didn't ask for magic (well, at least not in this forum :-) ), but for a server that makes it easy to implement REST or REST-like functionality. And so it looks like Yaws does that, even if it wasn't advertised like it is for Cowboy, for example. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Mar 9 08:48:36 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 09 Mar 2016 07:48:36 +0000 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice In-Reply-To: References: Message-ID: actually I was wondering if enacl is used in somewhat production ready? The usag of dirty schedulers makes me nervous :) - benoit On Thu, Feb 4, 2016 at 5:52 PM Mark Steele wrote: > Hi Jesper, > > I'll take a look at your library, looks like it might make have some good > enhancements over the bindings (especially the use of dirty schedulers) I'm > currently using (which is erlang-nacl from Tony Garnock-Jones). > > Cheers, > > Mark Steele > CISSP, GPEN, GCIA, CSM > mark@REDACTED > > LinkedIn: https://ca.linkedin.com/in/markrsteele > Github: https://github.com/marksteele > Personal: http://www.control-alt-del.org > > On Thu, Feb 4, 2016 at 7:08 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> >> On Thu, Feb 4, 2016 at 5:38 AM, Mark Steele >> wrote: >> >>> >>> - libsodium (via NIF bindings) >>> >>> >> I wonder if your approach is much different from >> https://github.com/jlouis/enacl in any way? We may win by coalescing the >> work there. >> >> >> -- >> J. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Mar 9 10:14:43 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 9 Mar 2016 10:14:43 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <1625329.6E7TUCNP1e@burrito> Message-ID: <56DFE983.90801@ninenines.eu> On 03/09/2016 12:47 AM, Steve Vinoski wrote: > I wrote it in 2008, 8 years after Roy Fielding wrote his thesis that > defines REST. In the intervening ~16 years, REST has not changed, but > the number of people who misuse the term and apply it incorrectly has > grown tremendously. No web server can make your application RESTful. > Some can help, sure, but if you want to write a system that's actually > RESTful, as defined by Roy's thesis, then you need to read and > understand that thesis. Even implementing Alan Dean's HTTP flowchart as > Webmachine does isn't enough to make your application RESTful. There is > no magic. My article simply shows how to use Yaws to help with > organizing your code and show you how to get at the information you need > to help make your application RESTful. His thesis is not very interesting in the context of writing REST APIs, which is what most people do. Most of REST is provided by simply using HTTP, even if you don't use all of it, because REST is an architectural construct. On top of basic HTTP the key points are: * Stateless (the client sends everything needed to complete the request) * Caching (boils down to defining what responses can be cached) * Uniform interface The latter involves the concept of resources and representations, which involves the content-* and accept-* headers. This is where Webmachine/Cowboy will help you the most. The holy grail is HATEOAS, and that part is what most of everyone doesn't do today. Interaction with a server is driven by hypermedia, or in layman terms, client follows hyperlinks to access other resources. This involves media types that have a concept of hyperlinks, though. For example, HTML has a few elements that define them and that's how browsers or bots navigate websites. Any API that sends you application/json does not follow HATEOAS (even if they define a convention to identify links in the documentation). Conventions are only applicable to one site so if this API gives you a link to another site your client won't know what to do anymore. Fielding has refined the REST definition a few times since his thesis, and in particular HATEOAS, so I'd encourage reading his most recent writings if you are interested in designing REST APIs. You might also notice that while HATEOAS is useful, it kind of leaves you wanting more, in which case you might want to look at the works around the Semantic Web. But be careful not to step into madness. :-) -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From chandrashekhar.mullaparthi@REDACTED Wed Mar 9 11:27:05 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 9 Mar 2016 10:27:05 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <92d3e03a-cc17-7717-e58c-76968d76e217@cs.otago.ac.nz> References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> <92d3e03a-cc17-7717-e58c-76968d76e217@cs.otago.ac.nz> Message-ID: On 8 March 2016 at 23:18, Richard A. O'Keefe wrote: > On 8/03/16 8:24 pm, Chandru wrote: > >> On 7 March 2016 at 16:04, Lo?c Hoguin > essen@REDACTED>> wrote: >> >> >> I am completely with you on that. If the function was called >> intercalculate I'd never find it and would continue writing my own. >> >> >> I second this. I'm part of the vast number of unwashed masses who've >> never heard of the term intercalculate, >> > > Please, if you are going to pour scorn on a word, at least COPY > the thing correctly. The proposal is not "intercalCULate" but > "intercalate", > and it's a real English word with a history going back at least 400 years > in English, 500 years in French, and 2500 years in Latin. > Okay, sorry, my mistake. > > I think it's a bad name not because nobody knows what it means (I'd expect > any skilled native speaker of English to have met it several times) but > because > it means to insert ONE (unusual) thing into a sequence of (ordinary) > things, > as in one leap day or one leap month into a year. I looked up the definition and while I couldn't find one which restricts the insertion to ONE thing, it certainly doesn't say it is the repeated insertion into a sequence, so clearly intercalate is the wrong choice here. > > and if I came across it in a developer's code would think that they were >> being a bit too clever. Like many others I have written this piece of code >> several times and invariably named it 'concat_with_separator' - a mouthful >> but it conveys (at least to me) what exactly the function is doing. >> > Since 1997 (if not before) the SML Basis Library has included > > fun concatWith _ [] = "" > | concatWith separator (string :: strings) = > concat (string :: map (fn x => separator^x) strings) > > (*) I am using that as a specification. It would be a horrible > implementation. > > I personally would be happy with concat_with or concat_with_separator. I find the proposal for the use of 'join' wrong. In plain English, joining things doesn't usually involve inserting things in between. regards, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Wed Mar 9 11:31:09 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Wed, 9 Mar 2016 12:31:09 +0200 Subject: [erlang-questions] some help with hackney and multipart Message-ID: Hello, I'm trying to send an email via mailgun.com using the hackney and I have some issues sending attachments (which requires multipart). https://documentation.mailgun.com/api-sending.html#sending Basically my interest fields are: from to subject text attachment File attachment. You can post multiple attachment values. Important: You must use multipart/form-data encoding when sending attachments. I tried the following: PayloadBase =[ {<<"from">>, From}, {<<"to">>, To}, {<<"subject">>, Subject}, {<<"text">>, TextBody}, {<<"html">>, HtmlBody} ], Payload = case Attachment of null -> {form, PayloadBase}; _-> {multipart, PayloadBase ++ [{file, Attachment}]} end, But for some reason the attachment is not sent.. Everything else works as expected. I don't see how I can set the filed name to "attachment" as required by mailgun .. at this this is what I suspect beeing wrong Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Wed Mar 9 11:54:18 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Wed, 9 Mar 2016 12:54:18 +0200 Subject: [erlang-questions] zlib driver in erlang seems very slow Message-ID: Hello guys, While investigating some benchmarks with zlib on erlang I suspected that the elrang driver might be too slow. Initially I wrote a small c++ app linking with zlib doing almost the same stuff like my erlang testing script and I found a big gap in performance between them. Initial thought was that the difference might come from the overhead added by erlang itself. (converting from elrang terms into c++ data and vice versa) Then I started creating my own nif library for zlib and I found that comparing with erlang driver it's around 30 % faster for doing the same job. After other investigations on zlib I found some interesting articles regarding intel and cloudflare patches and also zlibng that performs much better than the original zlib. After I linked my NIF with this zlib libraries I got a gain of around 69 % compared with erlang driver. The testing script is here: https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl Basically reads a file line by line and compress it N times. For example: benchmark:run(ezlib,"file path here", 200, 6, 10, 1). benchmark:run(erlang,"file path here", 200, 6, 10, 1). This is compressing the file 200 times with a compression level of 6 window size 10 and memory level 1. First one using my library second one using erlang driver. Some benchmark results you can see on the https://github.com/silviucpp/ezlib I can understand that the erlang driver api is generic and allows you to customize a lot if things and mine it's not and it's built around my need to use it with ejabberd but still I'm almost sure if I change the interface the performance difference will be the same. Maybe my benchmark is not realistic or I'm missing something. Feedback it's welcome. Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 9 11:55:21 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 9 Mar 2016 11:55:21 +0100 Subject: [erlang-questions] some help with hackney and multipart In-Reply-To: References: Message-ID: Hi, Try compare what is sent in body of request made by example with your one sent by hackney curl -s --user 'api:YOUR_API_KEY' \ https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \ -F from='Excited User ' \ -F to='foo@REDACTED' \ -F cc='bar@REDACTED' \ -F bcc='baz@REDACTED' \ -F subject='Hello' \ -F text='Testing some Mailgun awesomness!' \ --form-string html='HTML version of the body' \ -F attachment=@files/cartman.jpg \ -F attachment=@files/cartman.png Your {multipart, PayloadBase ++ [{file, Attachment}]} doesn't make much sense to me. Try something like {multipart, [{form, PayloadBase},{file, Attachment}]} (untested). Hynek On Wed, Mar 9, 2016 at 11:31 AM, Caragea Silviu wrote: > > > Hello, > > I'm trying to send an email via mailgun.com using the hackney and I have > some issues sending attachments (which requires multipart). > > https://documentation.mailgun.com/api-sending.html#sending > > Basically my interest fields are: > > from > to > subject > text > attachment File attachment. You can post multiple attachment values. > Important: You must use multipart/form-data encoding when sending > attachments. > > I tried the following: > > PayloadBase =[ > {<<"from">>, From}, > {<<"to">>, To}, > {<<"subject">>, Subject}, > {<<"text">>, TextBody}, > {<<"html">>, HtmlBody} > ], > > Payload = case Attachment of > null -> > {form, PayloadBase}; > _-> > {multipart, PayloadBase ++ [{file, Attachment}]} > end, > > But for some reason the attachment is not sent.. Everything else works as > expected. > I don't see how I can set the filed name to "attachment" as required by > mailgun .. at this this is what I suspect beeing wrong > > Silviu > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Wed Mar 9 12:02:35 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Wed, 9 Mar 2016 12:02:35 +0100 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: References: Message-ID: <96B7ACAF-C8B9-40C5-8D74-F36ED27A2202@gmail.com> If you care about compression speed zlib is a terrible choice anyway. Also in your NIF you seem to be doing the entire compression on a scheduler thread all in one go. This is very bad. NIF calls should not take more than ~1ms. Sergej > On 09 Mar 2016, at 11:54, Caragea Silviu wrote: > > Hello guys, > > While investigating some benchmarks with zlib on erlang I suspected that the elrang driver might be too slow. > Initially I wrote a small c++ app linking with zlib doing almost the same stuff like my erlang testing script and I found a big gap in performance between them. > Initial thought was that the difference might come from the overhead added by erlang itself. (converting from elrang terms into c++ data and vice versa) > > Then I started creating my own nif library for zlib and I found that comparing with erlang driver it's around 30 % faster for doing the same job. > After other investigations on zlib I found some interesting articles regarding intel and cloudflare patches and also zlibng that performs much better than > the original zlib. After I linked my NIF with this zlib libraries I got a gain of around 69 % compared with erlang driver. > > The testing script is here: https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl > > Basically reads a file line by line and compress it N times. > > For example: > > benchmark:run(ezlib,"file path here", 200, 6, 10, 1). > benchmark:run(erlang,"file path here", 200, 6, 10, 1). > > This is compressing the file 200 times with a compression level of 6 window size 10 and memory level 1. First one using my library second one using erlang driver. > > Some benchmark results you can see on the https://github.com/silviucpp/ezlib > > I can understand that the erlang driver api is generic and allows you to customize a lot if things and mine it's not and it's built > around my need to use it with ejabberd but still I'm almost sure if I change the interface the performance difference will be the same. > > Maybe my benchmark is not realistic or I'm missing something. > Feedback it's welcome. > > Silviu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Wed Mar 9 12:04:15 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Wed, 9 Mar 2016 13:04:15 +0200 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: <96B7ACAF-C8B9-40C5-8D74-F36ED27A2202@gmail.com> References: <96B7ACAF-C8B9-40C5-8D74-F36ED27A2202@gmail.com> Message-ID: Yes because I'm compressing small chunks that anyway takes less than 1-2 ms Silviu On Wed, Mar 9, 2016 at 1:02 PM, Sergej Jure?ko wrote: > If you care about compression speed zlib is a terrible choice anyway. Also > in your NIF you seem to be doing the entire compression on a scheduler > thread all in one go. This is very bad. NIF calls should not take more than > ~1ms. > > Sergej > > On 09 Mar 2016, at 11:54, Caragea Silviu wrote: > > Hello guys, > > While investigating some benchmarks with zlib on erlang I suspected that > the elrang driver might be too slow. > Initially I wrote a small c++ app linking with zlib doing almost the same > stuff like my erlang testing script and I found a big gap in performance > between them. > Initial thought was that the difference might come from the overhead added > by erlang itself. (converting from elrang terms into c++ data and vice > versa) > > Then I started creating my own nif library for zlib and I found that > comparing with erlang driver it's around 30 % faster for doing the same job. > After other investigations on zlib I found some interesting articles > regarding intel and cloudflare patches and also zlibng that performs much > better than > the original zlib. After I linked my NIF with this zlib libraries I got a > gain of around 69 % compared with erlang driver. > > The testing script is here: > https://github.com/silviucpp/ezlib/blob/master/testing/benchmark.erl > > Basically reads a file line by line and compress it N times. > > For example: > > benchmark:run(ezlib,"file path here", 200, 6, 10, 1). > benchmark:run(erlang,"file path here", 200, 6, 10, 1). > > This is compressing the file 200 times with a compression level of 6 > window size 10 and memory level 1. First one using my library second one > using erlang driver. > > Some benchmark results you can see on the > https://github.com/silviucpp/ezlib > > I can understand that the erlang driver api is generic and allows you to > customize a lot if things and mine it's not and it's built > around my need to use it with ejabberd but still I'm almost sure if I > change the interface the performance difference will be the same. > > Maybe my benchmark is not realistic or I'm missing something. > Feedback it's welcome. > > Silviu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Mar 9 12:08:37 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 9 Mar 2016 12:08:37 +0100 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: References: Message-ID: <56E00435.4060909@ninenines.eu> On 03/09/2016 11:54 AM, Caragea Silviu wrote: > Some benchmark results you can see on the https://github.com/silviucpp/ezlib > > I can understand that the erlang driver api is generic and allows you to > customize a lot if things and mine it's not and it's built > around my need to use it with ejabberd but still I'm almost sure if I > change the interface the performance difference will be the same. > > Maybe my benchmark is not realistic or I'm missing something. > Feedback it's welcome. I have recently come to realize the same thing as you. A few discussions I had with users pointed out that the performance of the Erlang zlib might be even worse when used concurrently. Have you done some measurements on that? Also you will probably need an inflateChunk equivalent if you are doing anything public facing. I don't think it changes much performance-wise though. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From silviu.cpp@REDACTED Wed Mar 9 12:15:55 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Wed, 9 Mar 2016 13:15:55 +0200 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: <56E00435.4060909@ninenines.eu> References: <56E00435.4060909@ninenines.eu> Message-ID: Hello Loic, I didn't tested concurrently but I will do it shouldn't be very difficult to change the testing script. Even if from what I know zlib doesn't have any mutex or something to lock the access to resources inside. Regarding inflateChunk I might add it, right now its only an interface that makes integration in the projects I'm using faster. Silviu On Wed, Mar 9, 2016 at 1:08 PM, Lo?c Hoguin wrote: > On 03/09/2016 11:54 AM, Caragea Silviu wrote: > >> Some benchmark results you can see on the >> https://github.com/silviucpp/ezlib >> >> I can understand that the erlang driver api is generic and allows you to >> customize a lot if things and mine it's not and it's built >> around my need to use it with ejabberd but still I'm almost sure if I >> change the interface the performance difference will be the same. >> >> Maybe my benchmark is not realistic or I'm missing something. >> Feedback it's welcome. >> > > I have recently come to realize the same thing as you. A few discussions I > had with users pointed out that the performance of the Erlang zlib might be > even worse when used concurrently. Have you done some measurements on that? > > Also you will probably need an inflateChunk equivalent if you are doing > anything public facing. I don't think it changes much performance-wise > though. > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Wed Mar 9 12:37:50 2016 From: grahamrhay@REDACTED (Graham Hay) Date: Wed, 9 Mar 2016 11:37:50 +0000 Subject: [erlang-questions] REST server In-Reply-To: <56DFE983.90801@ninenines.eu> References: <1625329.6E7TUCNP1e@burrito> <56DFE983.90801@ninenines.eu> Message-ID: > The holy grail is HATEOAS, and that part is what most of everyone doesn't do > today. Interaction with a server is driven by hypermedia, or in layman > terms, client follows hyperlinks to access other resources. >From the horse's mouth: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven From silviu.cpp@REDACTED Wed Mar 9 13:28:34 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Wed, 9 Mar 2016 14:28:34 +0200 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: References: <56E00435.4060909@ninenines.eu> Message-ID: Yes so your suggestion to test concurrently was very good. I updated the testing script and running it on my machine with 1000 concurrent processes It's crashing the VM using erlang driver . I suspect because out of memory. My machine has only 16 GB and I see at some point 48 GB in the beam process. Anyway before crashing all the time I see cpu consumption being 100 % maximum on beam process. The same test using my library complete fine. also the cpu I see cpu staying at 500 % - 600 % and sometimes 700% which means it's using almost all the cores I have. Memory wise is not using more than : 12 GB Something is strange. You can review my testing program maybe I'm wrong somewhere. Silviu On Wed, Mar 9, 2016 at 1:15 PM, Caragea Silviu wrote: > Hello Loic, > > I didn't tested concurrently but I will do it shouldn't be very difficult > to change the testing script. Even if from what I know zlib doesn't have > any mutex or something to lock the access to resources inside. > Regarding inflateChunk I might add it, right now its only an interface > that makes integration in the projects I'm using faster. > > Silviu > > On Wed, Mar 9, 2016 at 1:08 PM, Lo?c Hoguin wrote: > >> On 03/09/2016 11:54 AM, Caragea Silviu wrote: >> >>> Some benchmark results you can see on the >>> https://github.com/silviucpp/ezlib >>> >>> I can understand that the erlang driver api is generic and allows you to >>> customize a lot if things and mine it's not and it's built >>> around my need to use it with ejabberd but still I'm almost sure if I >>> change the interface the performance difference will be the same. >>> >>> Maybe my benchmark is not realistic or I'm missing something. >>> Feedback it's welcome. >>> >> >> I have recently come to realize the same thing as you. A few discussions >> I had with users pointed out that the performance of the Erlang zlib might >> be even worse when used concurrently. Have you done some measurements on >> that? >> >> Also you will probably need an inflateChunk equivalent if you are doing >> anything public facing. I don't think it changes much performance-wise >> though. >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Mar 9 13:39:58 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 9 Mar 2016 13:39:58 +0100 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: References: <56E00435.4060909@ninenines.eu> Message-ID: <56E0199E.9010604@ninenines.eu> Erlang/zlib using only one core fits the discussion I had about it. Not sure about the OOM crash though. By the way I don't know if there's anything out there about it but Joe mentioned a lock in zlib here: https://joearms.github.io/2013/03/28/solving-the-wrong-problem.html - Little details, and I don't think they open sourced their zlib implementation though. On 03/09/2016 01:28 PM, Caragea Silviu wrote: > Yes so your suggestion to test concurrently was very good. I updated the > testing script and running it on my machine with 1000 concurrent > processes It's crashing the VM using erlang driver . I suspect because > out of memory. My machine has only 16 GB and I see at some point 48 GB > in the beam process. > > Anyway before crashing all the time I see cpu consumption being 100 % > maximum on beam process. > > The same test using my library complete fine. also the cpu I see cpu > staying at 500 % - 600 % and sometimes 700% which means it's using > almost all the cores I have. Memory wise is not using more than : 12 GB > > Something is strange. You can review my testing program maybe I'm wrong > somewhere. > > Silviu > > > > > On Wed, Mar 9, 2016 at 1:15 PM, Caragea Silviu > wrote: > > Hello Loic, > > I didn't tested concurrently but I will do it shouldn't be very > difficult to change the testing script. Even if from what I know > zlib doesn't have any mutex or something to lock the access to > resources inside. > Regarding inflateChunk I might add it, right now its only an > interface that makes integration in the projects I'm using faster. > > Silviu > > On Wed, Mar 9, 2016 at 1:08 PM, Lo?c Hoguin > wrote: > > On 03/09/2016 11:54 AM, Caragea Silviu wrote: > > Some benchmark results you can see on the > https://github.com/silviucpp/ezlib > > I can understand that the erlang driver api is generic and > allows you to > customize a lot if things and mine it's not and it's built > around my need to use it with ejabberd but still I'm almost > sure if I > change the interface the performance difference will be the > same. > > Maybe my benchmark is not realistic or I'm missing something. > Feedback it's welcome. > > > I have recently come to realize the same thing as you. A few > discussions I had with users pointed out that the performance of > the Erlang zlib might be even worse when used concurrently. Have > you done some measurements on that? > > Also you will probably need an inflateChunk equivalent if you > are doing anything public facing. I don't think it changes much > performance-wise though. > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > > > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From mark@REDACTED Wed Mar 9 15:23:17 2016 From: mark@REDACTED (Mark Steele) Date: Wed, 9 Mar 2016 09:23:17 -0500 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice In-Reply-To: References: Message-ID: I've been testing a branch that uses enacl. Haven't had much time to spend in this lately, but it appears to work fine. Cheers Mark On Wed, Mar 9, 2016 at 2:48 AM, Benoit Chesneau wrote: > actually I was wondering if enacl is used in somewhat production ready? > The usag of dirty schedulers makes me nervous :) > > - benoit > > On Thu, Feb 4, 2016 at 5:52 PM Mark Steele > wrote: > >> Hi Jesper, >> >> I'll take a look at your library, looks like it might make have some good >> enhancements over the bindings (especially the use of dirty schedulers) I'm >> currently using (which is erlang-nacl from Tony Garnock-Jones). >> >> Cheers, >> >> Mark Steele >> CISSP, GPEN, GCIA, CSM >> mark@REDACTED >> >> LinkedIn: https://ca.linkedin.com/in/markrsteele >> Github: https://github.com/marksteele >> Personal: http://www.control-alt-del.org >> >> On Thu, Feb 4, 2016 at 7:08 AM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> >>> On Thu, Feb 4, 2016 at 5:38 AM, Mark Steele >>> wrote: >>> >>>> >>>> - libsodium (via NIF bindings) >>>> >>>> >>> I wonder if your approach is much different from >>> https://github.com/jlouis/enacl in any way? We may win by coalescing >>> the work there. >>> >>> >>> -- >>> J. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Mar 9 15:28:05 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 9 Mar 2016 15:28:05 +0100 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice In-Reply-To: References: Message-ID: On Wed, Mar 9, 2016 at 8:48 AM, Benoit Chesneau wrote: > actually I was wondering if enacl is used in somewhat production ready? > The usag of dirty schedulers makes me nervous :) The code itself is quite production ready and has a full quickcheck model. I know of users who run it in production as well. As for the current status of Dirty Schedulers, I hope they will be default-included in the Erlang build soon. As for *their* stability, I send the baton to Steve Vinoski and the OTP team. If you use another library without dirty schedulers, you are solving eventual unknown problems in the DS space, but opening yourself to the problems of scheduler collapse, since many of the libsodium calls cannot be cooperatively broken into a continuation chain. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Wed Mar 9 15:45:23 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Wed, 9 Mar 2016 08:45:23 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? Message-ID: # Quick Background I inherited an Erlang system that's a fairly typical HTTP, almost-ReST API. The nature of the API and what it represents made Erlang a good choice for its concurrency aspects. As for the reliability aspects, we rely totally on a typical HTTP/S load balancer setup for HA. We do rolling restarts behind the LB and not hot code upgrades as enabled by Erlang because of the complexity of Erlang upgrades. # The Question Why should we run embedded mode in production? It seems that's the unspoken law for production systems, but why? Is there some performance overhead running in interactive mode? I've read about Code Loading Strategy in the System Principles User's Guide: http://erlang.org/doc/system_principles/system_principles.html#id56789 I've also read about the mode differences in the code server: http://erlang.org/doc/man/code.html I don't see anything in any of the stdlib/OTP docs about *why* you would choose embedded vs interactive mode. I'm leaning heavily toward using interactive mode everywhere, dev through production, because it's so much simpler and easy to understand. Someone please talk me out of it if that's a terribly Bad Idea. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Tue Mar 8 14:11:44 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Tue, 08 Mar 2016 14:11:44 +0100 Subject: [erlang-questions] Webmachine content_types_accepted w/ POST In-Reply-To: <56DEC0FA.4030600@ninenines.eu> References: <56deb26f.c6ab8c0a.5e8c0.3ae9@mx.google.com> <56DEC0FA.4030600@ninenines.eu> Message-ID: <56e04f4b.a810700a.5a9bf.6d07@mx.google.com> Lo?c Hoguin writes: > [ text/plain ] > On 03/08/2016 12:07 PM, Torben Hoffmann wrote: >> Hi, >> >> I have a URI where I accept a POST. >> >> It has to support one or multiple entities for creation as JSON in the >> body. >> So post_is_create/2 should return false. >> Hence I have to implement process_post/2 to deal with the POST. >> So far, so good. >> >> But I have to write >> >> content_types_accepted(RD, Ctx) -> >> {[{"application/json", bogus_handler_name}], RD, Ctx}. >> >> for the JSON body to be accepted. > > Cowboy REST has the same behavior. I recommend putting undefined > personally. I suppose '' would also work. > Nice to know. undefined seems like a good value. Cheers, Torben -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From marc@REDACTED Wed Mar 9 17:36:28 2016 From: marc@REDACTED (Marc Worrell) Date: Wed, 9 Mar 2016 17:36:28 +0100 Subject: [erlang-questions] [ANN] Zotonic 0.14 released Message-ID: We released Zotonic version 0.14. Zotonic is the Erlang Content Management System and framework. See http://zotonic.com This release is the first following our new numbering system. The next release will be 0.15 on Monday, April 4. In 0.14 we had some major and minor changes. First, the branch we are working on is now called "0.x?. The ?0.x? is also the stable branch. The master branch is where our 1.x development work happens and should be considered unstable. Other changes in this release: * Added data model notification. * Added managed ACL rules. * Added zotonic wait and zotonic rpc commands. * Added new user category configuration parameter. * Fixed redirect to login when user has no permissions by showing 403 page with login form. * Fixed several import and export bugs. * Fixed Dutch translations for mod_acl_user_groups. * Fixed status page login button. * Improved documentation. * Improved m.req model. See the full release notes at http://zotonic.com/docs/latest/developer-guide/releasenotes/rel_0.14.0.html Cheers, The Zotonic core team. -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Wed Mar 9 18:25:54 2016 From: grahamrhay@REDACTED (Graham Hay) Date: Wed, 9 Mar 2016 17:25:54 +0000 Subject: [erlang-questions] zlib driver in erlang seems very slow In-Reply-To: <56E0199E.9010604@ninenines.eu> References: <56E00435.4060909@ninenines.eu> <56E0199E.9010604@ninenines.eu> Message-ID: Sadly Concurix has "pivoted" to instrumenting nodejs apps. On 9 March 2016 at 12:39, Lo?c Hoguin wrote: > Erlang/zlib using only one core fits the discussion I had about it. Not sure > about the OOM crash though. > > By the way I don't know if there's anything out there about it but Joe > mentioned a lock in zlib here: > https://joearms.github.io/2013/03/28/solving-the-wrong-problem.html - Little > details, and I don't think they open sourced their zlib implementation > though. > > On 03/09/2016 01:28 PM, Caragea Silviu wrote: >> >> Yes so your suggestion to test concurrently was very good. I updated the >> testing script and running it on my machine with 1000 concurrent >> processes It's crashing the VM using erlang driver . I suspect because >> out of memory. My machine has only 16 GB and I see at some point 48 GB >> in the beam process. >> >> Anyway before crashing all the time I see cpu consumption being 100 % >> maximum on beam process. >> >> The same test using my library complete fine. also the cpu I see cpu >> staying at 500 % - 600 % and sometimes 700% which means it's using >> almost all the cores I have. Memory wise is not using more than : 12 GB >> >> Something is strange. You can review my testing program maybe I'm wrong >> somewhere. >> >> Silviu >> >> >> >> >> On Wed, Mar 9, 2016 at 1:15 PM, Caragea Silviu > > wrote: >> >> Hello Loic, >> >> I didn't tested concurrently but I will do it shouldn't be very >> difficult to change the testing script. Even if from what I know >> zlib doesn't have any mutex or something to lock the access to >> resources inside. >> Regarding inflateChunk I might add it, right now its only an >> interface that makes integration in the projects I'm using faster. >> >> Silviu >> >> On Wed, Mar 9, 2016 at 1:08 PM, Lo?c Hoguin > > wrote: >> >> On 03/09/2016 11:54 AM, Caragea Silviu wrote: >> >> Some benchmark results you can see on the >> https://github.com/silviucpp/ezlib >> >> I can understand that the erlang driver api is generic and >> allows you to >> customize a lot if things and mine it's not and it's built >> around my need to use it with ejabberd but still I'm almost >> sure if I >> change the interface the performance difference will be the >> same. >> >> Maybe my benchmark is not realistic or I'm missing something. >> Feedback it's welcome. >> >> >> I have recently come to realize the same thing as you. A few >> discussions I had with users pointed out that the performance of >> the Erlang zlib might be even worse when used concurrently. Have >> you done some measurements on that? >> >> Also you will probably need an inflateChunk equivalent if you >> are doing anything public facing. I don't think it changes much >> performance-wise though. >> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> >> >> > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t@REDACTED Wed Mar 9 19:21:44 2016 From: t@REDACTED (Tristan Sloughter) Date: Wed, 09 Mar 2016 12:21:44 -0600 Subject: [erlang-questions] [ANN] Rebar3 first stable release 3.0.0 Message-ID: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> We are happy to announce the first stable release of rebar3 with version 3.0.0. A bit over a year ago, we had set out to make rebar great again. We believe we have done it, and now have a solid core on which to build more tools and grow a better community. Now, this doesn't mean there are no things to improve or that the code base is bug-free. It does however mean that we are now committed to backwards compatibility as we move forward with rebar3's next improvements. If you have never tried rebar3, the big item names since rebar 2.x are: * repeatable builds with a big commitment to OTP principles * commands that know their own dependencies * support for profiles, allowing contextual configuration of applications (for example, test dependencies can be kept apart from regular deps) * documentation is seen as a feature * support for packages through hex.pm * new plugin interface and mechanisms, with command namespaces * mechanisms to override dependencies' configurations * switching releases to relx * and a lot of improvements to existing commands such as templates, dialyzer, shell, and so on. You will find documentation and how to get started with rebar3 on the website: http://www.rebar3.org/docs/getting-started For package publishing checkout the documentation on hex.pm: https://hex.pm/docs/rebar3_publish If you feel like getting started, we have had numerous posts about the new features and user experiences: * http://ferd.ca/dev/rebar3-shell.html * http://blog.erlware.org/rebar3-features-part-1-local-install-and-upgrade/ * http://blog.erlware.org/rebar3-features-part-2-dependency-tree/ * http://blog.erlware.org/rebar3-features-part-3-overrides/ * http://blog.erlware.org/rebar3-features-part-4-profiles/ * http://blog.erlware.org/rebar3-features-part-5-dependency-branch-handling/ * http://blog.erlware.org/rebar3-features-part-6-_checkouts-2/ * http://blog.erlware.org/rebar3-hex-plugin/ Heinz Gies wrote about his experience migrating to rebar3 form rebar2 at http://blog.licenser.net/blog/2015/07/10/migrating-to-rebar3/ The How I Start Erlang guide was updated to use rebar3: https://howistart.org/posts/erlang/1 And a guide about managing package has been written by Bruce Yinhe: https://medium.com/@brucify/using-rebar3-to-manage-erlang-packages-282f78adff1e#.ay10goc0z Lastly, for those who have already adopted the tool, here is what has changed since beta-4: * rebar3/902: Less color[1] * rebar3/921: Fix error reports on missing include paths[2] * rebar3/922: warn on incorrectly specified test options in rebar.config[3] * rebar3/923: support temporary cdn change with HEX_CDN os var[4] * rebar3/924: only add package list of versions to registry if it has the right build tool support[5] * rebar3/925: upgrade eunit_formatters for OTP 18 support[6] * rebar3/927: upgrade eunit_formatters to 0.3.1 for OTP18 bug fix[7] * rebar3/928: Ct output improvements[8] * rebar3/930: Handle force flags in leading position[9] * rebar3/934: don't add a provider if the same namespace and name already exists[10] * rebar3/935: Default to no eunit formatter if verbose specified[11] * rebar3/941: preserve attributes when copying files in rebar_utils:cp_r for unix[12] * rebar3/942: special handling of relx configs in profiles[13] * rebar3/943: remove backward_compat entry from travis s3[14] * rebar3/945: auto-update the registry if a pkg isn't found, fail if it still isn't found[15] * rebar3/948: Fix a small bug in the MIB compiler when building dependencies[16] * rebar3/949: fetch eunit_formatters config not from the command args but from the config[17] * rebar3/963: Tup merge tests[18] * rebar3/965: fix tupple merging for realsies.[19] * rebar3/966: allow ct suites to be specified at root of project (or root of app)[20] * rebar3/967: symlink mib hrl output in apps `include' directories[21] * rebar3/970: upgrade certifi to latest release[22] * rebar3/982: Allow bootstrap to pick up existing Hex cache and deps[23] * rebar3/983: Add support for total code coverage[24] * rebar3/986: A bad template index does not crash; shows warning[25] * rebar3/987: Plugin templates[26] * rebar3/988: Fix wrong relative path resolution[27] * rebar3/989: convert ~> versions to highest matching[28] * rebar3/991: Upgrade relx to v3.11.0[29] * rebar3/992: Bump cth_readable to 1.1.1[30] * rebar3/993: contributors -> maintaiers in template app data[31] * rebar3/994: warn if the directories eunit' orct' show up in `src_dirs'[32] * rebar3/995: Support old-style shell for rebar3 shell[33] * rebar3/996: Updates cth_readable so it supports dumb terminals[34] * rebar3/999: Fix windows stuff[35] * rebar3/1005: only need to compare ref and not ref+url in git resource[36] * rebar3/1006: only apply default and prod profile to dependencies[37] * rebar3/1007: install project app plugins after discovering them not before[38] * rebar3/1008: Unquote templates, add a warning instead.[39] * rebar3/1009: merge overlay entries into a single {overlay, list()} for relx[40] * rebar3/1010: upgrade relx to 3.12.0[41] * rebar3/1012: Remove triple brackets in bbmustache templates[42] * rebar3/1013: upgrade bbmustache and relx[43] * rebar3/1016: when using the --file' argument toeunit' paths were being converted[44] * rebar3/1017: check at runtime instead of compile time for file:list_dir_all/1[45] * rebar3/1018: change detection of valid modules for eunit[46] * rebar3/1021: convert 'app' to 'application' in eunit_opts to match cmdline args[47] * rebar3/1022: Display error message when bad config is loaded[48] * rebar3/1023: Rework README and CONTRIBUTING documents[49] * rebar3/1024: deduplicate default test set generated by rebar3 eunit[50] * rebar3/1025: add unstable install/upgrade instructions to readme[51] * rebar3/1029: local install and local upgrade[52] * rebar3/1031: add profile option to clean task[53] * rebar3/1033: Fix bash completion regression (cf66dfd6ba) and make lopt strings more resilient[54] * rebar3/1035: Add module directory to include path[55] * rebar3/1041: Add test case for relx overlay vars[56] * rebar3/1043: don't strip the project apps when running ct with just a root suite specified[57] * rebar3/1045: don't lose overrides in an app when installing plugins it uses[58] * rebar3/1046: add user-agent to http request headers[59] * rebar3/1047: Ignore unknown warning when dialyzer < 2.8[60] * rebar3/1048: Add secondary hook for rebar_prv_compile[61] * rebar3/1050: check top level config for minimum or blacklisted otps at start[62] * rebar3/1053: upgrade relx to 3.15.0[63] * rebar3/1061: Make lock files future-proof[64] * rebar3/1062: update relx to 3.16.0 to include color intesity update[65] * rebar3/1065: give top level plugin providers precedence over default providers[66] * rebar3/1066: Bump cth_readable[67] * rebar3/1067: set default color intensity to high[68] * rebar3/1068: upgrade relx to 3.17.0[69] * rebar3/1070: Hex improvements[70] * rebar3/1071: break up do/1 function in install_deps to make upgrade less confusing[71] * rebar3/1073: Support --setcookie option[72] * rebar3/1075: add project_providers after initing default providers but allow overrides[73] * rebar3/1076: Add me to THANKS for suffering through Tristan's code[74] * rebar3/1082: Fixed rebar.config shell_apps setting[75] * rebar3/1090: fix auto-registry update to work even when not a locked pkg-vsn[76] * rebar3/1091: Run all hooks[77] * rebar3/1092: Add short-options to the eunit provider.[78] * rebar3/1098: error on a cover spec in ct_opts[79] * rebar3/1099: add support for common tests include flag[80] * rebar3/1100: Fix quoting problem in zsh completion[81] * rebar3/1101: Take CT options errors and turn them to warnings[82] * rebar3/1102: include project_plugins in plugins that can be upgraded[83] * rebar3/1103: bump certifi to 0.4.0[84] * rebar3/1104: move dialyze setting of debug_info to overrides in profile[85] * rebar3/1106: define the 'EUNIT' macro in the test profile[86] * rebar3/1107: upgrade cth_readable[87] * rebar3/1108: make omar happy[88] * rebar3/1110: pass loglevel used in rebar3 to relx[89] Links: 1. https://github.com/rebar/rebar3/pull/902 2. https://github.com/rebar/rebar3/pull/921 3. https://github.com/rebar/rebar3/pull/922 4. https://github.com/rebar/rebar3/pull/923 5. https://github.com/rebar/rebar3/pull/924 6. https://github.com/rebar/rebar3/pull/925 7. https://github.com/rebar/rebar3/pull/927 8. https://github.com/rebar/rebar3/pull/928 9. https://github.com/rebar/rebar3/pull/930 10. https://github.com/rebar/rebar3/pull/934 11. https://github.com/rebar/rebar3/pull/935 12. https://github.com/rebar/rebar3/pull/941 13. https://github.com/rebar/rebar3/pull/942 14. https://github.com/rebar/rebar3/pull/943 15. https://github.com/rebar/rebar3/pull/945 16. https://github.com/rebar/rebar3/pull/948 17. https://github.com/rebar/rebar3/pull/949 18. https://github.com/rebar/rebar3/pull/963 19. https://github.com/rebar/rebar3/pull/965 20. https://github.com/rebar/rebar3/pull/966 21. https://github.com/rebar/rebar3/pull/967 22. https://github.com/rebar/rebar3/pull/970 23. https://github.com/rebar/rebar3/pull/982 24. https://github.com/rebar/rebar3/pull/983 25. https://github.com/rebar/rebar3/pull/986 26. https://github.com/rebar/rebar3/pull/987 27. https://github.com/rebar/rebar3/pull/988 28. https://github.com/rebar/rebar3/pull/989 29. https://github.com/rebar/rebar3/pull/991 30. https://github.com/rebar/rebar3/pull/992 31. https://github.com/rebar/rebar3/pull/993 32. https://github.com/rebar/rebar3/pull/994 33. https://github.com/rebar/rebar3/pull/995 34. https://github.com/rebar/rebar3/pull/996 35. https://github.com/rebar/rebar3/pull/999 36. https://github.com/rebar/rebar3/pull/1005 37. https://github.com/rebar/rebar3/pull/1006 38. https://github.com/rebar/rebar3/pull/1007 39. https://github.com/rebar/rebar3/pull/1008 40. https://github.com/rebar/rebar3/pull/1009 41. https://github.com/rebar/rebar3/pull/1010 42. https://github.com/rebar/rebar3/pull/1012 43. https://github.com/rebar/rebar3/pull/1013 44. https://github.com/rebar/rebar3/pull/1016 45. https://github.com/rebar/rebar3/pull/1017 46. https://github.com/rebar/rebar3/pull/1018 47. https://github.com/rebar/rebar3/pull/1021 48. https://github.com/rebar/rebar3/pull/1022 49. https://github.com/rebar/rebar3/pull/1023 50. https://github.com/rebar/rebar3/pull/1024 51. https://github.com/rebar/rebar3/pull/1025 52. https://github.com/rebar/rebar3/pull/1029 53. https://github.com/rebar/rebar3/pull/1031 54. https://github.com/rebar/rebar3/pull/1033 55. https://github.com/rebar/rebar3/pull/1035 56. https://github.com/rebar/rebar3/pull/1041 57. https://github.com/rebar/rebar3/pull/1043 58. https://github.com/rebar/rebar3/pull/1045 59. https://github.com/rebar/rebar3/pull/1046 60. https://github.com/rebar/rebar3/pull/1047 61. https://github.com/rebar/rebar3/pull/1048 62. https://github.com/rebar/rebar3/pull/1050 63. https://github.com/rebar/rebar3/pull/1053 64. https://github.com/rebar/rebar3/pull/1061 65. https://github.com/rebar/rebar3/pull/1062 66. https://github.com/rebar/rebar3/pull/1065 67. https://github.com/rebar/rebar3/pull/1066 68. https://github.com/rebar/rebar3/pull/1067 69. https://github.com/rebar/rebar3/pull/1068 70. https://github.com/rebar/rebar3/pull/1070 71. https://github.com/rebar/rebar3/pull/1071 72. https://github.com/rebar/rebar3/pull/1073 73. https://github.com/rebar/rebar3/pull/1075 74. https://github.com/rebar/rebar3/pull/1076 75. https://github.com/rebar/rebar3/pull/1082 76. https://github.com/rebar/rebar3/pull/1090 77. https://github.com/rebar/rebar3/pull/1091 78. https://github.com/rebar/rebar3/pull/1092 79. https://github.com/rebar/rebar3/pull/1098 80. https://github.com/rebar/rebar3/pull/1099 81. https://github.com/rebar/rebar3/pull/1100 82. https://github.com/rebar/rebar3/pull/1101 83. https://github.com/rebar/rebar3/pull/1102 84. https://github.com/rebar/rebar3/pull/1103 85. https://github.com/rebar/rebar3/pull/1104 86. https://github.com/rebar/rebar3/pull/1106 87. https://github.com/rebar/rebar3/pull/1107 88. https://github.com/rebar/rebar3/pull/1108 89. https://github.com/rebar/rebar3/pull/1110 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rj@REDACTED Wed Mar 9 19:23:16 2016 From: rj@REDACTED (Richard Jones) Date: Wed, 9 Mar 2016 18:23:16 +0000 Subject: [erlang-questions] [ANN] Rebar3 first stable release 3.0.0 In-Reply-To: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> References: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> Message-ID: congratulations! thanks for making rebar great again ?? On 9 Mar 2016 6:21 p.m., "Tristan Sloughter" wrote: > We are happy to announce the first stable release of rebar3 with version > 3.0.0. > > A bit over a year ago, we had set out to make rebar great again. We > believe we have done it, and now have a solid core on which to build more > tools and grow a better community. > > Now, this doesn't mean there are no things to improve or that the code > base is bug-free. It does however mean that we are now committed to > backwards compatibility as we move forward with rebar3's next improvements. > > If you have never tried rebar3, the big item names since rebar 2.x are: > > - repeatable builds with a big commitment to OTP principles > - commands that know their own dependencies > - support for profiles, allowing contextual configuration of > applications (for example, test dependencies can be kept apart from regular > deps) > - documentation is seen as a feature > - support for packages through hex.pm > - new plugin interface and mechanisms, with command namespaces > - mechanisms to override dependencies' configurations > - switching releases to relx > - and a lot of improvements to existing commands such as templates, > dialyzer, shell, and so on. > > You will find documentation and how to get started with rebar3 on the > website: http://www.rebar3.org/docs/getting-started > > For package publishing checkout the documentation on hex.pm: > https://hex.pm/docs/rebar3_publish > > If you feel like getting started, we have had numerous posts about the new > features and user experiences: > > - http://ferd.ca/dev/rebar3-shell.html > - > http://blog.erlware.org/rebar3-features-part-1-local-install-and-upgrade/ > - http://blog.erlware.org/rebar3-features-part-2-dependency-tree/ > - http://blog.erlware.org/rebar3-features-part-3-overrides/ > - http://blog.erlware.org/rebar3-features-part-4-profiles/ > - > http://blog.erlware.org/rebar3-features-part-5-dependency-branch-handling/ > - http://blog.erlware.org/rebar3-features-part-6-_checkouts-2/ > - http://blog.erlware.org/rebar3-hex-plugin/ > > Heinz Gies wrote about his experience migrating to rebar3 form rebar2 at > http://blog.licenser.net/blog/2015/07/10/migrating-to-rebar3/ > > The How I Start Erlang guide was updated to use rebar3: > https://howistart.org/posts/erlang/1 > > And a guide about managing package has been written by Bruce Yinhe: > https://medium.com/@brucify/using-rebar3-to-manage-erlang-packages-282f78adff1e#.ay10goc0z > > Lastly, for those who have already adopted the tool, here is what has > changed since beta-4: > > - rebar3/902: Less color > - rebar3/921: Fix error reports on missing include paths > > - rebar3/922: warn on incorrectly specified test options in > rebar.config > - rebar3/923: support temporary cdn change with HEX_CDN os var > > - rebar3/924: only add package list of versions to registry if it has > the right build tool support > - rebar3/925: upgrade eunit_formatters for OTP 18 support > > - rebar3/927: upgrade eunit_formatters to 0.3.1 for OTP18 bug fix > > - rebar3/928: Ct output improvements > > - rebar3/930: Handle force flags in leading position > > - rebar3/934: don't add a provider if the same namespace and name > already exists > - rebar3/935: Default to no eunit formatter if verbose specified > > - rebar3/941: preserve attributes when copying files in > rebar_utils:cp_r for unix > - rebar3/942: special handling of relx configs in profiles > > - rebar3/943: remove backward_compat entry from travis s3 > > - rebar3/945: auto-update the registry if a pkg isn't found, fail if > it still isn't found > - rebar3/948: Fix a small bug in the MIB compiler when building > dependencies > - rebar3/949: fetch eunit_formatters config not from the command args > but from the config > - rebar3/963: Tup merge tests > > - rebar3/965: fix tupple merging for realsies. > > - rebar3/966: allow ct suites to be specified at root of project (or > root of app) > - rebar3/967: symlink mib hrl output in apps `include' directories > > - rebar3/970: upgrade certifi to latest release > > - rebar3/982: Allow bootstrap to pick up existing Hex cache and deps > > - rebar3/983: Add support for total code coverage > > - rebar3/986: A bad template index does not crash; shows warning > > - rebar3/987: Plugin templates > > - rebar3/988: Fix wrong relative path resolution > > - rebar3/989: convert ~> versions to highest matching > > - rebar3/991: Upgrade relx to v3.11.0 > > - rebar3/992: Bump cth_readable to 1.1.1 > > - rebar3/993: contributors -> maintaiers in template app data > > - rebar3/994: warn if the directories eunit' orct' show up in > `src_dirs' > - rebar3/995: Support old-style shell for rebar3 shell > > - rebar3/996: Updates cth_readable so it supports dumb terminals > > - rebar3/999: Fix windows stuff > > - rebar3/1005: only need to compare ref and not ref+url in git resource > > - rebar3/1006: only apply default and prod profile to dependencies > > - rebar3/1007: install project app plugins after discovering them not > before > - rebar3/1008: Unquote templates, add a warning instead. > > - rebar3/1009: merge overlay entries into a single {overlay, list()} > for relx > - rebar3/1010: upgrade relx to 3.12.0 > > - rebar3/1012: Remove triple brackets in bbmustache templates > > - rebar3/1013: upgrade bbmustache and relx > > - rebar3/1016: when using the --file' argument toeunit' paths were > being converted > - rebar3/1017: check at runtime instead of compile time for > file:list_dir_all/1 > - rebar3/1018: change detection of valid modules for eunit > > - rebar3/1021: convert 'app' to 'application' in eunit_opts to match > cmdline args > - rebar3/1022: Display error message when bad config is loaded > > - rebar3/1023: Rework README and CONTRIBUTING documents > > - rebar3/1024: deduplicate default test set generated by rebar3 eunit > > - rebar3/1025: add unstable install/upgrade instructions to readme > > - rebar3/1029: local install and local upgrade > > - rebar3/1031: add profile option to clean task > > - rebar3/1033: Fix bash completion regression (cf66dfd6ba) and make > lopt strings more resilient > - rebar3/1035: Add module directory to include path > > - rebar3/1041: Add test case for relx overlay vars > > - rebar3/1043: don't strip the project apps when running ct with just > a root suite specified > - rebar3/1045: don't lose overrides in an app when installing plugins > it uses > - rebar3/1046: add user-agent to http request headers > > - rebar3/1047: Ignore unknown warning when dialyzer < 2.8 > > - rebar3/1048: Add secondary hook for rebar_prv_compile > > - rebar3/1050: check top level config for minimum or blacklisted otps > at start > - rebar3/1053: upgrade relx to 3.15.0 > > - rebar3/1061: Make lock files future-proof > > - rebar3/1062: update relx to 3.16.0 to include color intesity update > > - rebar3/1065: give top level plugin providers precedence over default > providers > - rebar3/1066: Bump cth_readable > > - rebar3/1067: set default color intensity to high > > - rebar3/1068: upgrade relx to 3.17.0 > > - rebar3/1070: Hex improvements > > - rebar3/1071: break up do/1 function in install_deps to make upgrade > less confusing > - rebar3/1073: Support --setcookie option > > - rebar3/1075: add project_providers after initing default providers > but allow overrides > - rebar3/1076: Add me to THANKS for suffering through Tristan's code > > - rebar3/1082: Fixed rebar.config shell_apps setting > > - rebar3/1090: fix auto-registry update to work even when not a locked > pkg-vsn > - rebar3/1091: Run all hooks > > - rebar3/1092: Add short-options to the eunit provider. > > - rebar3/1098: error on a cover spec in ct_opts > > - rebar3/1099: add support for common tests include flag > > - rebar3/1100: Fix quoting problem in zsh completion > > - rebar3/1101: Take CT options errors and turn them to warnings > > - rebar3/1102: include project_plugins in plugins that can be upgraded > > - rebar3/1103: bump certifi to 0.4.0 > > - rebar3/1104: move dialyze setting of debug_info to overrides in > profile > - rebar3/1106: define the 'EUNIT' macro in the test profile > > - rebar3/1107: upgrade cth_readable > > - rebar3/1108: make omar happy > > - rebar3/1110: pass loglevel used in rebar3 to relx > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Wed Mar 9 19:35:37 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 9 Mar 2016 13:35:37 -0500 Subject: [erlang-questions] [ANN] Rebar3 first stable release 3.0.0 In-Reply-To: References: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> Message-ID: <56E06CF9.7050809@khandkar.net> +1 I'm feeling this rebern! On 3/9/16 1:23 PM, Richard Jones wrote: > congratulations! thanks for making rebar great again ?? > On 9 Mar 2016 6:21 p.m., "Tristan Sloughter" wrote: > >> We are happy to announce the first stable release of rebar3 with version >> 3.0.0. >> >> A bit over a year ago, we had set out to make rebar great again. We >> believe we have done it, and now have a solid core on which to build more >> tools and grow a better community. >> >> Now, this doesn't mean there are no things to improve or that the code >> base is bug-free. It does however mean that we are now committed to >> backwards compatibility as we move forward with rebar3's next improvements. >> >> If you have never tried rebar3, the big item names since rebar 2.x are: >> >> - repeatable builds with a big commitment to OTP principles >> - commands that know their own dependencies >> - support for profiles, allowing contextual configuration of >> applications (for example, test dependencies can be kept apart from regular >> deps) >> - documentation is seen as a feature >> - support for packages through hex.pm >> - new plugin interface and mechanisms, with command namespaces >> - mechanisms to override dependencies' configurations >> - switching releases to relx >> - and a lot of improvements to existing commands such as templates, >> dialyzer, shell, and so on. >> >> You will find documentation and how to get started with rebar3 on the >> website: http://www.rebar3.org/docs/getting-started >> >> For package publishing checkout the documentation on hex.pm: >> https://hex.pm/docs/rebar3_publish >> >> If you feel like getting started, we have had numerous posts about the new >> features and user experiences: >> >> - http://ferd.ca/dev/rebar3-shell.html >> - >> http://blog.erlware.org/rebar3-features-part-1-local-install-and-upgrade/ >> - http://blog.erlware.org/rebar3-features-part-2-dependency-tree/ >> - http://blog.erlware.org/rebar3-features-part-3-overrides/ >> - http://blog.erlware.org/rebar3-features-part-4-profiles/ >> - >> http://blog.erlware.org/rebar3-features-part-5-dependency-branch-handling/ >> - http://blog.erlware.org/rebar3-features-part-6-_checkouts-2/ >> - http://blog.erlware.org/rebar3-hex-plugin/ >> >> Heinz Gies wrote about his experience migrating to rebar3 form rebar2 at >> http://blog.licenser.net/blog/2015/07/10/migrating-to-rebar3/ >> >> The How I Start Erlang guide was updated to use rebar3: >> https://howistart.org/posts/erlang/1 >> >> And a guide about managing package has been written by Bruce Yinhe: >> https://medium.com/@brucify/using-rebar3-to-manage-erlang-packages-282f78adff1e#.ay10goc0z >> >> Lastly, for those who have already adopted the tool, here is what has >> changed since beta-4: >> >> - rebar3/902: Less color >> - rebar3/921: Fix error reports on missing include paths >> >> - rebar3/922: warn on incorrectly specified test options in >> rebar.config >> - rebar3/923: support temporary cdn change with HEX_CDN os var >> >> - rebar3/924: only add package list of versions to registry if it has >> the right build tool support >> - rebar3/925: upgrade eunit_formatters for OTP 18 support >> >> - rebar3/927: upgrade eunit_formatters to 0.3.1 for OTP18 bug fix >> >> - rebar3/928: Ct output improvements >> >> - rebar3/930: Handle force flags in leading position >> >> - rebar3/934: don't add a provider if the same namespace and name >> already exists >> - rebar3/935: Default to no eunit formatter if verbose specified >> >> - rebar3/941: preserve attributes when copying files in >> rebar_utils:cp_r for unix >> - rebar3/942: special handling of relx configs in profiles >> >> - rebar3/943: remove backward_compat entry from travis s3 >> >> - rebar3/945: auto-update the registry if a pkg isn't found, fail if >> it still isn't found >> - rebar3/948: Fix a small bug in the MIB compiler when building >> dependencies >> - rebar3/949: fetch eunit_formatters config not from the command args >> but from the config >> - rebar3/963: Tup merge tests >> >> - rebar3/965: fix tupple merging for realsies. >> >> - rebar3/966: allow ct suites to be specified at root of project (or >> root of app) >> - rebar3/967: symlink mib hrl output in apps `include' directories >> >> - rebar3/970: upgrade certifi to latest release >> >> - rebar3/982: Allow bootstrap to pick up existing Hex cache and deps >> >> - rebar3/983: Add support for total code coverage >> >> - rebar3/986: A bad template index does not crash; shows warning >> >> - rebar3/987: Plugin templates >> >> - rebar3/988: Fix wrong relative path resolution >> >> - rebar3/989: convert ~> versions to highest matching >> >> - rebar3/991: Upgrade relx to v3.11.0 >> >> - rebar3/992: Bump cth_readable to 1.1.1 >> >> - rebar3/993: contributors -> maintaiers in template app data >> >> - rebar3/994: warn if the directories eunit' orct' show up in >> `src_dirs' >> - rebar3/995: Support old-style shell for rebar3 shell >> >> - rebar3/996: Updates cth_readable so it supports dumb terminals >> >> - rebar3/999: Fix windows stuff >> >> - rebar3/1005: only need to compare ref and not ref+url in git resource >> >> - rebar3/1006: only apply default and prod profile to dependencies >> >> - rebar3/1007: install project app plugins after discovering them not >> before >> - rebar3/1008: Unquote templates, add a warning instead. >> >> - rebar3/1009: merge overlay entries into a single {overlay, list()} >> for relx >> - rebar3/1010: upgrade relx to 3.12.0 >> >> - rebar3/1012: Remove triple brackets in bbmustache templates >> >> - rebar3/1013: upgrade bbmustache and relx >> >> - rebar3/1016: when using the --file' argument toeunit' paths were >> being converted >> - rebar3/1017: check at runtime instead of compile time for >> file:list_dir_all/1 >> - rebar3/1018: change detection of valid modules for eunit >> >> - rebar3/1021: convert 'app' to 'application' in eunit_opts to match >> cmdline args >> - rebar3/1022: Display error message when bad config is loaded >> >> - rebar3/1023: Rework README and CONTRIBUTING documents >> >> - rebar3/1024: deduplicate default test set generated by rebar3 eunit >> >> - rebar3/1025: add unstable install/upgrade instructions to readme >> >> - rebar3/1029: local install and local upgrade >> >> - rebar3/1031: add profile option to clean task >> >> - rebar3/1033: Fix bash completion regression (cf66dfd6ba) and make >> lopt strings more resilient >> - rebar3/1035: Add module directory to include path >> >> - rebar3/1041: Add test case for relx overlay vars >> >> - rebar3/1043: don't strip the project apps when running ct with just >> a root suite specified >> - rebar3/1045: don't lose overrides in an app when installing plugins >> it uses >> - rebar3/1046: add user-agent to http request headers >> >> - rebar3/1047: Ignore unknown warning when dialyzer < 2.8 >> >> - rebar3/1048: Add secondary hook for rebar_prv_compile >> >> - rebar3/1050: check top level config for minimum or blacklisted otps >> at start >> - rebar3/1053: upgrade relx to 3.15.0 >> >> - rebar3/1061: Make lock files future-proof >> >> - rebar3/1062: update relx to 3.16.0 to include color intesity update >> >> - rebar3/1065: give top level plugin providers precedence over default >> providers >> - rebar3/1066: Bump cth_readable >> >> - rebar3/1067: set default color intensity to high >> >> - rebar3/1068: upgrade relx to 3.17.0 >> >> - rebar3/1070: Hex improvements >> >> - rebar3/1071: break up do/1 function in install_deps to make upgrade >> less confusing >> - rebar3/1073: Support --setcookie option >> >> - rebar3/1075: add project_providers after initing default providers >> but allow overrides >> - rebar3/1076: Add me to THANKS for suffering through Tristan's code >> >> - rebar3/1082: Fixed rebar.config shell_apps setting >> >> - rebar3/1090: fix auto-registry update to work even when not a locked >> pkg-vsn >> - rebar3/1091: Run all hooks >> >> - rebar3/1092: Add short-options to the eunit provider. >> >> - rebar3/1098: error on a cover spec in ct_opts >> >> - rebar3/1099: add support for common tests include flag >> >> - rebar3/1100: Fix quoting problem in zsh completion >> >> - rebar3/1101: Take CT options errors and turn them to warnings >> >> - rebar3/1102: include project_plugins in plugins that can be upgraded >> >> - rebar3/1103: bump certifi to 0.4.0 >> >> - rebar3/1104: move dialyze setting of debug_info to overrides in >> profile >> - rebar3/1106: define the 'EUNIT' macro in the test profile >> >> - rebar3/1107: upgrade cth_readable >> >> - rebar3/1108: make omar happy >> >> - rebar3/1110: pass loglevel used in rebar3 to relx >> From gumm@REDACTED Wed Mar 9 20:02:42 2016 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 9 Mar 2016 13:02:42 -0600 Subject: [erlang-questions] [ANN] Rebar3 first stable release 3.0.0 In-Reply-To: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> References: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> Message-ID: Awesome! Looking forward to transitioning from rebar2 with my stuff. Thanks for all the hard work to make this happen. Following from the sidelines on the rebar mailing list has been informative. Now to just get my hands dirty after all the hard work is done! Cheers! -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Mar 9, 2016 12:22 PM, "Tristan Sloughter" wrote: > We are happy to announce the first stable release of rebar3 with version > 3.0.0. > > A bit over a year ago, we had set out to make rebar great again. We > believe we have done it, and now have a solid core on which to build more > tools and grow a better community. > > Now, this doesn't mean there are no things to improve or that the code > base is bug-free. It does however mean that we are now committed to > backwards compatibility as we move forward with rebar3's next improvements. > > If you have never tried rebar3, the big item names since rebar 2.x are: > > - repeatable builds with a big commitment to OTP principles > - commands that know their own dependencies > - support for profiles, allowing contextual configuration of > applications (for example, test dependencies can be kept apart from regular > deps) > - documentation is seen as a feature > - support for packages through hex.pm > - new plugin interface and mechanisms, with command namespaces > - mechanisms to override dependencies' configurations > - switching releases to relx > - and a lot of improvements to existing commands such as templates, > dialyzer, shell, and so on. > > You will find documentation and how to get started with rebar3 on the > website: http://www.rebar3.org/docs/getting-started > > For package publishing checkout the documentation on hex.pm: > https://hex.pm/docs/rebar3_publish > > If you feel like getting started, we have had numerous posts about the new > features and user experiences: > > - http://ferd.ca/dev/rebar3-shell.html > - > http://blog.erlware.org/rebar3-features-part-1-local-install-and-upgrade/ > - http://blog.erlware.org/rebar3-features-part-2-dependency-tree/ > - http://blog.erlware.org/rebar3-features-part-3-overrides/ > - http://blog.erlware.org/rebar3-features-part-4-profiles/ > - > http://blog.erlware.org/rebar3-features-part-5-dependency-branch-handling/ > - http://blog.erlware.org/rebar3-features-part-6-_checkouts-2/ > - http://blog.erlware.org/rebar3-hex-plugin/ > > Heinz Gies wrote about his experience migrating to rebar3 form rebar2 at > http://blog.licenser.net/blog/2015/07/10/migrating-to-rebar3/ > > The How I Start Erlang guide was updated to use rebar3: > https://howistart.org/posts/erlang/1 > > And a guide about managing package has been written by Bruce Yinhe: > https://medium.com/@brucify/using-rebar3-to-manage-erlang-packages-282f78adff1e#.ay10goc0z > > Lastly, for those who have already adopted the tool, here is what has > changed since beta-4: > > - rebar3/902: Less color > - rebar3/921: Fix error reports on missing include paths > > - rebar3/922: warn on incorrectly specified test options in > rebar.config > - rebar3/923: support temporary cdn change with HEX_CDN os var > > - rebar3/924: only add package list of versions to registry if it has > the right build tool support > - rebar3/925: upgrade eunit_formatters for OTP 18 support > > - rebar3/927: upgrade eunit_formatters to 0.3.1 for OTP18 bug fix > > - rebar3/928: Ct output improvements > > - rebar3/930: Handle force flags in leading position > > - rebar3/934: don't add a provider if the same namespace and name > already exists > - rebar3/935: Default to no eunit formatter if verbose specified > > - rebar3/941: preserve attributes when copying files in > rebar_utils:cp_r for unix > - rebar3/942: special handling of relx configs in profiles > > - rebar3/943: remove backward_compat entry from travis s3 > > - rebar3/945: auto-update the registry if a pkg isn't found, fail if > it still isn't found > - rebar3/948: Fix a small bug in the MIB compiler when building > dependencies > - rebar3/949: fetch eunit_formatters config not from the command args > but from the config > - rebar3/963: Tup merge tests > > - rebar3/965: fix tupple merging for realsies. > > - rebar3/966: allow ct suites to be specified at root of project (or > root of app) > - rebar3/967: symlink mib hrl output in apps `include' directories > > - rebar3/970: upgrade certifi to latest release > > - rebar3/982: Allow bootstrap to pick up existing Hex cache and deps > > - rebar3/983: Add support for total code coverage > > - rebar3/986: A bad template index does not crash; shows warning > > - rebar3/987: Plugin templates > > - rebar3/988: Fix wrong relative path resolution > > - rebar3/989: convert ~> versions to highest matching > > - rebar3/991: Upgrade relx to v3.11.0 > > - rebar3/992: Bump cth_readable to 1.1.1 > > - rebar3/993: contributors -> maintaiers in template app data > > - rebar3/994: warn if the directories eunit' orct' show up in > `src_dirs' > - rebar3/995: Support old-style shell for rebar3 shell > > - rebar3/996: Updates cth_readable so it supports dumb terminals > > - rebar3/999: Fix windows stuff > > - rebar3/1005: only need to compare ref and not ref+url in git resource > > - rebar3/1006: only apply default and prod profile to dependencies > > - rebar3/1007: install project app plugins after discovering them not > before > - rebar3/1008: Unquote templates, add a warning instead. > > - rebar3/1009: merge overlay entries into a single {overlay, list()} > for relx > - rebar3/1010: upgrade relx to 3.12.0 > > - rebar3/1012: Remove triple brackets in bbmustache templates > > - rebar3/1013: upgrade bbmustache and relx > > - rebar3/1016: when using the --file' argument toeunit' paths were > being converted > - rebar3/1017: check at runtime instead of compile time for > file:list_dir_all/1 > - rebar3/1018: change detection of valid modules for eunit > > - rebar3/1021: convert 'app' to 'application' in eunit_opts to match > cmdline args > - rebar3/1022: Display error message when bad config is loaded > > - rebar3/1023: Rework README and CONTRIBUTING documents > > - rebar3/1024: deduplicate default test set generated by rebar3 eunit > > - rebar3/1025: add unstable install/upgrade instructions to readme > > - rebar3/1029: local install and local upgrade > > - rebar3/1031: add profile option to clean task > > - rebar3/1033: Fix bash completion regression (cf66dfd6ba) and make > lopt strings more resilient > - rebar3/1035: Add module directory to include path > > - rebar3/1041: Add test case for relx overlay vars > > - rebar3/1043: don't strip the project apps when running ct with just > a root suite specified > - rebar3/1045: don't lose overrides in an app when installing plugins > it uses > - rebar3/1046: add user-agent to http request headers > > - rebar3/1047: Ignore unknown warning when dialyzer < 2.8 > > - rebar3/1048: Add secondary hook for rebar_prv_compile > > - rebar3/1050: check top level config for minimum or blacklisted otps > at start > - rebar3/1053: upgrade relx to 3.15.0 > > - rebar3/1061: Make lock files future-proof > > - rebar3/1062: update relx to 3.16.0 to include color intesity update > > - rebar3/1065: give top level plugin providers precedence over default > providers > - rebar3/1066: Bump cth_readable > > - rebar3/1067: set default color intensity to high > > - rebar3/1068: upgrade relx to 3.17.0 > > - rebar3/1070: Hex improvements > > - rebar3/1071: break up do/1 function in install_deps to make upgrade > less confusing > - rebar3/1073: Support --setcookie option > > - rebar3/1075: add project_providers after initing default providers > but allow overrides > - rebar3/1076: Add me to THANKS for suffering through Tristan's code > > - rebar3/1082: Fixed rebar.config shell_apps setting > > - rebar3/1090: fix auto-registry update to work even when not a locked > pkg-vsn > - rebar3/1091: Run all hooks > > - rebar3/1092: Add short-options to the eunit provider. > > - rebar3/1098: error on a cover spec in ct_opts > > - rebar3/1099: add support for common tests include flag > > - rebar3/1100: Fix quoting problem in zsh completion > > - rebar3/1101: Take CT options errors and turn them to warnings > > - rebar3/1102: include project_plugins in plugins that can be upgraded > > - rebar3/1103: bump certifi to 0.4.0 > > - rebar3/1104: move dialyze setting of debug_info to overrides in > profile > - rebar3/1106: define the 'EUNIT' macro in the test profile > > - rebar3/1107: upgrade cth_readable > > - rebar3/1108: make omar happy > > - rebar3/1110: pass loglevel used in rebar3 to relx > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 10 00:41:14 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 10 Mar 2016 12:41:14 +1300 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> <56DDA67C.90204@ninenines.eu> <92d3e03a-cc17-7717-e58c-76968d76e217@cs.otago.ac.nz> Message-ID: <816626e7-f1ae-509c-9cee-a8303dedb169@cs.otago.ac.nz> On 9/03/16 11:27 pm, Chandru wrote: > > I looked up the definition and while I couldn't find one which > restricts the insertion to ONE thing, it certainly doesn't say it is > the repeated insertion into a sequence, so clearly intercalate is the > wrong choice here. OED sense 1: To insert (an additional day, days, or month) in the calendar in order to bring the current reckoning of time into harmony with the natural solar year. [The Gregorian calendar inserts ONE day into the year. The Hebrew calendar inserts ONE month into the year. Some lunar calendars insert ONE day into a month to catch up with what the real moon is doing.] OED sense 2: To insert or interpose something additional, extraneous, or out of the ordinary course, between the ordinary members of any series or the successive parts of any whole [One of the examples actually refers to inserting ONE letter into a word.] From zxq9@REDACTED Thu Mar 10 03:11:05 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 10 Mar 2016 11:11:05 +0900 Subject: [erlang-questions] REST server In-Reply-To: References: Message-ID: <4409525.Hhk3QtFrg4@burrito> On Wednesday 09 March 2016 08:03:12 Vlad Dumitrescu wrote: > ...And so it looks like Yaws does that, even if it wasn't advertised... YAWS is one of the most totally un-advertised-yet-actually-awesome pieces of software I've ever encountered. It is in deep stealth mode as far as most of the world is concerned, despite being exactly what a lot of folks are looking for. Anywhere but Erlang I would find this very strange. -Craig From zxq9@REDACTED Thu Mar 10 03:32:24 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 10 Mar 2016 11:32:24 +0900 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: References: Message-ID: <12463897.5JasyehX32@burrito> Hi, Ryan. On Wednesday 09 March 2016 08:45:23 Ryan Stewart wrote: > # The Question > > Why should we run embedded mode in production? It seems that's the unspoken > law for production systems, but why? Is there some performance overhead > running in interactive mode? I've read about Code Loading Strategy in the > System Principles User's Guide: > http://erlang.org/doc/system_principles/system_principles.html#id56789 > > I've also read about the mode differences in the code server: > http://erlang.org/doc/man/code.html > > I don't see anything in any of the stdlib/OTP docs about *why* you would > choose embedded vs interactive mode. I'm leaning heavily toward using > interactive mode everywhere, dev through production, because it's so much > simpler and easy to understand. Someone please talk me out of it if that's > a terribly Bad Idea. I've often wondered this myself. It is amazing and wonderful that Erlang is able to deal so smoothly with the embedded case and even provides a huge set of built in tools for getting systems running that way and manipulating them -- but very few of us non-telecom folks are deploying to telephone switches. I've written an entire system around dealing with Erlang code as signed source bundles from a repo and dealing with the runtime as a more ordinary language runtime which pulls and builds needed modules/version as necessary within a given environment context (somewhat like Python's virtualenv). I used this mostly for deploying client side code and keeping it up to date without the hassle of building releases everywhere or making the users themselves deal with constant "There's an upgrade! Install it or people will think you have cooties!" messages (which they will disregard anyway). This was easier for new coders (coming from backgrounds where this is the norm), small business system administrators (who are used to maintaining runtime compliance with a given set of software they have to use), and users (who are used to this sort of "I click it and things happen" utility). But it is clearly heresy to do this, so I tend to not mention it much. I've even wondered if I should have been embarrassed to have committed such sacrilege. The original version of this system is still in production, but behind closed doors (>.<). It was a bit of a monolith anyway (grew into its own in an, um, "organic" way as I added to it). I intend to reincarnate it as a set of smallish utilities, but haven't yet had the time, and as far as I can tell there is just about zero interest in the community for anything but releases, so just haven't put much effort there. In the back of my mind I have always wondered: "This isn't so hard... why isn't this considered normal?" And after experiencing no problems in production I always wondered if I had just lucked out dodged some horrible situation too foul to document by sheer chance -- or if this really is indeed not a big deal and our obsession with releases is just part of the tribal tradition. So... why? -Craig From mjtruog@REDACTED Thu Mar 10 04:11:46 2016 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 09 Mar 2016 19:11:46 -0800 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: References: Message-ID: <56E0E5F2.6030506@gmail.com> On 03/09/2016 06:45 AM, Ryan Stewart wrote: > # Quick Background > > I inherited an Erlang system that's a fairly typical HTTP, almost-ReST API. The nature of the API and what it represents made Erlang a good choice for its concurrency aspects. As for the reliability aspects, we rely totally on a typical HTTP/S load balancer setup for HA. We do rolling restarts behind the LB and not hot code upgrades as enabled by Erlang because of the complexity of Erlang upgrades. > > # The Question > > Why should we run embedded mode in production? It seems that's the unspoken law for production systems, but why? Is there some performance overhead running in interactive mode? I've read about Code Loading Strategy in the System Principles User's Guide: > http://erlang.org/doc/system_principles/system_principles.html#id56789 > > I've also read about the mode differences in the code server: > http://erlang.org/doc/man/code.html > > I don't see anything in any of the stdlib/OTP docs about *why* you would choose embedded vs interactive mode. I'm leaning heavily toward using interactive mode everywhere, dev through production, because it's so much simpler and easy to understand. Someone please talk me out of it if that's a terribly Bad Idea. > > Ryan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions My understanding is that embedded is preferred for any production use, to make sure the system will fail-fast upon startup if there are any problems loading dependencies. The best time to have something fail is when it first starts, due to its lifetime being undefined and possibly infinite (ignoring heat death in the universe, and other natural disasters that should be written out of legal agreements :-). Ideally this concept is extended into the runtime of the server, into initialization and configuration source code, to make sure the server can fail-fast upon startup when the server is misconfigured, rather than waiting an arbitrary number of hours or days to find out that a problem exists. This approach helps to avoid a reliance on a fire-fighting mentality that becomes dependant on monitoring for feedback on a system's health due to the source code being a potentially unknowable black-box for some organizational reason. The interactive mode helps when testing, since things are loaded automatically and you can have less concern about the dependencies, since you are actively creating or modifying the dependencies. In production, you want an iron fist's control on all the dependencies, to make sure everything is repeatable and the service is stable, otherwise the development is pursuing stability in a serious way. So, that means that production usage should demand that all the dependencies come from a specific place at a specific version that is known and tracked, to be easily replicated, without any ambiguity. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Thu Mar 10 04:35:21 2016 From: jinni.park@REDACTED (Park, Sungjin) Date: Thu, 10 Mar 2016 12:35:21 +0900 Subject: [erlang-questions] gen_server:cast from escript In-Reply-To: References: Message-ID: You didn't connect to target node. Have to call net_adm:ping(Node) first to connect. On Tue, Mar 8, 2016 at 3:47 PM, YuanZhiqian wrote: > Hi guys, > > I was trying to call gen_server:cast from a escript file, everything > that used to work well in erl shell won't work in escript, I have no idea > what made the difference, here is the code, could anyone help me to find a > clue? > > > > #!/usr/bin/env escript > > %%! -name bk@REDACTED -setcookie budget_keeper > > main(Argv) -> > {Node, File} = case Argv of > [] -> > *{'budget_keeper@REDACTED > ', "cache"*}; > [F] -> > {'budget_keeper@REDACTED', F}; > [N, F] -> > {N, F} > end, > io:format("~p ~p ~p ~p~n", [node(), erlang:get_cookie(), Node, File]), > gen_server:cast({bk_main, Node}, {dump_data, File}). > > > > As shown above, the target process is called "bk_main" which is on the > node 'budget_keeper@REDACTED', I have run the same code in erl shell, > and bk_main can get the notice, but nothing happened when I ran the same > code in this script. > > Best regards > Zhiqian > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Mar 10 07:28:50 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 10 Mar 2016 09:28:50 +0300 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E0E5F2.6030506@gmail.com> References: <56E0E5F2.6030506@gmail.com> Message-ID: Have you ever looked in strace log when a moderate erlang system is booting from flash on ARM? =) We (in flussonic) are not using embedded mode and don't use releases, but we explicitly load all beams and it is faster then load them ondemand. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Mar 10 08:00:03 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 9 Mar 2016 23:00:03 -0800 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: <0587800e-644b-bf84-b0e0-dc9f7f9ca91d@cs.otago.ac.nz> Message-ID: 2016-03-07 15:55 GMT-08:00 Richard A. O'Keefe : > the passionate pursuit of programming power from > precise analogies. > Parables, perhaps? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Mar 10 08:32:27 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 10 Mar 2016 08:32:27 +0100 Subject: [erlang-questions] REST server In-Reply-To: <4409525.Hhk3QtFrg4@burrito> References: <4409525.Hhk3QtFrg4@burrito> Message-ID: I've stumbled on a possible issue with yaws: since it contains C code, it's probably a bit difficult to distribute in binary form, right? best regards, Vlad On Thu, Mar 10, 2016 at 3:11 AM, zxq9 wrote: > On Wednesday 09 March 2016 08:03:12 Vlad Dumitrescu wrote: > > ...And so it looks like Yaws does that, even if it wasn't advertised... > > YAWS is one of the most totally un-advertised-yet-actually-awesome pieces > of software I've ever encountered. It is in deep stealth mode as far as > most of the world is concerned, despite being exactly what a lot of folks > are looking for. > > Anywhere but Erlang I would find this very strange. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ferenc.holzhauser@REDACTED Thu Mar 10 09:04:12 2016 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Thu, 10 Mar 2016 09:04:12 +0100 Subject: [erlang-questions] REST server In-Reply-To: References: <4409525.Hhk3QtFrg4@burrito> Message-ID: I believe those C parts can be taken out. Last time I looked (version 1.98) C was used to interface PAM authentication and for sendfile. As I recall both can be disabled during the build process. I have not really used the official build process though, so I'm not sure anymore where/how. Regards, Ferenc On 10 March 2016 at 08:32, Vlad Dumitrescu wrote: > I've stumbled on a possible issue with yaws: since it contains C code, > it's probably a bit difficult to distribute in binary form, right? > > best regards, > Vlad > > > On Thu, Mar 10, 2016 at 3:11 AM, zxq9 wrote: > >> On Wednesday 09 March 2016 08:03:12 Vlad Dumitrescu wrote: >> > ...And so it looks like Yaws does that, even if it wasn't advertised... >> >> YAWS is one of the most totally un-advertised-yet-actually-awesome pieces >> of software I've ever encountered. It is in deep stealth mode as far as >> most of the world is concerned, despite being exactly what a lot of folks >> are looking for. >> >> Anywhere but Erlang I would find this very strange. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Thu Mar 10 17:31:44 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Thu, 10 Mar 2016 18:31:44 +0200 Subject: [erlang-questions] plists:foreach performances Message-ID: Hello, Fun = fun(_X) -> ok end, List = lists:seq(1, 120000), plists:foreach(Fun, List). Takes a lot of time to complete on Erlang 18.2 any other way to have a function that spawn a number of processes and then returns when all processes died ? Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From andras.boroska@REDACTED Thu Mar 10 17:43:01 2016 From: andras.boroska@REDACTED (=?UTF-8?Q?Boroska_Andr=C3=A1s?=) Date: Thu, 10 Mar 2016 16:43:01 +0000 Subject: [erlang-questions] plists:foreach performances In-Reply-To: References: Message-ID: Do you run this in the interactive shell? In that case put it in a module and compile it. Measuring anything in the shell is not relevant. Andras -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Thu Mar 10 17:45:33 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Thu, 10 Mar 2016 18:45:33 +0200 Subject: [erlang-questions] plists:foreach performances In-Reply-To: References: Message-ID: No , I have a function: tt(X)-> Fun = fun(_X) -> ok end, List = lists:seq(1, X), plists:foreach(Fun, List). and I'm calling this one : module:tt(130000). Silviu On Thu, Mar 10, 2016 at 6:43 PM, Boroska Andr?s wrote: > Do you run this in the interactive shell? In that case put it in a module > and compile it. > > Measuring anything in the shell is not relevant. > > Andras > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Mar 10 19:13:37 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 10 Mar 2016 10:13:37 -0800 Subject: [erlang-questions] plists:foreach performances In-Reply-To: References: Message-ID: not sure which plist you're using, but the one that I could find looked not entirely erlang-idiomatic; it tried to manage the process scheduling itself, perhaps as a test implementation of the way that other languages might do it. Here's how I might tackle that problem in pure erlang. https://gist.github.com/anonymous/94ace7cd94fd5930f3e4 F. On Thu, Mar 10, 2016 at 8:45 AM, Caragea Silviu wrote: > No , > > I have a function: > > tt(X)-> > Fun = fun(_X) -> > ok > end, > > List = lists:seq(1, X), > plists:foreach(Fun, List). > > and I'm calling this one : module:tt(130000). > > Silviu > > > > On Thu, Mar 10, 2016 at 6:43 PM, Boroska Andr?s > wrote: > >> Do you run this in the interactive shell? In that case put it in a module >> and compile it. >> >> Measuring anything in the shell is not relevant. >> >> Andras >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Thu Mar 10 20:33:50 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Thu, 10 Mar 2016 21:33:50 +0200 Subject: [erlang-questions] plists:foreach performances In-Reply-To: References: Message-ID: Hello Felix, The repo is : https://github.com/silviucpp/plists.git I've tested your code and it's much slower than plists. In fact I discovered that using {processes, schedulers} in plists speedup this a lot and outperform everything. tt(X)-> Fun = fun(_X) -> ok end, List = lists:seq(1, X), timer:tc(fun()-> plists:foreach(Fun, List, {processes, schedulers}) end). tt2(X) -> timer:tc(fun()-> lotsa:run(fun lotsa:sample_fun/1, X) end). (loadtest@REDACTED)1> load_test:tt2(130000). {15197488,done} (loadtest@REDACTED)2> load_test:tt2(130000). {15784610,done} (loadtest@REDACTED)3> load_test:tt2(130000). {17058412,done} (loadtest@REDACTED)4> load_test:tt(130000). {13020,ok} (loadtest@REDACTED)5> load_test:tt(130000). {10628,ok} (loadtest@REDACTED)6> load_test:tt(130000). {9918,ok} If {processes, schedulers}is not used performance are like in your code. Silviu On Thu, Mar 10, 2016 at 8:13 PM, Felix Gallo wrote: > not sure which plist you're using, but the one that I could find looked > not entirely erlang-idiomatic; it tried to manage the process scheduling > itself, perhaps as a test implementation of the way that other languages > might do it. > > Here's how I might tackle that problem in pure erlang. > > https://gist.github.com/anonymous/94ace7cd94fd5930f3e4 > > F. > > On Thu, Mar 10, 2016 at 8:45 AM, Caragea Silviu > wrote: > >> No , >> >> I have a function: >> >> tt(X)-> >> Fun = fun(_X) -> >> ok >> end, >> >> List = lists:seq(1, X), >> plists:foreach(Fun, List). >> >> and I'm calling this one : module:tt(130000). >> >> Silviu >> >> >> >> On Thu, Mar 10, 2016 at 6:43 PM, Boroska Andr?s > > wrote: >> >>> Do you run this in the interactive shell? In that case put it in a >>> module and compile it. >>> >>> Measuring anything in the shell is not relevant. >>> >>> Andras >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Mar 10 21:14:01 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 10 Mar 2016 12:14:01 -0800 Subject: [erlang-questions] plists:foreach performances In-Reply-To: References: Message-ID: Yeah, that's because {processes, schedulers} limits the number of processes to the number of schedulers as an optimization. Like all optimization, tread carefully and know the implications before reaching for that first. On Thu, Mar 10, 2016 at 11:33 AM, Caragea Silviu wrote: > Hello Felix, > > The repo is : https://github.com/silviucpp/plists.git > > I've tested your code and it's much slower than plists. In fact I > discovered that using {processes, schedulers} in plists speedup this a lot > and outperform everything. > > tt(X)-> > Fun = fun(_X) -> > ok > end, > > List = lists:seq(1, X), > timer:tc(fun()-> plists:foreach(Fun, List, {processes, schedulers}) end). > > tt2(X) -> > timer:tc(fun()-> lotsa:run(fun lotsa:sample_fun/1, X) end). > > > (loadtest@REDACTED)1> load_test:tt2(130000). > {15197488,done} > (loadtest@REDACTED)2> load_test:tt2(130000). > {15784610,done} > (loadtest@REDACTED)3> load_test:tt2(130000). > {17058412,done} > (loadtest@REDACTED)4> load_test:tt(130000). > {13020,ok} > (loadtest@REDACTED)5> load_test:tt(130000). > {10628,ok} > (loadtest@REDACTED)6> load_test:tt(130000). > {9918,ok} > > If {processes, schedulers}is not used performance are like in your code. > > Silviu > > > On Thu, Mar 10, 2016 at 8:13 PM, Felix Gallo wrote: > >> not sure which plist you're using, but the one that I could find looked >> not entirely erlang-idiomatic; it tried to manage the process scheduling >> itself, perhaps as a test implementation of the way that other languages >> might do it. >> >> Here's how I might tackle that problem in pure erlang. >> >> https://gist.github.com/anonymous/94ace7cd94fd5930f3e4 >> >> F. >> >> On Thu, Mar 10, 2016 at 8:45 AM, Caragea Silviu >> wrote: >> >>> No , >>> >>> I have a function: >>> >>> tt(X)-> >>> Fun = fun(_X) -> >>> ok >>> end, >>> >>> List = lists:seq(1, X), >>> plists:foreach(Fun, List). >>> >>> and I'm calling this one : module:tt(130000). >>> >>> Silviu >>> >>> >>> >>> On Thu, Mar 10, 2016 at 6:43 PM, Boroska Andr?s < >>> andras.boroska@REDACTED> wrote: >>> >>>> Do you run this in the interactive shell? In that case put it in a >>>> module and compile it. >>>> >>>> Measuring anything in the shell is not relevant. >>>> >>>> Andras >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Thu Mar 10 21:36:44 2016 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Thu, 10 Mar 2016 14:36:44 -0600 Subject: [erlang-questions] Built-in io:format syntax for printing lists? Message-ID: Could Erlang please offer a builtin way to print lists, e.g. for debugging purposes? Related: I swear there used to be "~a" for printing just about anything, but it appears to have been removed, no? -- Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Thu Mar 10 21:40:47 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 10 Mar 2016 14:40:47 -0600 Subject: [erlang-questions] Built-in io:format syntax for printing lists? In-Reply-To: References: Message-ID: I'm pretty sure "~w" or "~p" is what you're looking for. I believe the former will prevent printing the list as a string. On Thu, Mar 10, 2016 at 2:36 PM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > Could Erlang please offer a builtin way to print lists, e.g. for debugging > purposes? > > Related: I swear there used to be "~a" for printing just about anything, > but it appears to have been removed, no? > > -- > Cheers, > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Mar 10 21:43:05 2016 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 10 Mar 2016 12:43:05 -0800 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E1D491.9040908@gmail.com> References: <56E0E5F2.6030506@gmail.com> <56E1D491.9040908@gmail.com> Message-ID: <56E1DC59.9040803@gmail.com> On 03/10/2016 12:09 PM, Ryan wrote: > On 03/09/2016 09:11 PM, Michael Truog wrote: >> My understanding is that embedded is preferred for any production use, to make sure the system will fail-fast upon startup if there are any problems loading dependencies. The best time to have something fail is when it first starts, due to its lifetime being undefined and possibly infinite (ignoring heat death in the universe, and other natural disasters that should be written out of legal agreements :-). Ideally this concept is extended into the runtime of the server, into initialization and configuration source code, to make sure the server can fail-fast upon startup when the server is misconfigured, rather than waiting an arbitrary number of hours or days to find out that a problem exists. This approach helps to avoid a reliance on a fire-fighting mentality that becomes dependant on monitoring for feedback on a system's health due to the source code being a potentially unknowable black-box for some organizational reason. >> > This is a really good point, so let's clarify this. As I understand it, the difference between the two modes is that in embedded mode, all of the modules declared in your boot script are loaded at startup. To your point, if any modules happened to be missing, then startup would fail at this point. That's good. We like early failure. As far as I can tell, though, that's the only difference between embedded and interactive. After that initial "load all the modules", the next step would be to start all of your required applications. Whether by command line flag or boot script, that's going to progress the same way in either mode, assuming all the same modules are present. You talk about a initialization and configuration failing. Won't that cause identical issues in either mode, regardless of when the modules are loaded? I think all we're talking about here is whether code is loaded eagerly or lazily, and starting applications or processes works the same in either case. The embedded/interactive functionality really is focused on module loading either at startup or lazily, as you have described. I only mentioned initialization and configuration source code, due to how this fail-fast concept can be applied to source code. While it may seem that the embedded/interactive choice is not an important one, with execution generally happening in the same way, it can be important due to some code paths being infrequent and problems with the dependencies like modules with the same name (and unfortunately sometimes it then depends on the search directory order, which can lead to problems during execution that are counter-intuitive). > >> The interactive mode helps when testing, since things are loaded automatically and you can have less concern about the dependencies, since you are actively creating or modifying the dependencies. In production, you want an iron fist's control on all the dependencies, to make sure everything is repeatable and the service is stable, otherwise the development is pursuing stability in a serious way. So, that means that production usage should demand that all the dependencies come from a specific place at a specific version that is known and tracked, to be easily replicated, without any ambiguity. >> > Again, I completely agree that you want everything set in stone for production. I have a two-fold reply to your points. > > First, IMHO running two different ways in dev/test vs production only *increases* the chance that errors will slip in. Part of my quest here is to make it so that dev/test environments behave as similarly to production as possible so as to eliminate issues before they make it that far. I meant using interactive mode for manual usage of the Erlang shell, not real testing of a release. Only using interactive mode for development testing of random segments of Erlang source code. Even that usage of interactive mode can be problematic due to the undocumented differences between the Erlang shell execution and normal Erlang module execution. So, all releases for testing and production should be real releases running in embedded mode. The interactive mode just helps you quickly use the Erlang shell to check stuff. > > Second, as to having complete control over dependencies in production, I don't think I'm talking about dependency management, just about code loading. What dependencies get deployed where is part of the release process. That's different from the embedded/interactive discussion, isn't it? In fact, I know that running embedded doesn't protect you from dependency problems because just a few months ago, we had a production release go bad because of a dependency issue. Again, we currently run *embedded mode* in production. The problem was that some application wasn't explicitly declared as a dependency in the right .app file, and it wasn't included in the .rel file, so it didn't get bundled into the release. The odd thing was that a few modules from that application *did* make it into the release. I'm not sure whether it was rebar or OTP that was responsible for that, but it made for a very confusing situation, even for a couple of fairly experienced devs who have been on this > project for 2 years now. This same version ran *perfectly* in dev and test. It was only in production that the bug manifested. That can be weird. I know there can be problems with reltool including dependencies that are not dependencies of the main application, due to xref being used internally by reltool instead of just looking at the .app dependencies. That only affects using applications dynamically though, and doing that is uncommon. Normally all the Erlang applications are part of a static hierarchy and you only use a single boot file during the lifetime of the Erlang VM. > > Now, let me give a strong caveat that it's possible that we're misunderstanding something about how our release gets built, since it was someone else who wrote that part of the code, and he's no longer around. My point, though, is that dependency management is a build-time problem, isn't it? When you build a project, whether for testing or for production, that's when the right dependencies should be put in place. If you wait until runtime, you're too late. Stuff will crash. Yes, release building is a build-time concern, but making sure the release is ran in a dependable way is what relates to the embedded/interactive mode decision. Always using the embedded mode when a release is ran will help make sure the release is executed dependably. You may have the initial startup cost of loading all your modules due to embedded mode but that delay is very small with the Erlang VM and I have never seen it as a problem (even with an ARM and slow SSD memory). > > Thanks very much for your reply. I really want to understand the basic issues here. Please let me know if I'm totally off base here. I have many years of dev experience, but not much with Erlang, so I'm still trying to find my way around. > > Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Thu Mar 10 20:37:14 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 13:37:14 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <12463897.5JasyehX32@burrito> References: <12463897.5JasyehX32@burrito> Message-ID: <56E1CCEA.6010708@gmail.com> On 03/09/2016 08:32 PM, zxq9 wrote: > I've often wondered this myself. It is amazing and wonderful that Erlang is > able to deal so smoothly with the embedded case and even provides a huge > set of built in tools for getting systems running that way and manipulating > them -- but very few of us non-telecom folks are deploying to telephone > switches. > > I've written an entire system around dealing with Erlang code as signed > source bundles from a repo and dealing with the runtime as a more ordinary > language runtime which pulls and builds needed modules/version as necessary > within a given environment context (somewhat like Python's virtualenv). I > used this mostly for deploying client side code and keeping it up to date > without the hassle of building releases everywhere or making the users > themselves deal with constant "There's an upgrade! Install it or people > will think you have cooties!" messages (which they will disregard anyway). > > This was easier for new coders (coming from backgrounds where this is the > norm), small business system administrators (who are used to maintaining > runtime compliance with a given set of software they have to use), and > users (who are used to this sort of "I click it and things happen" utility). > > But it is clearly heresy to do this, so I tend to not mention it much. > I've even wondered if I should have been embarrassed to have committed > such sacrilege. > > The original version of this system is still in production, but behind > closed doors (>.<). It was a bit of a monolith anyway (grew into its own > in an, um, "organic" way as I added to it). I intend to reincarnate it > as a set of smallish utilities, but haven't yet had the time, and as far > as I can tell there is just about zero interest in the community for > anything but releases, so just haven't put much effort there. > > In the back of my mind I have always wondered: "This isn't so hard... why > isn't this considered normal?" And after experiencing no problems in > production I always wondered if I had just lucked out dodged some > horrible situation too foul to document by sheer chance -- or if this > really is indeed not a big deal and our obsession with releases is just > part of the tribal tradition. > > So... why? > Thanks a *ton* for that detailed feedback, Craig! I'm glad to know I'm not the only one questioning the norm, and it's really good to hear that at least one production system has run this way without obvious issues. I agree that interactive mode seems to be more natural for coders coming from other language backgrounds to understand. That way, "code path" can be understood roughly as "java class path" or "ruby gem home", etc. The system you're describing sounds interestingly like a for-fun project I built to dynamically build and execute new versions of code on demand for serving HTTP requests, depending on what code version the request asked for. I'm going to continue pursuing this idea, and unless someone can convince me otherwise, I'll try switching one of our production nodes to run interactive so I can compare its behavior to the others that are running embedded. Ryan From zzantozz@REDACTED Thu Mar 10 21:09:53 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 14:09:53 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E0E5F2.6030506@gmail.com> References: <56E0E5F2.6030506@gmail.com> Message-ID: <56E1D491.9040908@gmail.com> On 03/09/2016 09:11 PM, Michael Truog wrote: > My understanding is that embedded is preferred for any production use, > to make sure the system will fail-fast upon startup if there are any > problems loading dependencies. The best time to have something fail > is when it first starts, due to its lifetime being undefined and > possibly infinite (ignoring heat death in the universe, and other > natural disasters that should be written out of legal agreements :-). > Ideally this concept is extended into the runtime of the server, into > initialization and configuration source code, to make sure the server > can fail-fast upon startup when the server is misconfigured, rather > than waiting an arbitrary number of hours or days to find out that a > problem exists. This approach helps to avoid a reliance on a > fire-fighting mentality that becomes dependant on monitoring for > feedback on a system's health due to the source code being a > potentially unknowable black-box for some organizational reason. > This is a really good point, so let's clarify this. As I understand it, the difference between the two modes is that in embedded mode, all of the modules declared in your boot script are loaded at startup. To your point, if any modules happened to be missing, then startup would fail at this point. That's good. We like early failure. As far as I can tell, though, that's the only difference between embedded and interactive. After that initial "load all the modules", the next step would be to start all of your required applications. Whether by command line flag or boot script, that's going to progress the same way in either mode, assuming all the same modules are present. You talk about a initialization and configuration failing. Won't that cause identical issues in either mode, regardless of when the modules are loaded? I think all we're talking about here is whether code is loaded eagerly or lazily, and starting applications or processes works the same in either case. > The interactive mode helps when testing, since things are loaded > automatically and you can have less concern about the dependencies, > since you are actively creating or modifying the dependencies. In > production, you want an iron fist's control on all the dependencies, > to make sure everything is repeatable and the service is stable, > otherwise the development is pursuing stability in a serious way. So, > that means that production usage should demand that all the > dependencies come from a specific place at a specific version that is > known and tracked, to be easily replicated, without any ambiguity. > Again, I completely agree that you want everything set in stone for production. I have a two-fold reply to your points. First, IMHO running two different ways in dev/test vs production only *increases* the chance that errors will slip in. Part of my quest here is to make it so that dev/test environments behave as similarly to production as possible so as to eliminate issues before they make it that far. Second, as to having complete control over dependencies in production, I don't think I'm talking about dependency management, just about code loading. What dependencies get deployed where is part of the release process. That's different from the embedded/interactive discussion, isn't it? In fact, I know that running embedded doesn't protect you from dependency problems because just a few months ago, we had a production release go bad because of a dependency issue. Again, we currently run *embedded mode* in production. The problem was that some application wasn't explicitly declared as a dependency in the right .app file, and it wasn't included in the .rel file, so it didn't get bundled into the release. The odd thing was that a few modules from that application *did* make it into the release. I'm not sure whether it was rebar or OTP that was responsible for that, but it made for a very confusing situation, even for a couple of fairly experienced devs who have been on this project for 2 years now. This same version ran *perfectly* in dev and test. It was only in production that the bug manifested. Now, let me give a strong caveat that it's possible that we're misunderstanding something about how our release gets built, since it was someone else who wrote that part of the code, and he's no longer around. My point, though, is that dependency management is a build-time problem, isn't it? When you build a project, whether for testing or for production, that's when the right dependencies should be put in place. If you wait until runtime, you're too late. Stuff will crash. Thanks very much for your reply. I really want to understand the basic issues here. Please let me know if I'm totally off base here. I have many years of dev experience, but not much with Erlang, so I'm still trying to find my way around. Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Thu Mar 10 21:18:55 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 14:18:55 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: References: <56E0E5F2.6030506@gmail.com> Message-ID: <56E1D6AF.4000209@gmail.com> On 03/10/2016 12:28 AM, Max Lapshin wrote: > Have you ever looked in strace log when a moderate erlang system is > booting from flash on ARM? =) > I haven't, no. What would I see if I did? Profiling is a huge question in my mind! To me, it seems that's going to be the biggest issue in the embedded/interactive debate, and I'm currently trying to figure out what are the important characteristics of our system that I should measure in trying to change from embedded to interactive. > We (in flussonic) are not using embedded mode and don't use releases, > but we explicitly load all beams and it is faster then load them ondemand. How long have you been running a non-embedded node? Have you done any comparisons between embedded and interactive mode? The beam loading time is something I expected to be a notable difference between modes. Is the trouble with the actual loading of the bytecode or is it the time it takes to locate the beam file on the code path? Have you tried the "-code_path_cache" startup flag? It purports to make code lookup a constant-time operation: http://erlang.org/doc/man/code.html#id104406 When you say you "explicitly load all beams", how do you achieve that? Do you just use erlang:ensure_loaded(Module)? How do you come up with the list of modules to load eagerly? Boot script? App descriptor (.app) files? I'm very interested, because I expect I might want to do something similar. Ryan From zzantozz@REDACTED Thu Mar 10 22:39:21 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 15:39:21 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E1DC59.9040803@gmail.com> References: <56E0E5F2.6030506@gmail.com> <56E1D491.9040908@gmail.com> <56E1DC59.9040803@gmail.com> Message-ID: <56E1E989.7060903@gmail.com> On 03/10/2016 02:43 PM, Michael Truog wrote: > The embedded/interactive functionality really is focused on module > loading either at startup or lazily, as you have described. I only > mentioned initialization and configuration source code, due to how this > fail-fast concept can be applied to source code. While it may seem that > the embedded/interactive choice is not an important one, with execution > generally happening in the same way, it can be important due to some > code paths being infrequent and problems with the dependencies like > modules with the same name (and unfortunately sometimes it then depends > on the search directory order, which can lead to problems during > execution that are counter-intuitive). > I can totally understand the "modules with same name" problem, being from a Java background, where the same problem happens with class names. In this thread, a fellow named Max Lapshin indicated that he's using interactive mode and loading all modules at startup. Do you think that doing that would meet this concern of yours, assuming that during loading, one could detect if there were multiple modules of the same name on the code path? I'm not sure how to do that, but surely there's a way. > I meant using interactive mode for manual usage of the Erlang shell, not > real testing of a release. Only using interactive mode for development > testing of random segments of Erlang source code. Even that usage of > interactive mode can be problematic due to the undocumented differences > between the Erlang shell execution and normal Erlang module execution. > So, all releases for testing and production should be real releases > running in embedded mode. The interactive mode just helps you quickly > use the Erlang shell to check stuff. > I think this gets to the heart of what I'm trying to figure out: are there "undocumented differences" between interactive and embedded mode that are going to affect a long-running, production deployment? I'm assuming that what you refer to as "normal Erlang module execution" is embedded mode, and that's what this thread is all about. Why is embedded mode considered "normal"? Why can't we just run interactive mode everywhere? Assuming your concerns about dependencies can be addressed in a fail-fast way, is there really a good reason not to? > That can be weird. I know there can be problems with reltool including > dependencies that are not dependencies of the main application, due to > xref being used internally by reltool instead of just looking at the > .app dependencies. That only affects using applications dynamically > though, and doing that is uncommon. Normally all the Erlang applications > are part of a static hierarchy and you only use a single boot file > during the lifetime of the Erlang VM. > Ahh, xref would explain it. I think you're kind of proving my point about dependencies, though. If I can concretely say that my dependency list is [X,Y,Z], then why introduce the complexity overhead of the erlang release tools? They only seem to obfuscate things and introduce more opportunity for mistakes. They certainly have some surprising behavior built in. If I know my dependencies, then I should put the same set of dependencies in all environments, and load the code the same way everywhere. Isn't that the most foolproof way to ensure no surprises? As to the boot file, I haven't tested extensively, but I believe you can use a boot file with either embedded or interactive mode, and it works the same way except for the eager/lazy module loading. > Yes, release building is a build-time concern, but making sure the > release is ran in a dependable way is what relates to the > embedded/interactive mode decision. Always using the embedded mode when > a release is ran will help make sure the release is executed > dependably. You may have the initial startup cost of loading all your > modules due to embedded mode but that delay is very small with the > Erlang VM and I have never seen it as a problem (even with an ARM and > slow SSD memory). > I'm not concerned with the initial loading cost of modules. I'm concerned with bugs going to production. I think what you said here is what's the primary thrust of my question: "Always using the embedded mode when a release is ran will help make sure the release is executed dependably." That's the sort of assertion I've found scattered around the net, and that's what I'm questioning here. How do you know that? How can embedded be more dependable than interactive when the only difference is in eager vs. lazy module loading? Keep in mind that we agree that dependency management is a *build-time* concern, so the proper dependencies are theoretically guaranteed to be in place by the time the system is started. How is the runtime mode of the system going to affect its dependability? For follow-up, can you give me some concrete example(s) in which running in embedded mode would (did) prevent some issue that running in interactive wouldn't catch? From roberto.ostinelli@REDACTED Thu Mar 10 23:53:18 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Thu, 10 Mar 2016 23:53:18 +0100 Subject: [erlang-questions] mnesia async vs sync Message-ID: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> Dear list, To my understanding the main difference between the mnesia sync and async activities is that the latter does not wait for a confirmation from all the nodes that a write has been successful before returning. Thus, I would expect that the write speed of an async activity remains similar even when the number of node increases; however according to my benchmarks this is not true. The write speed decreases when the number of nodes increase. Can someone enlighten me of why this is? Thank you in advance. r. From zzantozz@REDACTED Thu Mar 10 23:58:14 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 16:58:14 -0600 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> Message-ID: <56E1FC06.9090702@gmail.com> On 03/10/2016 04:53 PM, Roberto Ostinelli wrote: > Dear list, > To my understanding the main difference between the mnesia sync and async activities is that the latter does not wait for a confirmation from all the nodes that a write has been successful before returning. > > Thus, I would expect that the write speed of an async activity remains similar even when the number of node increases; however according to my benchmarks this is not true. The write speed decreases when the number of nodes increase. > > Can someone enlighten me of why this is? > > Thank you in advance. > A quick glance through the mnesia module suggests that async doesn't mean 100% fire and forget. It appears that some cluster messaging is still performed before returning on a call with Kind=async. That would suggest that larger clusters will have slower performance. I don't know mnesia, so I can't give any further insight than that. From roberto.ostinelli@REDACTED Fri Mar 11 00:10:49 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 11 Mar 2016 00:10:49 +0100 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: <56E1FC06.9090702@gmail.com> References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> Message-ID: > On 10 mar 2016, at 23:58, Ryan wrote: > > A quick glance through the mnesia module suggests that async doesn't mean 100% fire and forget. It appears that some cluster messaging is still performed before returning on a call with Kind=async. That would suggest that larger clusters will have slower performance. I don't know mnesia, so I can't give any further insight than that. Thank you for your input. This is interesting since the documentation [1] states: "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context as the function mnesia:async_dirty/1,2. The difference is that the operations are performed synchronously. The caller waits for the updates to be performed on all active replicas." Maybe I'm not reading this right? Best, r. [1] http://erlang.org/doc/apps/mnesia/Mnesia_chap4.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Fri Mar 11 00:25:11 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 17:25:11 -0600 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> Message-ID: <56E20257.40304@gmail.com> On 03/10/2016 05:10 PM, Roberto Ostinelli wrote: > Thank you for your input. > > This is interesting since the documentation [1] states: > > "By passing the same "fun" as an argument to the function > mnesia:sync_dirty(Fun [, Args]) > , it is performed > in almost the same context as the function mnesia:async_dirty/1,2 > . The difference > is that the operations are performed synchronously. The caller waits > for the updates to be performed on all active replicas." > > Maybe I'm not reading this right? > I agree that those docs imply an async_dirty call doesn't wait for anything beyond the one node. It specifically says, under async_dirty, that, "The functions wait for the operation to be performed on one node but not the others. If the table resides locally, no waiting occurs." I may have misinterpreted the code I was looking at, or the docs could be misleading. I'll wait eagerly for someone more well-informed to come along and shed some light on the situation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Mar 11 00:53:13 2016 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 10 Mar 2016 15:53:13 -0800 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E1E989.7060903@gmail.com> References: <56E0E5F2.6030506@gmail.com> <56E1D491.9040908@gmail.com> <56E1DC59.9040803@gmail.com> <56E1E989.7060903@gmail.com> Message-ID: <56E208E9.7070600@gmail.com> On 03/10/2016 01:39 PM, Ryan wrote: > On 03/10/2016 02:43 PM, Michael Truog wrote: >> The embedded/interactive functionality really is focused on module >> loading either at startup or lazily, as you have described. I only >> mentioned initialization and configuration source code, due to how this >> fail-fast concept can be applied to source code. While it may seem that >> the embedded/interactive choice is not an important one, with execution >> generally happening in the same way, it can be important due to some >> code paths being infrequent and problems with the dependencies like >> modules with the same name (and unfortunately sometimes it then depends >> on the search directory order, which can lead to problems during >> execution that are counter-intuitive). >> > I can totally understand the "modules with same name" problem, being from a Java background, where the same problem happens with class names. In this thread, a fellow named Max Lapshin indicated that he's using interactive mode and loading all modules at startup. Do you think that doing that would meet this concern of yours, assuming that during loading, one could detect if there were multiple modules of the same name on the code path? I'm not sure how to do that, but surely there's a way. I only believe interactive mode should be used when using the Erlang shell manually, or when running CT tests or eunit tests. This is to make sure everything is fail-fast. I understand you are concerned with concrete justification of that, which I will have below. Dealing with modules that have the same name is something that should be automatically caught when a release is built, but there are probably situations when that doesn't happen. At the very least release creation should fail when seeing the same application in separate paths, during an attempt to build the release. So it should be easy to see that release creation helps avoid errors, instead of haphazard loading of modules randomly (due to the function call path) with interactive mode. > >> I meant using interactive mode for manual usage of the Erlang shell, not >> real testing of a release. Only using interactive mode for development >> testing of random segments of Erlang source code. Even that usage of >> interactive mode can be problematic due to the undocumented differences >> between the Erlang shell execution and normal Erlang module execution. >> So, all releases for testing and production should be real releases >> running in embedded mode. The interactive mode just helps you quickly >> use the Erlang shell to check stuff. >> > I think this gets to the heart of what I'm trying to figure out: are there "undocumented differences" between interactive and embedded mode that are going to affect a long-running, production deployment? I'm assuming that what you refer to as "normal Erlang module execution" is embedded mode, and that's what this thread is all about. Why is embedded mode considered "normal"? Why can't we just run interactive mode everywhere? Assuming your concerns about dependencies can be addressed in a fail-fast way, is there really a good reason not to? The "undocumented differences" I am referring to are specific to using the Erlang shell for executing Erlang source code when compared to compiled BEAM in Erlang modules. Some execution differences exist, which should discourage any usage of the Erlang shell for any type of production deployment, even if it is only for a testing host. The embedded/interactive mode concern is separate and focused on how modules are loaded, but interactive is the default for normal Erlang shell usage. > >> That can be weird. I know there can be problems with reltool including >> dependencies that are not dependencies of the main application, due to >> xref being used internally by reltool instead of just looking at the >> .app dependencies. That only affects using applications dynamically >> though, and doing that is uncommon. Normally all the Erlang applications >> are part of a static hierarchy and you only use a single boot file >> during the lifetime of the Erlang VM. >> > Ahh, xref would explain it. I think you're kind of proving my point about dependencies, though. If I can concretely say that my dependency list is [X,Y,Z], then why introduce the complexity overhead of the erlang release tools? They only seem to obfuscate things and introduce more opportunity for mistakes. They certainly have some surprising behavior built in. If I know my dependencies, then I should put the same set of dependencies in all environments, and load the code the same way everywhere. Isn't that the most foolproof way to ensure no surprises? This is a surprise if you include Erlang application that aren't explicitly started by the release. The release is a necessary thing to make sure all the files are in a single place, to be used together, so they can be managed as a release. You should understand that having a release is beneficial from other development outside of Erlang. If you don't believe in having releases, then all hope may be lost :-) This is really about managing how software changes to avoid risk. You may think you are fixing source code by changing it, but often you are really breaking something else. You should not need concrete evidence of this, since you are human and not perfect like the computer :-) (i.e., doing only what we tell it). > > As to the boot file, I haven't tested extensively, but I believe you can use a boot file with either embedded or interactive mode, and it works the same way except for the eager/lazy module loading. > >> Yes, release building is a build-time concern, but making sure the >> release is ran in a dependable way is what relates to the >> embedded/interactive mode decision. Always using the embedded mode when >> a release is ran will help make sure the release is executed >> dependably. You may have the initial startup cost of loading all your >> modules due to embedded mode but that delay is very small with the >> Erlang VM and I have never seen it as a problem (even with an ARM and >> slow SSD memory). >> > I'm not concerned with the initial loading cost of modules. I'm concerned with bugs going to production. I think what you said here is what's the primary thrust of my question: "Always using the embedded mode when a release is ran will help make sure the release is executed dependably." > > That's the sort of assertion I've found scattered around the net, and that's what I'm questioning here. How do you know that? How can embedded be more dependable than interactive when the only difference is in eager vs. lazy module loading? Keep in mind that we agree that dependency management is a *build-time* concern, so the proper dependencies are theoretically guaranteed to be in place by the time the system is started. How is the runtime mode of the system going to affect its dependability? > > For follow-up, can you give me some concrete example(s) in which running in embedded mode would (did) prevent some issue that running in interactive wouldn't catch? If there are any problems with the filesystem that are intermittent, you may see them randomly with interactive mode, but at least you would see them on startup with embedded mode. Some modules, like NIFs, can do stuff when they load, and that is best done all at once, to see if something breaks, rather than waiting an undefined amount of time later to find out about your problem in testing or production. If execution changes based on the modules being loaded, that can make the situation more complex. Having a known starting state is important for having dependable results in the system, and interactive mode only makes your starting state random based on how you change the build and source code. I am not sure if that is concrete enough, but having the system be fail-fast is a pretty fundamental concept. I have worked on a system that took a few days to die (each time requiring a few days, with specific input) due to a single bug, and that can quickly teach you that fail-fast is beneficial. Trying to solve all problems as a practice of fire-fighting doesn't lead to a dependable service, it just creates drama that consumes extra time and money, even if it was meant as a method of feeding egos. From zzantozz@REDACTED Fri Mar 11 01:02:29 2016 From: zzantozz@REDACTED (Ryan) Date: Thu, 10 Mar 2016 18:02:29 -0600 Subject: [erlang-questions] Embedded vs Interactive - Why embedded? In-Reply-To: <56E208E9.7070600@gmail.com> References: <56E0E5F2.6030506@gmail.com> <56E1D491.9040908@gmail.com> <56E1DC59.9040803@gmail.com> <56E1E989.7060903@gmail.com> <56E208E9.7070600@gmail.com> Message-ID: <56E20B15.4000308@gmail.com> On 03/10/2016 05:53 PM, Michael Truog wrote: > On 03/10/2016 01:39 PM, Ryan wrote: >> On 03/10/2016 02:43 PM, Michael Truog wrote: >>> The embedded/interactive functionality really is focused on module >>> loading either at startup or lazily, as you have described. I only >>> mentioned initialization and configuration source code, due to how this >>> fail-fast concept can be applied to source code. While it may seem that >>> the embedded/interactive choice is not an important one, with execution >>> generally happening in the same way, it can be important due to some >>> code paths being infrequent and problems with the dependencies like >>> modules with the same name (and unfortunately sometimes it then depends >>> on the search directory order, which can lead to problems during >>> execution that are counter-intuitive). >>> >> I can totally understand the "modules with same name" problem, being >> from a Java background, where the same problem happens with class >> names. In this thread, a fellow named Max Lapshin indicated that he's >> using interactive mode and loading all modules at startup. Do you >> think that doing that would meet this concern of yours, assuming that >> during loading, one could detect if there were multiple modules of the >> same name on the code path? I'm not sure how to do that, but surely >> there's a way. > > I only believe interactive mode should be used when using the Erlang > shell manually, or when running CT tests or eunit tests. This is to > make sure everything is fail-fast. I understand you are concerned with > concrete justification of that, which I will have below. Dealing with > modules that have the same name is something that should be > automatically caught when a release is built, but there are probably > situations when that doesn't happen. At the very least release creation > should fail when seeing the same application in separate paths, during > an attempt to build the release. So it should be easy to see that > release creation helps avoid errors, instead of haphazard loading of > modules randomly (due to the function call path) with interactive mode. > >> >>> I meant using interactive mode for manual usage of the Erlang shell, not >>> real testing of a release. Only using interactive mode for development >>> testing of random segments of Erlang source code. Even that usage of >>> interactive mode can be problematic due to the undocumented differences >>> between the Erlang shell execution and normal Erlang module execution. >>> So, all releases for testing and production should be real releases >>> running in embedded mode. The interactive mode just helps you quickly >>> use the Erlang shell to check stuff. >>> >> I think this gets to the heart of what I'm trying to figure out: are >> there "undocumented differences" between interactive and embedded mode >> that are going to affect a long-running, production deployment? I'm >> assuming that what you refer to as "normal Erlang module execution" is >> embedded mode, and that's what this thread is all about. Why is >> embedded mode considered "normal"? Why can't we just run interactive >> mode everywhere? Assuming your concerns about dependencies can be >> addressed in a fail-fast way, is there really a good reason not to? > > The "undocumented differences" I am referring to are specific to using > the Erlang shell for executing Erlang source code when compared to > compiled BEAM in Erlang modules. Some execution differences exist, > which should discourage any usage of the Erlang shell for any type of > production deployment, even if it is only for a testing host. The > embedded/interactive mode concern is separate and focused on how modules > are loaded, but interactive is the default for normal Erlang shell usage. > >> >>> That can be weird. I know there can be problems with reltool including >>> dependencies that are not dependencies of the main application, due to >>> xref being used internally by reltool instead of just looking at the >>> .app dependencies. That only affects using applications dynamically >>> though, and doing that is uncommon. Normally all the Erlang applications >>> are part of a static hierarchy and you only use a single boot file >>> during the lifetime of the Erlang VM. >>> >> Ahh, xref would explain it. I think you're kind of proving my point >> about dependencies, though. If I can concretely say that my dependency >> list is [X,Y,Z], then why introduce the complexity overhead of the >> erlang release tools? They only seem to obfuscate things and introduce >> more opportunity for mistakes. They certainly have some surprising >> behavior built in. If I know my dependencies, then I should put the >> same set of dependencies in all environments, and load the code the >> same way everywhere. Isn't that the most foolproof way to ensure no >> surprises? > > This is a surprise if you include Erlang application that aren't > explicitly started by the release. The release is a necessary thing to > make sure all the files are in a single place, to be used together, so > they can be managed as a release. You should understand that having a > release is beneficial from other development outside of Erlang. If you > don't believe in having releases, then all hope may be lost :-) This is > really about managing how software changes to avoid risk. You may think > you are fixing source code by changing it, but often you are really > breaking something else. You should not need concrete evidence of this, > since you are human and not perfect like the computer :-) (i.e., doing > only what we tell it). > >> >> As to the boot file, I haven't tested extensively, but I believe you >> can use a boot file with either embedded or interactive mode, and it >> works the same way except for the eager/lazy module loading. >> >>> Yes, release building is a build-time concern, but making sure the >>> release is ran in a dependable way is what relates to the >>> embedded/interactive mode decision. Always using the embedded mode when >>> a release is ran will help make sure the release is executed >>> dependably. You may have the initial startup cost of loading all your >>> modules due to embedded mode but that delay is very small with the >>> Erlang VM and I have never seen it as a problem (even with an ARM and >>> slow SSD memory). >>> >> I'm not concerned with the initial loading cost of modules. I'm >> concerned with bugs going to production. I think what you said here is >> what's the primary thrust of my question: "Always using the embedded >> mode when a release is ran will help make sure the release is executed >> dependably." >> >> That's the sort of assertion I've found scattered around the net, and >> that's what I'm questioning here. How do you know that? How can >> embedded be more dependable than interactive when the only difference >> is in eager vs. lazy module loading? Keep in mind that we agree that >> dependency management is a *build-time* concern, so the proper >> dependencies are theoretically guaranteed to be in place by the time >> the system is started. How is the runtime mode of the system going to >> affect its dependability? >> >> For follow-up, can you give me some concrete example(s) in which >> running in embedded mode would (did) prevent some issue that running >> in interactive wouldn't catch? > > If there are any problems with the filesystem that are intermittent, you > may see them randomly with interactive mode, but at least you would see > them on startup with embedded mode. Some modules, like NIFs, can do > stuff when they load, and that is best done all at once, to see if > something breaks, rather than waiting an undefined amount of time later > to find out about your problem in testing or production. If execution > changes based on the modules being loaded, that can make the situation > more complex. Having a known starting state is important for having > dependable results in the system, and interactive mode only makes your > starting state random based on how you change the build and source code. > > I am not sure if that is concrete enough, but having the system be > fail-fast is a pretty fundamental concept. I have worked on a system > that took a few days to die (each time requiring a few days, with > specific input) due to a single bug, and that can quickly teach you that > fail-fast is beneficial. Trying to solve all problems as a practice of > fire-fighting doesn't lead to a dependable service, it just creates > drama that consumes extra time and money, even if it was meant as a > method of feeding egos. > I hear what you're saying, and I need to think about it for a little while. My one thought right now is that the whole "fail fast on load" thing can still be achieved in interactive mode if you manually ensure all modules are loaded at startup. You don't need the full release and embedded mode to do that. From khitai.pang@REDACTED Fri Mar 11 05:48:20 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Fri, 11 Mar 2016 12:48:20 +0800 Subject: [erlang-questions] net_kernel:start/1 doesn't honor ERL_EPMD_PORT set by os:putenv/2 Message-ID: Hi, I have the following code: os:putenv("ERL_EPMD_ADDRESS", inet_parse:ntoa(IP)), os:putenv("ERL_EPMD_PORT", integer_to_list(5555)), os:cmd("epmd -daemon"), net_kernel:start([bob, longnames]), The ERL_EPMD_ADDRESS and ERL_EPMD_PORT environment variables are used by os:cmd("epmd -daemon") but net_kernel:start/1 fails: Protocol: "inet_tcp": register/listen error: econnrefused Apparently it still tries to connect to 4369, where no process is listening. If I start erl by "ERL_EPMD_PORT=5555 erl", net_kernel:start/1 in the above code works fine. So I have come to the conclusion that environment variables set by os:putenv/2 are not honored by net_kernel:start/1. Am I right? If yes, 1) is this a bug? 2) for a Erlang/OTP release, how to automatically set global environment variables for erl? Thanks Khitai From gleber.p@REDACTED Fri Mar 11 11:57:46 2016 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 11 Mar 2016 11:57:46 +0100 Subject: [erlang-questions] [ANN] Rebar3 first stable release 3.0.0 In-Reply-To: References: <1457547704.3409742.544432058.42894F4A@webmail.messagingengine.com> Message-ID: This is exciting! Is the official git repo nowadays https://github.com/erlang/rebar3 ? Does this mean that rebar is endorsed by OTP team? On Wed, Mar 9, 2016 at 8:02 PM, Jesse Gumm wrote: > Awesome! Looking forward to transitioning from rebar2 with my stuff. > > Thanks for all the hard work to make this happen. Following from the > sidelines on the rebar mailing list has been informative. Now to just get my > hands dirty after all the hard work is done! > > Cheers! > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > > On Mar 9, 2016 12:22 PM, "Tristan Sloughter" wrote: >> >> We are happy to announce the first stable release of rebar3 with version >> 3.0.0. >> >> A bit over a year ago, we had set out to make rebar great again. We >> believe we have done it, and now have a solid core on which to build more >> tools and grow a better community. >> >> Now, this doesn't mean there are no things to improve or that the code >> base is bug-free. It does however mean that we are now committed to >> backwards compatibility as we move forward with rebar3's next improvements. >> >> If you have never tried rebar3, the big item names since rebar 2.x are: >> >> repeatable builds with a big commitment to OTP principles >> commands that know their own dependencies >> support for profiles, allowing contextual configuration of applications >> (for example, test dependencies can be kept apart from regular deps) >> documentation is seen as a feature >> support for packages through hex.pm >> new plugin interface and mechanisms, with command namespaces >> mechanisms to override dependencies' configurations >> switching releases to relx >> and a lot of improvements to existing commands such as templates, >> dialyzer, shell, and so on. >> >> You will find documentation and how to get started with rebar3 on the >> website: http://www.rebar3.org/docs/getting-started >> >> For package publishing checkout the documentation on hex.pm: >> https://hex.pm/docs/rebar3_publish >> >> If you feel like getting started, we have had numerous posts about the new >> features and user experiences: >> >> http://ferd.ca/dev/rebar3-shell.html >> http://blog.erlware.org/rebar3-features-part-1-local-install-and-upgrade/ >> http://blog.erlware.org/rebar3-features-part-2-dependency-tree/ >> http://blog.erlware.org/rebar3-features-part-3-overrides/ >> http://blog.erlware.org/rebar3-features-part-4-profiles/ >> http://blog.erlware.org/rebar3-features-part-5-dependency-branch-handling/ >> http://blog.erlware.org/rebar3-features-part-6-_checkouts-2/ >> http://blog.erlware.org/rebar3-hex-plugin/ >> >> Heinz Gies wrote about his experience migrating to rebar3 form rebar2 at >> http://blog.licenser.net/blog/2015/07/10/migrating-to-rebar3/ >> >> The How I Start Erlang guide was updated to use rebar3: >> https://howistart.org/posts/erlang/1 >> >> And a guide about managing package has been written by Bruce Yinhe: >> https://medium.com/@brucify/using-rebar3-to-manage-erlang-packages-282f78adff1e#.ay10goc0z >> >> Lastly, for those who have already adopted the tool, here is what has >> changed since beta-4: >> >> rebar3/902: Less color >> rebar3/921: Fix error reports on missing include paths >> rebar3/922: warn on incorrectly specified test options in rebar.config >> rebar3/923: support temporary cdn change with HEX_CDN os var >> rebar3/924: only add package list of versions to registry if it has the >> right build tool support >> rebar3/925: upgrade eunit_formatters for OTP 18 support >> rebar3/927: upgrade eunit_formatters to 0.3.1 for OTP18 bug fix >> rebar3/928: Ct output improvements >> rebar3/930: Handle force flags in leading position >> rebar3/934: don't add a provider if the same namespace and name already >> exists >> rebar3/935: Default to no eunit formatter if verbose specified >> rebar3/941: preserve attributes when copying files in rebar_utils:cp_r for >> unix >> rebar3/942: special handling of relx configs in profiles >> rebar3/943: remove backward_compat entry from travis s3 >> rebar3/945: auto-update the registry if a pkg isn't found, fail if it >> still isn't found >> rebar3/948: Fix a small bug in the MIB compiler when building dependencies >> rebar3/949: fetch eunit_formatters config not from the command args but >> from the config >> rebar3/963: Tup merge tests >> rebar3/965: fix tupple merging for realsies. >> rebar3/966: allow ct suites to be specified at root of project (or root of >> app) >> rebar3/967: symlink mib hrl output in apps `include' directories >> rebar3/970: upgrade certifi to latest release >> rebar3/982: Allow bootstrap to pick up existing Hex cache and deps >> rebar3/983: Add support for total code coverage >> rebar3/986: A bad template index does not crash; shows warning >> rebar3/987: Plugin templates >> rebar3/988: Fix wrong relative path resolution >> rebar3/989: convert ~> versions to highest matching >> rebar3/991: Upgrade relx to v3.11.0 >> rebar3/992: Bump cth_readable to 1.1.1 >> rebar3/993: contributors -> maintaiers in template app data >> rebar3/994: warn if the directories eunit' orct' show up in `src_dirs' >> rebar3/995: Support old-style shell for rebar3 shell >> rebar3/996: Updates cth_readable so it supports dumb terminals >> rebar3/999: Fix windows stuff >> rebar3/1005: only need to compare ref and not ref+url in git resource >> rebar3/1006: only apply default and prod profile to dependencies >> rebar3/1007: install project app plugins after discovering them not before >> rebar3/1008: Unquote templates, add a warning instead. >> rebar3/1009: merge overlay entries into a single {overlay, list()} for >> relx >> rebar3/1010: upgrade relx to 3.12.0 >> rebar3/1012: Remove triple brackets in bbmustache templates >> rebar3/1013: upgrade bbmustache and relx >> rebar3/1016: when using the --file' argument toeunit' paths were being >> converted >> rebar3/1017: check at runtime instead of compile time for >> file:list_dir_all/1 >> rebar3/1018: change detection of valid modules for eunit >> rebar3/1021: convert 'app' to 'application' in eunit_opts to match cmdline >> args >> rebar3/1022: Display error message when bad config is loaded >> rebar3/1023: Rework README and CONTRIBUTING documents >> rebar3/1024: deduplicate default test set generated by rebar3 eunit >> rebar3/1025: add unstable install/upgrade instructions to readme >> rebar3/1029: local install and local upgrade >> rebar3/1031: add profile option to clean task >> rebar3/1033: Fix bash completion regression (cf66dfd6ba) and make lopt >> strings more resilient >> rebar3/1035: Add module directory to include path >> rebar3/1041: Add test case for relx overlay vars >> rebar3/1043: don't strip the project apps when running ct with just a root >> suite specified >> rebar3/1045: don't lose overrides in an app when installing plugins it >> uses >> rebar3/1046: add user-agent to http request headers >> rebar3/1047: Ignore unknown warning when dialyzer < 2.8 >> rebar3/1048: Add secondary hook for rebar_prv_compile >> rebar3/1050: check top level config for minimum or blacklisted otps at >> start >> rebar3/1053: upgrade relx to 3.15.0 >> rebar3/1061: Make lock files future-proof >> rebar3/1062: update relx to 3.16.0 to include color intesity update >> rebar3/1065: give top level plugin providers precedence over default >> providers >> rebar3/1066: Bump cth_readable >> rebar3/1067: set default color intensity to high >> rebar3/1068: upgrade relx to 3.17.0 >> rebar3/1070: Hex improvements >> rebar3/1071: break up do/1 function in install_deps to make upgrade less >> confusing >> rebar3/1073: Support --setcookie option >> rebar3/1075: add project_providers after initing default providers but >> allow overrides >> rebar3/1076: Add me to THANKS for suffering through Tristan's code >> rebar3/1082: Fixed rebar.config shell_apps setting >> rebar3/1090: fix auto-registry update to work even when not a locked >> pkg-vsn >> rebar3/1091: Run all hooks >> rebar3/1092: Add short-options to the eunit provider. >> rebar3/1098: error on a cover spec in ct_opts >> rebar3/1099: add support for common tests include flag >> rebar3/1100: Fix quoting problem in zsh completion >> rebar3/1101: Take CT options errors and turn them to warnings >> rebar3/1102: include project_plugins in plugins that can be upgraded >> rebar3/1103: bump certifi to 0.4.0 >> rebar3/1104: move dialyze setting of debug_info to overrides in profile >> rebar3/1106: define the 'EUNIT' macro in the test profile >> rebar3/1107: upgrade cth_readable >> rebar3/1108: make omar happy >> rebar3/1110: pass loglevel used in rebar3 to relx >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Fri Mar 11 14:51:18 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 11 Mar 2016 14:51:18 +0100 Subject: [erlang-questions] Built-in io:format syntax for printing lists? In-Reply-To: References: Message-ID: This looks like common lisp. In CL, ~a is the "print anything" marker: http://www.lispworks.com/documentation/lw50/CLHS/Body/22_cda.htm i.e., FORMAT control-strings use ~a for "Aesthetic" printing. Perhaps this is messed up with the ~p and ~w Sean mentions. On Thu, Mar 10, 2016 at 9:40 PM, Sean Cribbs wrote: > I'm pretty sure "~w" or "~p" is what you're looking for. I believe the > former will prevent printing the list as a string. > > On Thu, Mar 10, 2016 at 2:36 PM, Andrew Pennebaker < > andrew.pennebaker@REDACTED> wrote: > >> Could Erlang please offer a builtin way to print lists, e.g. for >> debugging purposes? >> >> Related: I swear there used to be "~a" for printing just about anything, >> but it appears to have been removed, no? >> >> -- >> Cheers, >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.carrone@REDACTED Fri Mar 11 17:48:55 2016 From: federico.carrone@REDACTED (Federico Carrone) Date: Fri, 11 Mar 2016 16:48:55 +0000 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: <291389291.rCvldqcINk@burrito> References: <1456920310.17621.32.camel@free.fr> <291389291.rCvldqcINk@burrito> Message-ID: Hi, I already started working on this after getting a few useful comments from Kenneth. I hope to have a simple HTML demo to get feedback from the community in one month or two. After getting feedback I will work on the pull request. If you have any comments/ideas please send me an email. Regards, Federico --- FYI I am using the recommendations of: http://practicaltypography.com/ And as Inspiration I am using: *Rust docs* https://doc.rust-lang.org/std/slice/ *Python documents redesigned* http://practicaltypography.com/python-docs-after/index.html *Racket docs* https://docs.racket-lang.org/guide/Lists__Iteration__and_Recursion.html *Elixir docs* http://elixir-lang.org/docs/stable/elixir/List.html#summary Regards, Federico. On Wed, Mar 2, 2016 at 9:58 AM zxq9 wrote: > On Wednesday 02 March 2016 12:38:52 Ali Sabil wrote: > > Thank you for bringing this up. An improvement to the presentation of the > > Erlang documentation is very much needed. > > > > I have started looking at this myself sometime ago, here is what I > gathered: > > ? The OTP documentation is using erl_docgen which is mostly XSLT > based > > ? Edoc is not that much used in OTP, the applications that do rely on > > it, use a custom layout (otpsgml_layout) to generate output that can be > > consumed by erl_docgen. > > ? Experimented with adding a modern HTML5 output to erl_docgen here: > > https://github.com/asabil/otp/tree/feature/html5-docs but this proved > > itself challenging. > > ? Started working on a modern doclet for edoc, based on the ex_doc > > layout: https://bitbucket.org/asabil/edoc-modern > > Hi Ali, > > I ran into the exact same thing and concluded it would be easier to treat > rendered docs as their own data source rather than try to decobble all the > mix-and-match that exists within the OTP sources themselves (though the > *right* approach would be to actually make the docs across OTP uniform -- > I felt this would be futile to even attempt alone and unsupported, who has > that much (un)paid time on their hands?). > > Tricky, to say the least. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Fri Mar 11 18:10:17 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Fri, 11 Mar 2016 09:10:17 -0800 Subject: [erlang-questions] HTML/CSS/Javascript in Erlang docs and Common Test In-Reply-To: References: <1456920310.17621.32.camel@free.fr> <291389291.rCvldqcINk@burrito> Message-ID: Have a look at https://github.com/erldocs/erldocs/blob/master/src/erldocs_core.erl It already goes through the trouble of reading docs from src or doc/src and applies some transformation before rendering HTML from XML. I have a Markdown-from-XML renderer in the works too. > On 11 Mar 2016, at 08:48, Federico Carrone wrote: > > Hi, > > I already started working on this after getting a few useful comments from Kenneth. > > I hope to have a simple HTML demo to get feedback from the community in one month or two. After getting feedback I will work on the pull request. > > If you have any comments/ideas please send me an email. > > Regards, > Federico > --- > > FYI I am using the recommendations of: > http://practicaltypography.com/ > > And as Inspiration I am using: > Rust docs > https://doc.rust-lang.org/std/slice/ > > Python documents redesigned > http://practicaltypography.com/python-docs-after/index.html > > Racket docs > https://docs.racket-lang.org/guide/Lists__Iteration__and_Recursion.html > > Elixir docs > http://elixir-lang.org/docs/stable/elixir/List.html#summary > > Regards, > Federico. > >> On Wed, Mar 2, 2016 at 9:58 AM zxq9 wrote: >> On Wednesday 02 March 2016 12:38:52 Ali Sabil wrote: >> > Thank you for bringing this up. An improvement to the presentation of the >> > Erlang documentation is very much needed. >> > >> > I have started looking at this myself sometime ago, here is what I gathered: >> > ? The OTP documentation is using erl_docgen which is mostly XSLT based >> > ? Edoc is not that much used in OTP, the applications that do rely on >> > it, use a custom layout (otpsgml_layout) to generate output that can be >> > consumed by erl_docgen. >> > ? Experimented with adding a modern HTML5 output to erl_docgen here: >> > https://github.com/asabil/otp/tree/feature/html5-docs but this proved >> > itself challenging. >> > ? Started working on a modern doclet for edoc, based on the ex_doc >> > layout: https://bitbucket.org/asabil/edoc-modern >> >> Hi Ali, >> >> I ran into the exact same thing and concluded it would be easier to treat >> rendered docs as their own data source rather than try to decobble all the >> mix-and-match that exists within the OTP sources themselves (though the >> *right* approach would be to actually make the docs across OTP uniform -- >> I felt this would be futile to even attempt alone and unsupported, who has >> that much (un)paid time on their hands?). >> >> Tricky, to say the least. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Sat Mar 12 21:02:45 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Sat, 12 Mar 2016 22:02:45 +0200 Subject: [erlang-questions] some help with hackney and multipart In-Reply-To: References: Message-ID: I tested and what I see is that hackeny also use for files the name "file". I don't see how I can customize this as time mailgun search for attachment. Also looking to https://github.com/benoitc/hackney/blob/de35c1ec93dca89568b8b32cf053d82277a90762/src/hackney_multipart.erl line 228 this seems hard-coded: Disposition = {<<"form-data">>, [{<<"name">>, <<"\"file\"">>}, {<<"filename">>, <<"\"", FName/binary, "\"">>}]}, Silviu On Wed, Mar 9, 2016 at 12:55 PM, Hynek Vychodil wrote: > Hi, > Try compare what is sent in body of request made by example with your one > sent by hackney > > curl -s --user 'api:YOUR_API_KEY' \ > https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \ > -F from='Excited User ' \ > -F to='foo@REDACTED' \ > -F cc='bar@REDACTED' \ > -F bcc='baz@REDACTED' \ > -F subject='Hello' \ > -F text='Testing some Mailgun awesomness!' \ > --form-string html='HTML version of the body' \ > -F attachment=@files/cartman.jpg \ > -F attachment=@files/cartman.png > > Your {multipart, PayloadBase ++ [{file, Attachment}]} doesn't make much > sense to me. Try something like {multipart, [{form, PayloadBase},{file, > Attachment}]} (untested). > > Hynek > > On Wed, Mar 9, 2016 at 11:31 AM, Caragea Silviu > wrote: > >> >> >> Hello, >> >> I'm trying to send an email via mailgun.com using the hackney and I have >> some issues sending attachments (which requires multipart). >> >> https://documentation.mailgun.com/api-sending.html#sending >> >> Basically my interest fields are: >> >> from >> to >> subject >> text >> attachment File attachment. You can post multiple attachment values. >> Important: You must use multipart/form-data encoding when sending >> attachments. >> >> I tried the following: >> >> PayloadBase =[ >> {<<"from">>, From}, >> {<<"to">>, To}, >> {<<"subject">>, Subject}, >> {<<"text">>, TextBody}, >> {<<"html">>, HtmlBody} >> ], >> >> Payload = case Attachment of >> null -> >> {form, PayloadBase}; >> _-> >> {multipart, PayloadBase ++ [{file, Attachment}]} >> end, >> >> But for some reason the attachment is not sent.. Everything else works as >> expected. >> I don't see how I can set the filed name to "attachment" as required by >> mailgun .. at this this is what I suspect beeing wrong >> >> Silviu >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Mar 13 23:40:12 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 13 Mar 2016 23:40:12 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <3BBAB72D-071E-41C7-B173-8127C0BB824F@gmail.com> References: <3BBAB72D-071E-41C7-B173-8127C0BB824F@gmail.com> Message-ID: Don't forget #m(...) for literal maps and #"..." for utf-8 encoded binary strings. Actually they are all well-defined. The #(...), #b(...) #m(...) define data types based on the CL syntax for vectors #(...). The others are also taken from standard CL. If you want to add these datatypes and constructs, which I do, then you are going to add new syntax however you do it. Also these are all literal values and not forms. It is the forms being extremely consistent which make writing macros very simple. It also allows you to generate "new" syntax which Elixir does not. Robert On 7 March 2016 at 15:42, Anthony Ramine wrote: > Homoiconicity is way overrated when it comes to metaprogamming. There is > nothing that Lisp does with parentheses that Elixir cannot with the 'do' > token. > > And no Lisp is truly simple or consistent. Not LFE at least. > > lists:map > #'list_to_integer/1 > #*0 > #o111 > +1.0 > #B(?) > #(?) > ; comments > #.(?) > > There is nothing simple nor consistent about this. > > Regards. > > Le 6 mars 2016 ? 05:13, Robert Virding a ?crit : > > I think if you seriously want to push macros then LFE, or any other lisp > for that matter, is a much better option. It is much more straight-forward > and readable. It also gives you the option of creating new syntax which > Elixir macros don't. Elixir has a slightly "lax" syntax which allows you to > hide that everything is a function call but you can't create new syntax. > > Of course some people lisp doesn't actually have any syntax but I prefer > to view it as being very simple and very consistent. :-) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Mar 14 12:39:28 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 14 Mar 2016 12:39:28 +0100 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: <56E20257.40304@gmail.com> References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> <56E20257.40304@gmail.com> Message-ID: So, I took the time to benchmark a little more. I'm hoping that some kind soul can help me understand this here. As said, fundamentally I don't see any speed difference when using sync_dirty and async_dirty. The more nodes you add, the slower insertions will be even if the docs for async_dirty state [1]: "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context as the function mnesia:async_dirty/1,2. The difference is that the operations are performed synchronously. The caller waits for the updates to be performed on all active replicas." "The functions wait for the operation to be performed on one node but not the others. If the table resides locally, no waiting occurs." These are the different types of write that I've used and their benchmarks: *mnesia:dirty_write/1* write_bench(Count) -> F = fun(N) -> mnesia:dirty_write(#test_table{key = N, value = N}) end, lists:foreach(F, lists:seq(1, Count)). 1> mnesia_test:write(['1@REDACTED']). Written in 1.494185 sec, at a rate of 334630.5845661682/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 2.865135 sec, at a rate of 174511.84673671573/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED ']). Written in 77.847125 sec, at a rate of 6422.844774293207/sec *mnesia:activity/2 with sync_dirty* write_bench(Count) -> F = fun(N) -> mnesia:activity(sync_dirty, fun() -> mnesia:dirty_write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). 1> mnesia_test:write(['1@REDACTED']). Written in 1.79734 sec, at a rate of 278188.8791213682/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 3.570957 sec, at a rate of 140018.48804116095/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED ']). Written in 92.945978 sec, at a rate of 5379.468921183443/sec *mnesia:activity/2 with async_dirty* write_bench(Count) -> F = fun(N) -> mnesia:activity(async_dirty, fun() -> mnesia:dirty_write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). 1> mnesia_test:write(['1@REDACTED']). Written in 1.638162 sec, at a rate of 305220.11864516453/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 3.255289 sec, at a rate of 153596.19376344158/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED ']). Written in 98.841335 sec, at a rate of 5058.61237102878/sec *mnesia:async_dirty/1* write_bench(Count) -> F = fun(N) -> mnesia:async_dirty(fun() -> mnesia:dirty_write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). 1> mnesia_test:write(['1@REDACTED']). Written in 1.688114 sec, at a rate of 296188.5275520492/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 3.166962 sec, at a rate of 157880.0124535754/sec 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED ']). Written in 93.074646 sec, at a rate of 5372.032250329483/sec Any ideas? Best, r. [1] http://erlang.org/doc/apps/mnesia/Mnesia_chap4.html %%%%%%%%%%%%%%%%%%%%%%% FULL MODULE %%%%%%%%%%%%%%%%%%%%%%% -module(mnesia_test). -compile(export_all). -record(test_table, { key = undefined :: any(), value = undefined :: any() }). write(Nodes) -> Count = 500000, connect_nodes(Nodes), start_mnesia_on(Nodes), create_table_in(Nodes), {Time, _} = timer:tc(?MODULE, write_bench, [Count]), io:format("Written in ~p sec, at a rate of ~p/sec~n", [ Time/1000000, Count/Time*1000000 ]). connect_nodes(Nodes) -> [true = net_kernel:connect_node(Node) || Node <- Nodes]. start_mnesia_on(Nodes) -> [rpc:call(Node, application, start, [mnesia]) || Node <- Nodes]. create_table_in(Nodes) -> mnesia:change_config(extra_db_nodes, Nodes), mnesia:create_table(test_table, [ {type, set}, {ram_copies, Nodes}, {attributes, record_info(fields, test_table)}, {storage_properties, [{ets, [{read_concurrency, true}]}]} ]). write_bench(Count) -> F = fun(N) -> mnesia:activity(ets, fun() -> mnesia:dirty_write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% On Fri, Mar 11, 2016 at 12:25 AM, Ryan wrote: > On 03/10/2016 05:10 PM, Roberto Ostinelli wrote: > > Thank you for your input. > > This is interesting since the documentation [1] states: > > "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun > [, Args]) , it is > performed in almost the same context as the function > mnesia:async_dirty/1,2 > . The difference is > that the operations are performed synchronously. The caller waits for the > updates to be performed on all active replicas." > > Maybe I'm not reading this right? > > I agree that those docs imply an async_dirty call doesn't wait for > anything beyond the one node. It specifically says, under async_dirty, > that, "The functions wait for the operation to be performed on one node but > not the others. If the table resides locally, no waiting occurs." > > I may have misinterpreted the code I was looking at, or the docs could be > misleading. I'll wait eagerly for someone more well-informed to come along > and shed some light on the situation. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon Mar 14 12:47:50 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 14 Mar 2016 11:47:50 +0000 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> <56E20257.40304@gmail.com> Message-ID: Try using mnesia:write instead of mnesia:dirty_write inside your fun. On Mon, Mar 14, 2016 at 12:39 PM Roberto Ostinelli wrote: > So, > I took the time to benchmark a little more. I'm hoping that some kind soul > can help me understand this here. > > As said, fundamentally I don't see any speed difference when using > sync_dirty and async_dirty. > The more nodes you add, the slower insertions will be even if the docs for > async_dirty state [1]: > > "By passing the same "fun" as an argument to the function > mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context > as the function mnesia:async_dirty/1,2. The difference is that the > operations are performed synchronously. The caller waits for the updates to > be performed on all active replicas." > > "The functions wait for the operation to be performed on one node but not > the others. If the table resides locally, no waiting occurs." > > > These are the different types of write that I've used and their benchmarks: > > > *mnesia:dirty_write/1* > > write_bench(Count) -> > F = fun(N) -> > mnesia:dirty_write(#test_table{key = N, value = N}) > end, > lists:foreach(F, lists:seq(1, Count)). > > > 1> mnesia_test:write(['1@REDACTED']). > Written in 1.494185 sec, at a rate of 334630.5845661682/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 2.865135 sec, at a rate of 174511.84673671573/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED',' > 4@REDACTED']). > Written in 77.847125 sec, at a rate of 6422.844774293207/sec > > > > *mnesia:activity/2 with sync_dirty* > > write_bench(Count) -> > F = fun(N) -> > mnesia:activity(sync_dirty, fun() -> > mnesia:dirty_write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > > 1> mnesia_test:write(['1@REDACTED']). > Written in 1.79734 sec, at a rate of 278188.8791213682/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 3.570957 sec, at a rate of 140018.48804116095/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED',' > 4@REDACTED']). > Written in 92.945978 sec, at a rate of 5379.468921183443/sec > > > > *mnesia:activity/2 with async_dirty* > > write_bench(Count) -> > F = fun(N) -> > mnesia:activity(async_dirty, fun() -> > mnesia:dirty_write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > 1> mnesia_test:write(['1@REDACTED']). > Written in 1.638162 sec, at a rate of 305220.11864516453/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 3.255289 sec, at a rate of 153596.19376344158/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED',' > 4@REDACTED']). > Written in 98.841335 sec, at a rate of 5058.61237102878/sec > > > > > *mnesia:async_dirty/1* > > write_bench(Count) -> > F = fun(N) -> > mnesia:async_dirty(fun() -> > mnesia:dirty_write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > 1> mnesia_test:write(['1@REDACTED']). > Written in 1.688114 sec, at a rate of 296188.5275520492/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 3.166962 sec, at a rate of 157880.0124535754/sec > > 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED',' > 4@REDACTED']). > Written in 93.074646 sec, at a rate of 5372.032250329483/sec > > > > Any ideas? > > Best, > r. > > [1] http://erlang.org/doc/apps/mnesia/Mnesia_chap4.html > > > %%%%%%%%%%%%%%%%%%%%%%% FULL MODULE %%%%%%%%%%%%%%%%%%%%%%% > > -module(mnesia_test). > -compile(export_all). > > -record(test_table, { > key = undefined :: any(), > value = undefined :: any() > }). > > write(Nodes) -> > Count = 500000, > > connect_nodes(Nodes), > start_mnesia_on(Nodes), > create_table_in(Nodes), > > {Time, _} = timer:tc(?MODULE, write_bench, [Count]), > > io:format("Written in ~p sec, at a rate of ~p/sec~n", [ > Time/1000000, > Count/Time*1000000 > ]). > > connect_nodes(Nodes) -> > [true = net_kernel:connect_node(Node) || Node <- Nodes]. > > start_mnesia_on(Nodes) -> > [rpc:call(Node, application, start, [mnesia]) || Node <- Nodes]. > > create_table_in(Nodes) -> > mnesia:change_config(extra_db_nodes, Nodes), > mnesia:create_table(test_table, [ > {type, set}, > {ram_copies, Nodes}, > {attributes, record_info(fields, test_table)}, > {storage_properties, [{ets, [{read_concurrency, true}]}]} > ]). > > write_bench(Count) -> > F = fun(N) -> > mnesia:activity(ets, fun() -> > mnesia:dirty_write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > > On Fri, Mar 11, 2016 at 12:25 AM, Ryan wrote: > >> On 03/10/2016 05:10 PM, Roberto Ostinelli wrote: >> >> Thank you for your input. >> >> This is interesting since the documentation [1] states: >> >> "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun >> [, Args]) , it is >> performed in almost the same context as the function >> mnesia:async_dirty/1,2 >> . The difference is >> that the operations are performed synchronously. The caller waits for the >> updates to be performed on all active replicas." >> >> Maybe I'm not reading this right? >> >> I agree that those docs imply an async_dirty call doesn't wait for >> anything beyond the one node. It specifically says, under async_dirty, >> that, "The functions wait for the operation to be performed on one node but >> not the others. If the table resides locally, no waiting occurs." >> >> I may have misinterpreted the code I was looking at, or the docs could be >> misleading. I'll wait eagerly for someone more well-informed to come along >> and shed some light on the situation. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Mar 14 13:54:37 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 14 Mar 2016 13:54:37 +0100 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> <56E20257.40304@gmail.com> Message-ID: On Mon, Mar 14, 2016 at 12:47 PM, Dan Gudmundsson wrote: > Try using mnesia:write instead of mnesia:dirty_write inside your fun. > Making this change makes sync_dirty much slower when there are more than just one node. So, this clarifies that probably using dirty_write with sync_dirty falls back to async_dirty anyway. However, the findings still persist: the more nodes you add, the slower the writing to mnesia is, even when using async_dirty. Any other ideas? Best, r. %%%%% RESULTS %%%%% write_bench(Count) -> F = fun(N) -> mnesia:activity(sync_dirty, fun() -> mnesia:write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). mnesia_test:write(['1@REDACTED']). Written in 1.756049 sec, at a rate of 284730.095800288/sec mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 37.094925 sec, at a rate of *13478*.932765061529/sec write_bench(Count) -> F = fun(N) -> mnesia:activity(async_dirty, fun() -> mnesia:write(#test_table{key = N, value = N}) end) end, lists:foreach(F, lists:seq(1, Count)). mnesia_test:write(['1@REDACTED']). Written in 1.914213 sec, at a rate of 261203.9517023445/sec mnesia_test:write(['1@REDACTED','2@REDACTED']). Written in 3.402219 sec, at a rate of 146962.9086193452/sec -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon Mar 14 16:27:58 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 14 Mar 2016 15:27:58 +0000 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> <56E20257.40304@gmail.com> Message-ID: On Mon, Mar 14, 2016 at 1:54 PM Roberto Ostinelli wrote: > On Mon, Mar 14, 2016 at 12:47 PM, Dan Gudmundsson > wrote: > >> Try using mnesia:write instead of mnesia:dirty_write inside your fun. >> > > Making this change makes sync_dirty much slower when there are more than > just one node. So, this clarifies that probably using dirty_write with > sync_dirty falls back to async_dirty anyway. > dirty_write does not care if you use async_dirty, sync_dirty or apply fun directly. > However, the findings still persist: the more nodes you add, the slower > the writing to mnesia is, even when using async_dirty. > > I don't see the problem with that, you run more code, and send more messages when you have more nodes. When running with one node, you only use the local node with no distribution at all, with several nodes you might block on send because tcp buffers are full and so on. You should use mnesia:write and compare async_ vs sync_. /Dan > Any other ideas? > > Best, > r. > > > > %%%%% RESULTS %%%%% > > write_bench(Count) -> > F = fun(N) -> > mnesia:activity(sync_dirty, fun() -> > mnesia:write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > > mnesia_test:write(['1@REDACTED']). > Written in 1.756049 sec, at a rate of 284730.095800288/sec > > mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 37.094925 sec, at a rate of *13478*.932765061529/sec > > > write_bench(Count) -> > F = fun(N) -> > mnesia:activity(async_dirty, fun() -> > mnesia:write(#test_table{key = N, value = N}) > end) > end, > lists:foreach(F, lists:seq(1, Count)). > > mnesia_test:write(['1@REDACTED']). > Written in 1.914213 sec, at a rate of 261203.9517023445/sec > > mnesia_test:write(['1@REDACTED','2@REDACTED']). > Written in 3.402219 sec, at a rate of 146962.9086193452/sec > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Mar 14 16:54:10 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 14 Mar 2016 16:54:10 +0100 Subject: [erlang-questions] mnesia async vs sync In-Reply-To: References: <44714EAB-510E-4C95-A6F2-25B43092BCCE@widetag.com> <56E1FC06.9090702@gmail.com> <56E20257.40304@gmail.com> Message-ID: On Mon, Mar 14, 2016 at 4:27 PM, Dan Gudmundsson wrote: > On Mon, Mar 14, 2016 at 1:54 PM Roberto Ostinelli > wrote: > >> On Mon, Mar 14, 2016 at 12:47 PM, Dan Gudmundsson >> wrote: >> >>> Try using mnesia:write instead of mnesia:dirty_write inside your fun. >>> >> >> Making this change makes sync_dirty much slower when there are more than >> just one node. So, this clarifies that probably using dirty_write with >> sync_dirty falls back to async_dirty anyway. >> > > dirty_write does not care if you use async_dirty, sync_dirty or apply fun > directly. > Paraphrasing what I've said, so yes. :) > > >> However, the findings still persist: the more nodes you add, the slower >> the writing to mnesia is, even when using async_dirty. >> >> > I don't see the problem with that, you run more code, and send more > messages when you have more nodes. > When running with one node, you only use the local node with no > distribution at all, with several nodes you might block on send because > tcp buffers are full and so on. > Sure, but that's not what is written in the docs. Here they are again: "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context as the function mnesia:async_dirty/1,2. The difference is that the operations are performed synchronously. The caller waits for the updates to be performed on all active replicas." and "The functions wait for the operation to be performed on one node but not the others. If the table resides locally, no waiting occurs." > You should use mnesia:write and compare async_ vs sync_. > Which is what I did in my previous email. :) The findings are that async_dirty also has a significant performance loss. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Mon Mar 14 21:08:02 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 04:08:02 +0800 Subject: [erlang-questions] Failed to start gproc globally Message-ID: Hi, I added {gproc, [ {gproc_dist, all} ]} into my sys.config and when I start the application I get =INFO REPORT==== 15-Mar-2016::04:01:49 === application: gproc exited: {{shutdown, {failed_to_start_child,gproc_dist, {'EXIT', {undef, [{gen_leader,start_link, [gproc_dist, ['myapp@REDACTED'], [{bcast_type,all}], gproc_dist,[], [{spawn_opt,[]}]], []}, {supervisor,do_start_child,2, [{file,"supervisor.erl"},{line,343}]}, {supervisor,start_children,3, [{file,"supervisor.erl"},{line,326}]}, {supervisor,init_children,2, [{file,"supervisor.erl"},{line,292}]}, {gen_server,init_it,6, [{file,"gen_server.erl"},{line,328}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,240}]}]}}}}, {gproc_app,start,[normal,[]]}} type: permanent {"Kernel pid terminated",application_controller,"{application_start_failure,gproc,{{shutdown,{failed_to_start_child,gproc_dist,{'EXIT',{undef,[{gen_leader,start_link,[gproc_dist,['myapp@REDACTED'],[{bcast_type,all}],gproc_dist,[],[{spawn_opt,[]}]],[]},{supervisor,do_start_child,2,[{file,\"supervisor.erl\"},{line,343}]},{supervisor,start_children,3,[{file,\"supervisor.erl\"},{line,326}]},{supervisor,init_children,2,[{file,\"supervisor.erl\"},{line,292}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,240}]}]}}}},{gproc_app,start,[normal,[]]}}}"} I want to specify gproc parameters in sys.config because that appears consistent to me, but I have failed to figure out the reason. Any advice is appreciated! Thanks Khitai From roberto.ostinelli@REDACTED Mon Mar 14 21:11:04 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 14 Mar 2016 21:11:04 +0100 Subject: [erlang-questions] Failed to start gproc globally In-Reply-To: References: Message-ID: <49C76EC3-D4D1-4538-9234-901550CC8655@widetag.com> It looks like you're starting it in distributed mode, for which you need gen_leader. Can you try adding it and trying again? From dmkolesnikov@REDACTED Mon Mar 14 21:25:27 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 14 Mar 2016 22:25:27 +0200 Subject: [erlang-questions] zlib memory leak Message-ID: <31BABB64-4F4B-406B-BF99-8C81CABA1359@gmail.com> Hello, I?ve got an interesting issue with zlib at otp-18.2.1 I?ve not checked other releases yet. I do have a file about 30GB of compressed data, it is expanded to 300GB of textual UTF8 data. The producer of the file claims that standard gzip is used. The file header [1] is: 1F 8B 08 04 00 00 00 00 00 00 24 03 My decompression program is very simple [2], it reads 64K binary chunks from file and inflates them using zlib:inflate(?). At some point of time, the inflate do not return and VM binary memory growth to infinity until it is crashed. The crash is reproducible all the time with my file. The file is not corrupted and gzip is capable to perform it check and inflate data. The file becomes readable by program if it is inflated - deflated again using command line gzip. The header of readable file is: 1F 8B 08 00 6E EC E6 56 00 03 D4 BD I am having a challenge to debug this issue further to zlib and understand root-cause. Do you have any suggestions on it? Best Regards, Dmitry P.S: the file, I am taking about, contains confidential data and cannot be disclosed to community. Reference: [1] http://www.zlib.org/rfc-gzip.html [2] https://github.com/fogfish/feta/blob/master/src/gz.erl From khitai.pang@REDACTED Tue Mar 15 03:31:27 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 10:31:27 +0800 Subject: [erlang-questions] Failed to start gproc globally In-Reply-To: <49C76EC3-D4D1-4538-9234-901550CC8655@widetag.com> References: <49C76EC3-D4D1-4538-9234-901550CC8655@widetag.com> Message-ID: Which one should I use? https://github.com/garret-smith/gen_leader_revival https://github.com/abecciu/gen_leader_revival https://github.com/KirinDave/gen_leader_revival https://github.com/lehoff/gen_leader It seems that the first one is still under active development. Thanks Khitai On 2016/3/15 4:11, Roberto Ostinelli wrote: > It looks like you're starting it in distributed mode, for which you need gen_leader. > > Can you try adding it and trying again? From pulsarpietro@REDACTED Mon Mar 14 23:53:32 2016 From: pulsarpietro@REDACTED (Pietro) Date: Mon, 14 Mar 2016 22:53:32 +0000 Subject: [erlang-questions] Beginner question Message-ID: <87d1qwk6xf.fsf@posteo.net> Hi all, I have recently started to implement a small academic project in Erlang and I have bumped into a problem which seems to show to myself I haven't grasped something important about the technology itself. That's my code : -module(test). -export([start/0, interact/2]). start() -> spawn (fun() -> startloop() end). interact(Pid, Request) -> Pid ! {self(), Request}, receive {Pid, Response} -> Response end. startloop() -> TableId = ets:new(dictionary, [set]), loop(TableId). loop(Table) -> receive {From, {insert, key, value}} -> From ! ets:insert(Table, {key, value}), loop(Table); {From, {lookup, key}} -> From ! ets:lookup(Table, key), loop(Table); Any -> Any end. The problem happens when I try to interact with the server I start using the command: Pid = test:start(). Then I run : test:interact(Pid, {insert, testkey, testvalue}). My ershell at this point hangs and nothing happens ... what is happening ? Please feel free to redirect me to a more appropriate newsgroup if my question is not appropriate or if there is a better one. Thanks in advance. Pietro. From goyalk@REDACTED Tue Mar 15 03:19:35 2016 From: goyalk@REDACTED (Kapil Goyal) Date: Tue, 15 Mar 2016 02:19:35 +0000 Subject: [erlang-questions] FIPS compliance Message-ID: Hi All, We use RabbitMQ and are working on running it in FIPS compliance. According to a post on RMQ forum (https://groups.google.com/forum/#!topic/rabbitmq-users/wUzUjgDQ9M8), Erlang is not FIPS compliance. Is this correct? If so, are there plans to be compliant in near future? Thanks Kapil -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Tue Mar 15 06:26:47 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 15 Mar 2016 01:26:47 -0400 Subject: [erlang-questions] Beginner question In-Reply-To: <87d1qwk6xf.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> Message-ID: On Mar 14, 2016, at 18:53, Pietro wrote: > > Hi all, > > I have recently started to implement a small academic project in Erlang > and I have bumped into a problem which seems to show to myself I haven't > grasped something important about the technology itself. > > That's my code : > > -module(test). > -export([start/0, interact/2]). > > > start() -> > spawn (fun() -> startloop() end). > > > interact(Pid, Request) -> > Pid ! {self(), Request}, > receive > {Pid, Response} -> Response > end. > > > startloop() -> > TableId = ets:new(dictionary, [set]), > loop(TableId). > > loop(Table) -> > receive > {From, {insert, key, value}} -> > From ! ets:insert(Table, {key, value}), > loop(Table); > {From, {lookup, key}} -> > From ! ets:lookup(Table, key), > loop(Table); > Any -> > Any > > end. > > > The problem happens when I try to interact with the server I start using > the command: > > Pid = test:start(). > > Then I run : > > test:interact(Pid, {insert, testkey, testvalue}). > > My ershell at this point hangs and nothing happens ... what is happening > ? Please feel free to redirect me to a more appropriate newsgroup if my > question is not appropriate or if there is a better one. interact/2 is waiting for a message structured differently from what loop/1 is sending. From dmkolesnikov@REDACTED Tue Mar 15 07:08:49 2016 From: dmkolesnikov@REDACTED (dmkolesnikov@REDACTED) Date: Tue, 15 Mar 2016 08:08:49 +0200 Subject: [erlang-questions] Beginner question In-Reply-To: <87d1qwk6xf.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> Message-ID: Hello, You are using atoms instead of variables in your program. Erlang variables starts with capital letter. The receive loop cannot match your message: > {From, {insert, key, value}} -> It expects one triple of atoms but you send different one. > test:interact(Pid, {insert, testkey, testvalue}). Dmitry Sent from my iPhone > On 15 Mar 2016, at 00:53, Pietro wrote: > > Hi all, > > I have recently started to implement a small academic project in Erlang > and I have bumped into a problem which seems to show to myself I haven't > grasped something important about the technology itself. > > That's my code : > > -module(test). > -export([start/0, interact/2]). > > > start() -> > spawn (fun() -> startloop() end). > > > interact(Pid, Request) -> > Pid ! {self(), Request}, > receive > {Pid, Response} -> Response > end. > > > startloop() -> > TableId = ets:new(dictionary, [set]), > loop(TableId). > > loop(Table) -> > receive > {From, {insert, key, value}} -> > From ! ets:insert(Table, {key, value}), > loop(Table); > {From, {lookup, key}} -> > From ! ets:lookup(Table, key), > loop(Table); > Any -> > Any > > end. > > > The problem happens when I try to interact with the server I start using > the command: > > Pid = test:start(). > > Then I run : > > test:interact(Pid, {insert, testkey, testvalue}). > > My ershell at this point hangs and nothing happens ... what is happening > ? Please feel free to redirect me to a more appropriate newsgroup if my > question is not appropriate or if there is a better one. > > Thanks in advance. > > Pietro. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From drew.varner@REDACTED Tue Mar 15 08:06:34 2016 From: drew.varner@REDACTED (Drew Varner) Date: Tue, 15 Mar 2016 03:06:34 -0400 Subject: [erlang-questions] FIPS compliance In-Reply-To: References: Message-ID: Kapil, Erlang's cryptography is not FIPS 140-2-certified. There was a pull request to add FIPS compliance via OpenSSL in FIPS mode, but it stalled [1]. Calls to OpenSSL's crypto canister must go through the Envelope (EVP) API calls. The crypto library in Erlang OTP 19 will use EVP calls (exclusively, I assume) [2]. However, EVP calls alone are not enough for FIPS 140-2 support. If the only problem is terminating incoming HTTP requests, you may be able to get away with proxying the request through a FIPS 140-2 load balancer. rabbitmq-server calls crypto for password hashing. You'd need to replace calls to crypto with calls to a FIPS provider and look for calls made outside of crypto (BIFs like md5, phash). - Drew [1] https://github.com/erlang/otp/pull/377 [2] http://youtu.be/YlNrWxH56_E > On Mar 14, 2016, at 10:19 PM, Kapil Goyal wrote: > > Hi All, > > We use RabbitMQ and are working on running it in FIPS compliance. According to a post on RMQ forum (https://groups.google.com/forum/#!topic/rabbitmq-users/wUzUjgDQ9M8), Erlang is not FIPS compliance. Is this correct? If so, are there plans to be compliant in near future? > > Thanks > Kapil > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Tue Mar 15 09:45:13 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 16:45:13 +0800 Subject: [erlang-questions] Failed to start gproc globally In-Reply-To: References: <49C76EC3-D4D1-4538-9234-901550CC8655@widetag.com> Message-ID: I used https://github.com/garret-smith/gen_leader_revival and gproc distribution starts fine. Thanks Khitai On 2016/3/15 10:31, Khitai Pang wrote: > Which one should I use? > > https://github.com/garret-smith/gen_leader_revival > https://github.com/abecciu/gen_leader_revival > https://github.com/KirinDave/gen_leader_revival > https://github.com/lehoff/gen_leader > > It seems that the first one is still under active development. > > Thanks > Khitai > > On 2016/3/15 4:11, Roberto Ostinelli wrote: >> It looks like you're starting it in distributed mode, for which you >> need gen_leader. >> >> Can you try adding it and trying again? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From khitai.pang@REDACTED Tue Mar 15 10:31:09 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 17:31:09 +0800 Subject: [erlang-questions] (no subject) Message-ID: I have been trying to use gproc with https://github.com/garret-smith/gen_leader_revival. I found that registering a name which is already registered will destroy the existing registration of the name, and a remote node is unaware of this. See the following steps: Register node1 shell as global name 'abcde' on node1, the name is visable on node2 and sending messages from node2 shell to 'abcde' works fine: (node1)1> gproc:reg({n, g, abcde}). true (node1)2> gproc:where({n, g, abcde}). <0.229.0> (node2)1> gproc:where({n, g, abcde}). <8049.229.0> (node2)2> gproc:send({n, g, abcde}, hello). hello (node1)3> flush(). Shell got hello ok Now on node1 shell register with the name again, it removes the existing registration: (node1)4> gproc:reg({n, g, abcde}). ** exception error: bad argument in function gproc:reg/1 called as gproc:reg({n,g,abcde}) (node1)5> gproc:where({n, g, abcde}). undefined But on node2 the shell still gets <8049.229.0>: (node2)3> gproc:where({n, g, abcde}). <8049.229.0> And sending message from node2 shell to 'abcde' still works 'fine', but the node1 shell doesn't get any message: (node2)4> gproc:send({n, g, abcde}, hello). hello (node1)6> flush(). ok Now if I register node1 shell as {n, g, abcde} again, everything works fine again: (node1)7> gproc:reg({n, g, abcde}). true (node2)5> gproc:where({n, g, abcde}). <8049.268.0> (node2)6> gproc:send({n, g, abcde}, hello). hello (node1) 8> flush(). Shell got hello ok This doesn't look right to me. 1) When registering with a name already registered, why doesn't gproc return something like {error, already_registered}? 2) After the registration is removed, why does a remove node still sees the non-existent registration? Thanks Khitai -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Tue Mar 15 10:32:07 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 17:32:07 +0800 Subject: [erlang-questions] gproc registration 'partitioning'? In-Reply-To: References: Message-ID: I have been trying to use gproc with https://github.com/garret-smith/gen_leader_revival. I found that registering a name which is already registered will destroy the existing registration of the name, and a remote node is unaware of this. See the following steps:Register node1 shell as global name 'abcde' on node1, the name is visable on node2 and sending messages from node2 shell to 'abcde' works fine:(node1)1> gproc:reg({n, g, abcde}).true(node1)2> gproc:where({n, g, abcde}).<0.229.0>(node2)1> gproc:where({n, g, abcde}).<8049.229.0>(node2)2> gproc:send({n, g, abcde}, hello).hello(node1)3> flush().Shell got hellookNow on node1 shell register with the name again, it removes the existing registration:(node1)4> gproc:reg({n, g, abcde}).** exception error: bad argument in function gproc:reg/1 called as gproc:reg({n,g,abcde})(node1)5> gproc:where({n, g, abcde}).undefinedBut on node2 the shell still gets <8049.229.0>:(node2)3> gproc:where({n, g, abcde}).<8049.229.0>And sending message from node2 shell to 'abcde' still works 'fine', but the node1 shell doesn't get any message:(node2)4> gproc:send({n, g, abcde}, hello).hello(node1)6> flush().okNow if I register node1 shell as {n, g, abcde} again, everything works fine again:(node1)7> gproc:reg({n, g, abcde}).true(node2)5> gproc:where({n, g, abcde}).<8049.268.0>(node2)6> gproc:send({n, g, abcde}, hello).hello(node1) 8> flush().Shell got hellookThis doesn't look right to me.1) When registering with a name already registered, why doesn't gproc return something like {error, already_registered}?2) After the registration is removed, why does a remove node still sees the non-existent registration?ThanksKhitai -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Tue Mar 15 10:33:35 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 15 Mar 2016 17:33:35 +0800 Subject: [erlang-questions] (no subject) In-Reply-To: References: Message-ID: sorry for the untitled email On 2016/3/15 17:31, Khitai Pang wrote: > I have been trying to use gproc with > https://github.com/garret-smith/gen_leader_revival. I found that > registering a name which is already registered will destroy the > existing registration of the name, and a remote node is unaware of > this. See the following steps: > > > Register node1 shell as global name 'abcde' on node1, the name is > visable on node2 and sending messages from node2 shell to 'abcde' > works fine: > > (node1)1> gproc:reg({n, g, abcde}). > true > (node1)2> gproc:where({n, g, abcde}). > <0.229.0> > > (node2)1> gproc:where({n, g, abcde}). > <8049.229.0> > (node2)2> gproc:send({n, g, abcde}, hello). > hello > > (node1)3> flush(). > Shell got hello > ok > > > Now on node1 shell register with the name again, it removes the > existing registration: > > (node1)4> gproc:reg({n, g, abcde}). > ** exception error: bad argument > in function gproc:reg/1 > called as gproc:reg({n,g,abcde}) > (node1)5> gproc:where({n, g, abcde}). > undefined > > > But on node2 the shell still gets <8049.229.0>: > > (node2)3> gproc:where({n, g, abcde}). > <8049.229.0> > > > And sending message from node2 shell to 'abcde' still works 'fine', > but the node1 shell doesn't get any message: > > (node2)4> gproc:send({n, g, abcde}, hello). > hello > > (node1)6> flush(). > ok > > > Now if I register node1 shell as {n, g, abcde} again, everything works > fine again: > > (node1)7> gproc:reg({n, g, abcde}). > true > > (node2)5> gproc:where({n, g, abcde}). > <8049.268.0> > (node2)6> gproc:send({n, g, abcde}, hello). > hello > > (node1) 8> flush(). > Shell got hello > ok > > > This doesn't look right to me. > 1) When registering with a name already registered, why doesn't gproc > return something like {error, already_registered}? > 2) After the registration is removed, why does a remove node still > sees the non-existent registration? > > > Thanks > Khitai > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From itshowlertime@REDACTED Tue Mar 15 09:52:57 2016 From: itshowlertime@REDACTED (Alex Howle) Date: Tue, 15 Mar 2016 08:52:57 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: Message-ID: I've been experiencing an issue and was wondering if anyone else has any experience in this area. I've stripped back the problem to its bare bones for the purposes of this mail. I have an Erlang 18.1 application that uses ETS to store an Erlang map structure. Using erts_debug:flat_size/1 I can approximate the map's size to be 1MB. Upon the necessary activity trigger the application spawns about 25 short-lived processes to perform the main work of the application. This activity trigger is fired roughly 9 times a second under normal operating conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to read from the map. What I've found is that the above implementation has a CPU profile that is quite "expensive" - each of the CPU cores (40 total comprised of 2 Processors with 10 hyperthreaded cores) frequently runs at 100%. The machine in question also has 32GB RAM of which about 9GB is used at peak. There is no swap usage whatsoever. Examination shows that copy_shallow is performing the most work. After changing the implementation so that the 25 spawned processes no longer read from the ETS table to retrieve the map structure and, instead the map is passed to the processes on spawn, the CPU usage on the server is considerably lower. Can anyone offer advice as to why I'm seeing the differing CPU profiles? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue Mar 15 12:32:24 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 15 Mar 2016 12:32:24 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: Message-ID: <56E7F2C8.1000006@ericsson.com> Each successful ets:lookup call is a copy operation of the entire term from ETS to the process heap. If you are comparing ets:lookup of big map to sending big map in message then I would expect ets:lookup to win, as copy_shallow (used by ets:lookup) is optimized to be faster than copy_struct (used by send). /Sverker, Erlang/OTP On 03/15/2016 09:52 AM, Alex Howle wrote: > > I've been experiencing an issue and was wondering if anyone else has > any experience in this area. I've stripped back the problem to its > bare bones for the purposes of this mail. > > I have an Erlang 18.1 application that uses ETS to store an Erlang map > structure. Using erts_debug:flat_size/1 I can approximate the map's > size to be 1MB. Upon the necessary activity trigger the application > spawns about 25 short-lived processes to perform the main work of the > application. This activity trigger is fired roughly 9 times a second > under normal operating conditions. Each of these 25 processes performs > 1 x ets:lookup/2 calls to read from the map. > > What I've found is that the above implementation has a CPU profile > that is quite "expensive" - each of the CPU cores (40 total comprised > of 2 Processors with 10 hyperthreaded cores) frequently runs at 100%. > The machine in question also has 32GB RAM of which about 9GB is used > at peak. There is no swap usage whatsoever. Examination shows that > copy_shallow is performing the most work. > > After changing the implementation so that the 25 spawned processes no > longer read from the ETS table to retrieve the map structure and, > instead the map is passed to the processes on spawn, the CPU usage on > the server is considerably lower. > > Can anyone offer advice as to why I'm seeing the differing CPU profiles? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Mar 15 12:38:20 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 15 Mar 2016 12:38:20 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: <56E7F2C8.1000006@ericsson.com> References: <56E7F2C8.1000006@ericsson.com> Message-ID: If your map is "flat" and you frequently access individual keys, then you can avoid copying by restructuring: If you have {Key, Map} then store [{{Key, K}, V} || {K, V} <- maps:to_list(Map)] and look up by using {Key, K} where K is the desired map key. This can break the large map into smaller chunks which may be more amenable to copying around. On Tue, Mar 15, 2016 at 12:32 PM, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > Each successful ets:lookup call is a copy operation of the entire term > from ETS to the process heap. > > If you are comparing ets:lookup of big map > to sending big map in message then I would expect > ets:lookup to win, as copy_shallow (used by ets:lookup) > is optimized to be faster than copy_struct (used by send). > > > /Sverker, Erlang/OTP > > > > On 03/15/2016 09:52 AM, Alex Howle wrote: > > I've been experiencing an issue and was wondering if anyone else has any > experience in this area. I've stripped back the problem to its bare bones > for the purposes of this mail. > > > > I have an Erlang 18.1 application that uses ETS to store an Erlang map > structure. Using erts_debug:flat_size/1 I can approximate the map's size to > be 1MB. Upon the necessary activity trigger the application spawns about 25 > short-lived processes to perform the main work of the application. This > activity trigger is fired roughly 9 times a second under normal operating > conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to > read from the map. > > > > What I've found is that the above implementation has a CPU profile that is > quite "expensive" - each of the CPU cores (40 total comprised of 2 > Processors with 10 hyperthreaded cores) frequently runs at 100%. The > machine in question also has 32GB RAM of which about 9GB is used at peak. > There is no swap usage whatsoever. Examination shows that copy_shallow is > performing the most work. > > > > After changing the implementation so that the 25 spawned processes no > longer read from the ETS table to retrieve the map structure and, instead > the map is passed to the processes on spawn, the CPU usage on the server is > considerably lower. > > > > Can anyone offer advice as to why I'm seeing the differing CPU profiles? > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kvratkin@REDACTED Tue Mar 15 13:07:19 2016 From: kvratkin@REDACTED (Kirill Ratkin) Date: Tue, 15 Mar 2016 15:07:19 +0300 Subject: [erlang-questions] zlib memory leak In-Reply-To: <31BABB64-4F4B-406B-BF99-8C81CABA1359@gmail.com> References: <31BABB64-4F4B-406B-BF99-8C81CABA1359@gmail.com> Message-ID: Hi Dmitry, To check your situation I created simple demo for your gz module ( https://gist.github.com/kvratkin/ec4736aaf65a79bc4469). As I can see file size about 1Gb unzipped properly. No errors or VM crashes. Maybe I use your module incorrectly ... Unfortunately I can't check as huge file as you have but I'll try file 10Gb soon. P.S. I use Debian Jessie, 64 bit, 1 Virtual CPU and Erlang installed from Erlang Solution repo. 2016-03-14 23:25 GMT+03:00 Dmitry Kolesnikov : > Hello, > > I?ve got an interesting issue with zlib at otp-18.2.1 I?ve not checked > other releases yet. > I do have a file about 30GB of compressed data, it is expanded to 300GB of > textual UTF8 data. > The producer of the file claims that standard gzip is used. The file > header [1] is: > 1F 8B 08 04 00 00 00 00 00 00 24 03 > > My decompression program is very simple [2], it reads 64K binary chunks > from file and inflates them using zlib:inflate(?). > > At some point of time, the inflate do not return and VM binary memory > growth to infinity until it is crashed. > The crash is reproducible all the time with my file. The file is not > corrupted and gzip is capable to perform it check and inflate data. The > file becomes readable by program if it is inflated - deflated again using > command line gzip. The header of readable file is: > 1F 8B 08 00 6E EC E6 56 00 03 D4 BD > > I am having a challenge to debug this issue further to zlib and understand > root-cause. > Do you have any suggestions on it? > > Best Regards, > Dmitry > > P.S: the file, I am taking about, contains confidential data and cannot be > disclosed to community. > > Reference: > [1] http://www.zlib.org/rfc-gzip.html > [2] https://github.com/fogfish/feta/blob/master/src/gz.erl > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zandra.hird@REDACTED Tue Mar 15 13:29:18 2016 From: zandra.hird@REDACTED (Zandra Hird) Date: Tue, 15 Mar 2016 13:29:18 +0100 Subject: [erlang-questions] net_kernel:start/1 doesn't honor ERL_EPMD_PORT set by os:putenv/2 In-Reply-To: References: Message-ID: <56E8001E.6000900@ericsson.com> Hi Khitai, net_kernel:start is looking at the init value of the EPMD port (the one specified when erlang was started), so changing the environment variable in Erlang like this doesn't have any effect. I agree that it would probably be more intuitive if it worked as you describe, so please submit a feature request to bugs.erlang.org (or if you have found any documentation saying that it should work as you say, please submit it as a bug and point to the documentation), and we'll look into it as soon as we get time to do so. If you want to contribute with a change for this, which you are very welcome to do, please look at the code in lib/kernel/src/erl_epmd.erl. Make sure to think it through though, for example by handling the case when the port environment variable is changed when the distribution is already started. :) Best Regards, Zandra On 03/11/2016 05:48 AM, Khitai Pang wrote: > Hi, > > I have the following code: > > os:putenv("ERL_EPMD_ADDRESS", inet_parse:ntoa(IP)), > os:putenv("ERL_EPMD_PORT", integer_to_list(5555)), > os:cmd("epmd -daemon"), > net_kernel:start([bob, longnames]), > > > The ERL_EPMD_ADDRESS and ERL_EPMD_PORT environment variables are used > by os:cmd("epmd -daemon") but net_kernel:start/1 fails: > > Protocol: "inet_tcp": register/listen error: econnrefused > > Apparently it still tries to connect to 4369, where no process is > listening. > > If I start erl by "ERL_EPMD_PORT=5555 erl", net_kernel:start/1 in the > above code works fine. > > So I have come to the conclusion that environment variables set by > os:putenv/2 are not honored by net_kernel:start/1. Am I right? If > yes, 1) is this a bug? 2) for a Erlang/OTP release, how to > automatically set global environment variables for erl? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From itshowlertime@REDACTED Tue Mar 15 16:07:32 2016 From: itshowlertime@REDACTED (Alex Howle) Date: Tue, 15 Mar 2016 15:07:32 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: The map is not being sent as a message. It is passed in to the spawned processes by being in scope of the spawned function. Pseudocode: A=bigmap, spawn(fun()-> do_something(A) end). On 15 Mar 2016 13:43, "Alex Howle" wrote: > The map is not being sent as a message. It is passed in to the spawned > processes by being in scope of the spawned function. > > Pseudocode: > > A=bigmap, > spawn(fun()-> do_something(A) end). > On 15 Mar 2016 11:32, "Sverker Eriksson" > wrote: > >> Each successful ets:lookup call is a copy operation of the entire term >> from ETS to the process heap. >> >> If you are comparing ets:lookup of big map >> to sending big map in message then I would expect >> ets:lookup to win, as copy_shallow (used by ets:lookup) >> is optimized to be faster than copy_struct (used by send). >> >> >> /Sverker, Erlang/OTP >> >> >> On 03/15/2016 09:52 AM, Alex Howle wrote: >> >> I've been experiencing an issue and was wondering if anyone else has any >> experience in this area. I've stripped back the problem to its bare bones >> for the purposes of this mail. >> >> >> >> I have an Erlang 18.1 application that uses ETS to store an Erlang map >> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >> be 1MB. Upon the necessary activity trigger the application spawns about 25 >> short-lived processes to perform the main work of the application. This >> activity trigger is fired roughly 9 times a second under normal operating >> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >> read from the map. >> >> >> >> What I've found is that the above implementation has a CPU profile that >> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >> machine in question also has 32GB RAM of which about 9GB is used at peak. >> There is no swap usage whatsoever. Examination shows that copy_shallow is >> performing the most work. >> >> >> >> After changing the implementation so that the 25 spawned processes no >> longer read from the ETS table to retrieve the map structure and, instead >> the map is passed to the processes on spawn, the CPU usage on the server is >> considerably lower. >> >> >> >> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Tue Mar 15 11:32:30 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Tue, 15 Mar 2016 11:32:30 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <22237.18270.703914.235613@gargle.gargle.HOWL> References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> Message-ID: <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> Mikael Pettersson writes: > [ text/plain ] > Zachary Kessin writes: > > I for one would love to see an ML-family language on the beam! > > I'm working on a Standard ML (SML'97) compiler targeting the > Erlang/OTP VM. I hope to have it finished enough to make it > public sometime this year. I have been thinking about this a bit, so I'll throw in some questions. What kind of type system are you using? I have been told that a type-and-effect system would be the way to capture the effects of statements on the mailboxes. Have you given any thought to upgrades? The problem is that an upgrade can change the types, so that leads to some thinking about versioning the types. Tricky business. Without some attention to this it will be hard to evolve a distributed system. And then there is interfacing to the external world. My instinct tells me that it would make sense to attach some sort of adapter code that will have the type signature val adapt: binary -> some_type Which leads me to... how will you describe binary pattern matching? Very open to further ping-pong offline. Cheers, Torben > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From send2philip@REDACTED Tue Mar 15 17:37:48 2016 From: send2philip@REDACTED (Philip Clarke) Date: Tue, 15 Mar 2016 16:37:48 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: Hi Alex, It would be interesting to see if the cpu is busy with garbage collection when the process is reading from ETS. I suspect that when the process is started without the map and then reads the map from ETS that it has to increase its heap size several times. In order to increase heap size a full GC has to be run and I have found this in the past to be quite expensive. I suspect that when you create the process with spawn(fun()-> do_something(A) end) that the process is initialised with the correct heap size thereby avoiding the GC and hence the better CPU performance. Perhaps you could rerun the program with a higher default heap size per process (*+hms)?* I would be interested to know how it goes. Regards Philip On 15 March 2016 at 15:07, Alex Howle wrote: > The map is not being sent as a message. It is passed in to the spawned > processes by being in scope of the spawned function. > > Pseudocode: > > A=bigmap, > spawn(fun()-> do_something(A) end). > On 15 Mar 2016 13:43, "Alex Howle" wrote: > >> The map is not being sent as a message. It is passed in to the spawned >> processes by being in scope of the spawned function. >> >> Pseudocode: >> >> A=bigmap, >> spawn(fun()-> do_something(A) end). >> On 15 Mar 2016 11:32, "Sverker Eriksson" >> wrote: >> >>> Each successful ets:lookup call is a copy operation of the entire term >>> from ETS to the process heap. >>> >>> If you are comparing ets:lookup of big map >>> to sending big map in message then I would expect >>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>> is optimized to be faster than copy_struct (used by send). >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>> >>> I've been experiencing an issue and was wondering if anyone else has any >>> experience in this area. I've stripped back the problem to its bare bones >>> for the purposes of this mail. >>> >>> >>> >>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>> short-lived processes to perform the main work of the application. This >>> activity trigger is fired roughly 9 times a second under normal operating >>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>> read from the map. >>> >>> >>> >>> What I've found is that the above implementation has a CPU profile that >>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>> performing the most work. >>> >>> >>> >>> After changing the implementation so that the 25 spawned processes no >>> longer read from the ETS table to retrieve the map structure and, instead >>> the map is passed to the processes on spawn, the CPU usage on the server is >>> considerably lower. >>> >>> >>> >>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seriy.pr@REDACTED Tue Mar 15 17:47:38 2016 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Tue, 15 Mar 2016 19:47:38 +0300 Subject: [erlang-questions] zlib memory leak Message-ID: Maybe your gzip file is some kind of sparse file at some point, so, even this 64K compressed chunk inflates to huge uncompressed value? In that case you may try http://erlang.org/doc/man/zlib.html#inflateChunk-2 Is VM memory grows linearly during program runtime or it grows instantly at some moment? P.S.: what is this `stdio` module? Hello, > I?ve got an interesting issue with zlib at otp-18.2.1 I?ve not checked > other releases yet. > I do have a file about 30GB of compressed data, it is expanded to 300GB of > textual UTF8 data. > The producer of the file claims that standard gzip is used. The file > header [1] is: > 1F 8B 08 04 00 00 00 00 00 00 24 03 > My decompression program is very simple [2], it reads 64K binary chunks > from file and inflates them using zlib:inflate(?). > At some point of time, the inflate do not return and VM binary memory > growth to infinity until it is crashed. > The crash is reproducible all the time with my file. The file is not > corrupted and gzip is capable to perform it check and inflate data. The > file becomes readable by program if it is inflated - deflated again using > command line gzip. The header of readable file is: > 1F 8B 08 00 6E EC E6 56 00 03 D4 BD > I am having a challenge to debug this issue further to zlib and understand > root-cause. > Do you have any suggestions on it? > Best Regards, > Dmitry > P.S: the file, I am taking about, contains confidential data and cannot be > disclosed to community. > Reference: > [1] http://www.zlib.org/rfc-gzip.html > [2] https://github.com/fogfish/feta/blob/master/src/gz.erl -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Tue Mar 15 19:13:20 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Tue, 15 Mar 2016 19:13:20 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: If you start it in this way, your map is copied from a heap of the parent process to a heap of the new process. The difference could be starting size of a heap. Sou you could try to start the process as spawn_op(fun()-> do_something(A) end, {}min_heap_size, Size}). With Size around 2 * erts_debug:flat_size(Map). On Tue, Mar 15, 2016 at 4:07 PM, Alex Howle wrote: > The map is not being sent as a message. It is passed in to the spawned > processes by being in scope of the spawned function. > > Pseudocode: > > A=bigmap, > spawn(fun()-> do_something(A) end). > On 15 Mar 2016 13:43, "Alex Howle" wrote: > >> The map is not being sent as a message. It is passed in to the spawned >> processes by being in scope of the spawned function. >> >> Pseudocode: >> >> A=bigmap, >> spawn(fun()-> do_something(A) end). >> On 15 Mar 2016 11:32, "Sverker Eriksson" >> wrote: >> >>> Each successful ets:lookup call is a copy operation of the entire term >>> from ETS to the process heap. >>> >>> If you are comparing ets:lookup of big map >>> to sending big map in message then I would expect >>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>> is optimized to be faster than copy_struct (used by send). >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>> >>> I've been experiencing an issue and was wondering if anyone else has any >>> experience in this area. I've stripped back the problem to its bare bones >>> for the purposes of this mail. >>> >>> >>> >>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>> short-lived processes to perform the main work of the application. This >>> activity trigger is fired roughly 9 times a second under normal operating >>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>> read from the map. >>> >>> >>> >>> What I've found is that the above implementation has a CPU profile that >>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>> performing the most work. >>> >>> >>> >>> After changing the implementation so that the 25 spawned processes no >>> longer read from the ETS table to retrieve the map structure and, instead >>> the map is passed to the processes on spawn, the CPU usage on the server is >>> considerably lower. >>> >>> >>> >>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Tue Mar 15 19:15:15 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Tue, 15 Mar 2016 19:15:15 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: I'm sorry, there is a typo. It should be spawn_opt(fun()-> do_something() end, [{min_heap_size, Size}]). On Tue, Mar 15, 2016 at 7:13 PM, Hynek Vychodil wrote: > If you start it in this way, your map is copied from a heap of the parent > process to a heap of the new process. The difference could be starting size > of a heap. Sou you could try to start the process as spawn_op(fun()-> > do_something(A) end, {}min_heap_size, Size}). > With Size around 2 * erts_debug:flat_size(Map). > > On Tue, Mar 15, 2016 at 4:07 PM, Alex Howle > wrote: > >> The map is not being sent as a message. It is passed in to the spawned >> processes by being in scope of the spawned function. >> >> Pseudocode: >> >> A=bigmap, >> spawn(fun()-> do_something(A) end). >> On 15 Mar 2016 13:43, "Alex Howle" wrote: >> >>> The map is not being sent as a message. It is passed in to the spawned >>> processes by being in scope of the spawned function. >>> >>> Pseudocode: >>> >>> A=bigmap, >>> spawn(fun()-> do_something(A) end). >>> On 15 Mar 2016 11:32, "Sverker Eriksson" >>> wrote: >>> >>>> Each successful ets:lookup call is a copy operation of the entire term >>>> from ETS to the process heap. >>>> >>>> If you are comparing ets:lookup of big map >>>> to sending big map in message then I would expect >>>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>>> is optimized to be faster than copy_struct (used by send). >>>> >>>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>>> >>>> I've been experiencing an issue and was wondering if anyone else has >>>> any experience in this area. I've stripped back the problem to its bare >>>> bones for the purposes of this mail. >>>> >>>> >>>> >>>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>>> short-lived processes to perform the main work of the application. This >>>> activity trigger is fired roughly 9 times a second under normal operating >>>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>>> read from the map. >>>> >>>> >>>> >>>> What I've found is that the above implementation has a CPU profile that >>>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>>> performing the most work. >>>> >>>> >>>> >>>> After changing the implementation so that the 25 spawned processes no >>>> longer read from the ETS table to retrieve the map structure and, instead >>>> the map is passed to the processes on spawn, the CPU usage on the server is >>>> considerably lower. >>>> >>>> >>>> >>>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Mar 15 19:20:10 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 15 Mar 2016 20:20:10 +0200 Subject: [erlang-questions] zlib memory leak In-Reply-To: References: Message-ID: <7C4186F9-0B44-438E-9B32-060058540C6A@gmail.com> No, this file is a utf8 text file, there are not sparse chunks and data is correct there. I am able to use my program to read this file if it is re-compressed with command-line gzip. I am suspecting that something wrong with zlib. BTW, I?ve build OTP from source no Mac, I need to check it on Linux. Best Regards, Dmitry > On Mar 15, 2016, at 6:47 PM, ?????? ???????? wrote: > > Maybe your gzip file is some kind of sparse file at some point, so, even this 64K compressed chunk inflates to huge uncompressed value? > In that case you may try http://erlang.org/doc/man/zlib.html#inflateChunk-2 > > Is VM memory grows linearly during program runtime or it grows instantly at some moment? > > P.S.: what is this `stdio` module? > > Hello, > I?ve got an interesting issue with zlib at otp-18.2.1 I?ve not checked other releases yet. > I do have a file about 30GB of compressed data, it is expanded to 300GB of textual UTF8 data. > The producer of the file claims that standard gzip is used. The file header [1] is: > 1F 8B 08 04 00 00 00 00 00 00 24 03 > My decompression program is very simple [2], it reads 64K binary chunks from file and inflates them using zlib:inflate(?). > At some point of time, the inflate do not return and VM binary memory growth to infinity until it is crashed. > The crash is reproducible all the time with my file. The file is not corrupted and gzip is capable to perform it check and inflate data. The file becomes readable by program if it is inflated - deflated again using command line gzip. The header of readable file is: > 1F 8B 08 00 6E EC E6 56 00 03 D4 BD > I am having a challenge to debug this issue further to zlib and understand root-cause. > Do you have any suggestions on it? > Best Regards, > Dmitry > P.S: the file, I am taking about, contains confidential data and cannot be disclosed to community. > Reference: > [1] http://www.zlib.org/rfc-gzip.html > [2] https://github.com/fogfish/feta/blob/master/src/gz.erl From ulf@REDACTED Tue Mar 15 20:24:34 2016 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 15 Mar 2016 12:24:34 -0700 Subject: [erlang-questions] (no subject) In-Reply-To: References: Message-ID: <53EECA26-265A-4AE6-AB64-78FC4525E262@wiger.net> I have made a PR for this: https://github.com/uwiger/gproc/pull/108 I?m not 100% happy with it, nor does it have a test case for the previously failing case. However, the original test suite passes, and the manual test you describe also works for me (and didn?t before). BR, Ulf W > On 15 Mar 2016, at 02:31, Khitai Pang wrote: > > I have been trying to use gproc with https://github.com/garret-smith/gen_leader_revival . I found that registering a name which is already registered will destroy the existing registration of the name, and a remote node is unaware of this. See the following steps: > > > Register node1 shell as global name 'abcde' on node1, the name is visable on node2 and sending messages from node2 shell to 'abcde' works fine: > > (node1)1> gproc:reg({n, g, abcde}). > true > (node1)2> gproc:where({n, g, abcde}). > <0.229.0> > > (node2)1> gproc:where({n, g, abcde}). > <8049.229.0> > (node2)2> gproc:send({n, g, abcde}, hello). > hello > > (node1)3> flush(). > Shell got hello > ok > > > Now on node1 shell register with the name again, it removes the existing registration: > > (node1)4> gproc:reg({n, g, abcde}). > ** exception error: bad argument > in function gproc:reg/1 > called as gproc:reg({n,g,abcde}) > (node1)5> gproc:where({n, g, abcde}). > undefined > > > But on node2 the shell still gets <8049.229.0>: > > (node2)3> gproc:where({n, g, abcde}). > <8049.229.0> > > > And sending message from node2 shell to 'abcde' still works 'fine', but the node1 shell doesn't get any message: > > (node2)4> gproc:send({n, g, abcde}, hello). > hello > > (node1)6> flush(). > ok > > > Now if I register node1 shell as {n, g, abcde} again, everything works fine again: > > (node1)7> gproc:reg({n, g, abcde}). > true > > (node2)5> gproc:where({n, g, abcde}). > <8049.268.0> > (node2)6> gproc:send({n, g, abcde}, hello). > hello > > (node1) 8> flush(). > Shell got hello > ok > > > This doesn't look right to me. > 1) When registering with a name already registered, why doesn't gproc return something like {error, already_registered}? > 2) After the registration is removed, why does a remove node still sees the non-existent registration? > > > Thanks > Khitai -------------- next part -------------- An HTML attachment was scrubbed... URL: From goyalk@REDACTED Tue Mar 15 20:28:40 2016 From: goyalk@REDACTED (Kapil Goyal) Date: Tue, 15 Mar 2016 19:28:40 +0000 Subject: [erlang-questions] FIPS compliance In-Reply-To: References: Message-ID: <7ec87645f35542debdb738db826fb3c6@EX13-MBX-014.vmware.com> Hi Drew, Thanks for the useful information. Looks like some customization to Erlang will be required even going forward. Regards Kapil From: Drew Varner [mailto:drew.varner@REDACTED] Sent: Tuesday, March 15, 2016 12:07 AM To: Kapil Goyal Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] FIPS compliance Kapil, Erlang's cryptography is not FIPS 140-2-certified. There was a pull request to add FIPS compliance via OpenSSL in FIPS mode, but it stalled [1]. Calls to OpenSSL's crypto canister must go through the Envelope (EVP) API calls. The crypto library in Erlang OTP 19 will use EVP calls (exclusively, I assume) [2]. However, EVP calls alone are not enough for FIPS 140-2 support. If the only problem is terminating incoming HTTP requests, you may be able to get away with proxying the request through a FIPS 140-2 load balancer. rabbitmq-server calls crypto for password hashing. You'd need to replace calls to crypto with calls to a FIPS provider and look for calls made outside of crypto (BIFs like md5, phash). - Drew [1] https://github.com/erlang/otp/pull/377 [2] http://youtu.be/YlNrWxH56_E On Mar 14, 2016, at 10:19 PM, Kapil Goyal > wrote: Hi All, We use RabbitMQ and are working on running it in FIPS compliance. According to a post on RMQ forum (https://groups.google.com/forum/#!topic/rabbitmq-users/wUzUjgDQ9M8), Erlang is not FIPS compliance. Is this correct? If so, are there plans to be compliant in near future? Thanks Kapil _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.pennebaker@REDACTED Tue Mar 15 22:10:04 2016 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 15 Mar 2016 16:10:04 -0500 Subject: [erlang-questions] Built-in io:format syntax for printing lists? In-Reply-To: References: Message-ID: :P On Friday, March 11, 2016, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > This looks like common lisp. In CL, ~a is the "print anything" marker: > > http://www.lispworks.com/documentation/lw50/CLHS/Body/22_cda.htm > > i.e., FORMAT control-strings use ~a for "Aesthetic" printing. Perhaps this > is messed up with the ~p and ~w Sean mentions. > > On Thu, Mar 10, 2016 at 9:40 PM, Sean Cribbs > wrote: > >> I'm pretty sure "~w" or "~p" is what you're looking for. I believe the >> former will prevent printing the list as a string. >> >> On Thu, Mar 10, 2016 at 2:36 PM, Andrew Pennebaker < >> andrew.pennebaker@REDACTED >> > wrote: >> >>> Could Erlang please offer a builtin way to print lists, e.g. for >>> debugging purposes? >>> >>> Related: I swear there used to be "~a" for printing just about anything, >>> but it appears to have been removed, no? >>> >>> -- >>> Cheers, >>> Andrew >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > J. > -- Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From pulsarpietro@REDACTED Tue Mar 15 21:28:14 2016 From: pulsarpietro@REDACTED (Pietro) Date: Tue, 15 Mar 2016 20:28:14 +0000 Subject: [erlang-questions] Beginner question References: <87d1qwk6xf.fsf@posteo.net> Message-ID: <87a8lzpjtt.fsf@posteo.net> dmkolesnikov@REDACTED writes: > Hello, > > You are using atoms instead of variables in your program. Erlang > variables starts with capital letter. > > The receive loop cannot match your message: > > {From, {insert, key, value}} -> > > It expects one triple of atoms but you send different one. > > test:interact(Pid, {insert, testkey, testvalue}). > > Dmitry > > Sent from my iPhone Ouch ! Thank you very much, I knew I were missing something crucial here, would you help me in my next step ? loop(Table) -> receive {From, {insert, Key, Value}} -> io:format("Here insert"), Return = ets:insert(Table, {Key, Value}), From ! Return, loop(Table); {From, {lookup, Key}} -> io:format("Here lookup"), Return = ets:lookup(Table, Key), From ! Return, loop(Table); {From, Any} -> From ! {self(), {error,Any}}, loop(Table) end. This code results in : 1> c(test). {ok,test} 2> Pid = test:start(). <0.40.0> 3> Pid ! {self(), {insert, a, 1}}. Here insert{<0.33.0>,{insert,a,1}} 4> Pid ! {self(), {lookup, a}}. Here lookup{<0.33.0>,{lookup,a}} It does not return the the stored values even though it clearly matches the patterns, where is the rub here ? > > On 15 Mar 2016, at 00:53, Pietro wrote: > > Hi all, > > I have recently started to implement a small academic project in > Erlang > and I have bumped into a problem which seems to show to myself I > haven't > grasped something important about the technology itself. > > That's my code : > > -module(test). > -export([start/0, interact/2]). > > > start() -> > spawn (fun() -> startloop() end). > > > interact(Pid, Request) -> > Pid ! {self(), Request}, > receive > {Pid, Response} -> Response > end. > > > startloop() -> > TableId = ets:new(dictionary, [set]), > loop(TableId). > > loop(Table) -> > receive > {From, {insert, key, value}} -> > From ! ets:insert(Table, {key, value}), > loop(Table); > {From, {lookup, key}} -> > From ! ets:lookup(Table, key), > loop(Table); > Any -> > Any > > end. > > > The problem happens when I try to interact with the server I start > using > the command: > > Pid = test:start(). > > Then I run : > > test:interact(Pid, {insert, testkey, testvalue}). > > My ershell at this point hangs and nothing happens ... what is > happening > ? Please feel free to redirect me to a more appropriate newsgroup > if my > question is not appropriate or if there is a better one. > > Thanks in advance. > > Pietro. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andrey@REDACTED Tue Mar 15 22:45:38 2016 From: andrey@REDACTED (Andrey Tsirulev) Date: Wed, 16 Mar 2016 00:45:38 +0300 Subject: [erlang-questions] [ANN] vprof profiler Message-ID: <56E88282.3000308@artplant.no> Hi all, vprof (Visual Erlang Profiler) is yet another Erlang profiler. https://github.com/artplant/vprof The usage pattern is similar to fprof but results can be viewed with WX-based GUI. Usage: vprof:trace(2). %% collect trace info to vprof.trace file for 2 seconds. This should be executed on the target node. vprof:profile(). %% run on any node; read input from vprof.trace and write summary to vprof.out vprof:gui(). %% show GUI for vprof.out (here's the example: https://raw.githubusercontent.com/artplant/vprof/master/guiexample.png ) The usual vprof usage scenario is the following: Run the stress test for the target node and gather vprof trace, profile it, view results, find potential performance improvements, improve, run the stress test again, trace & profile and compare two sessions by opening two vprof windows via vprof:gui("vprof1.out") and vprof:gui("vprof2.out") at the same time and comparing numbers for a specific function or process (or totals). Best regards, Andrey Tsirulev --- ??? ????????? ????????? ?? ?????? ??????????? Avast. https://www.avast.com/antivirus From rvirding@REDACTED Wed Mar 16 01:25:21 2016 From: rvirding@REDACTED (Robert Virding) Date: Wed, 16 Mar 2016 01:25:21 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: As others have said that this will copy the map from the spawner to the spawnee process. The interesting thing is what you do with afterwards. If you just carry it around as process local data then there is no extra copying except during GC which you can't do anything about. If, however, you then store the map as a map in an ETS table you will be copying the map between the process and the table for every access. This will be very inefficient if you just want to access one key in the map as you will first have to copy the whole map from the ETS table to the process, access the key, and then if you are writing to the map copy it all back to the ETS table. It would then be better to split the map into separate {Key,Value} tuples which you store as separate elements in the ETS table. ETS tables are great but you have to be aware of the copying. Erlang doesn't do shared global data! ETS tables aren't shared, just globally accessible. Robert On 15 March 2016 at 16:07, Alex Howle wrote: > The map is not being sent as a message. It is passed in to the spawned > processes by being in scope of the spawned function. > > Pseudocode: > > A=bigmap, > spawn(fun()-> do_something(A) end). > On 15 Mar 2016 13:43, "Alex Howle" wrote: > >> The map is not being sent as a message. It is passed in to the spawned >> processes by being in scope of the spawned function. >> >> Pseudocode: >> >> A=bigmap, >> spawn(fun()-> do_something(A) end). >> On 15 Mar 2016 11:32, "Sverker Eriksson" >> wrote: >> >>> Each successful ets:lookup call is a copy operation of the entire term >>> from ETS to the process heap. >>> >>> If you are comparing ets:lookup of big map >>> to sending big map in message then I would expect >>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>> is optimized to be faster than copy_struct (used by send). >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>> >>> I've been experiencing an issue and was wondering if anyone else has any >>> experience in this area. I've stripped back the problem to its bare bones >>> for the purposes of this mail. >>> >>> >>> >>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>> short-lived processes to perform the main work of the application. This >>> activity trigger is fired roughly 9 times a second under normal operating >>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>> read from the map. >>> >>> >>> >>> What I've found is that the above implementation has a CPU profile that >>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>> performing the most work. >>> >>> >>> >>> After changing the implementation so that the 25 spawned processes no >>> longer read from the ETS table to retrieve the map structure and, instead >>> the map is passed to the processes on spawn, the CPU usage on the server is >>> considerably lower. >>> >>> >>> >>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Mar 16 01:27:41 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 16 Mar 2016 13:27:41 +1300 Subject: [erlang-questions] Beginner question In-Reply-To: <87d1qwk6xf.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> Message-ID: On 15/03/16 11:53 am, Pietro wrote: > -module(test). > -export([start/0, interact/2]). > > > start() -> > spawn (fun() -> startloop() end). > > > interact(Pid, Request) -> > Pid ! {self(), Request}, > receive > {Pid, Response} -> Response > end. Here you expect the process you are interacting with to return {Pid,Response} pairs, where the first element of the tuple is that process's own Pid. > > > startloop() -> > TableId = ets:new(dictionary, [set]), > loop(TableId). > > loop(Table) -> > receive > {From, {insert, key, value}} -> > From ! ets:insert(Table, {key, value}), Here you do NOT send back what's expected, which is {self(), ets:insert(Table, {key,value})} I suspect that this clause should have been {From, {insert,Key,Value}} -> From ! {self(), ets:insert(Table, {Key,Value})}, loop(Table) > loop(Table); > {From, {lookup, key}} -> > From ! ets:lookup(Table, key), Here you do NOT send back what's expected, which is {self(), ets:lookup(Table, key)} I suspect that this clause should have been {From, {lookup,Key}} -> From ! {self(), ets:lookup(Table, Key)}, loop(Table) > loop(Table); > Any -> > Any > > end. > > > The problem happens when I try to interact with the server I start using > the command: > > Pid = test:start(). > > Then I run : > > test:interact(Pid, {insert, testkey, testvalue}). And here my suspicion is confirmed. Problem 1: Your responses did not conform to your intended protocol. Problem 2: You had constants (key) where you meant to have variables (Key), Your question is fine. These are slips anyone could have made. One thing would have helped to avoid them both, and that is describing the protocol of the new process in a comment. From erlang@REDACTED Wed Mar 16 05:22:30 2016 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 15 Mar 2016 21:22:30 -0700 Subject: [erlang-questions] Beginner question In-Reply-To: <87a8lzpjtt.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> <87a8lzpjtt.fsf@posteo.net> Message-ID: Here's a tip I *always* start writing message processing lops like this: loop(State) -> receive Any -> io:format("***unexpected ~p~n",[Any]), loop(State) end. Then I compile and run this with my problem. Of course, I'll get a printout since the program is incomplete. I then let these printouts guide the order in which I write the program - I add a pattern for the first unmatched message, recompile and run again. When there are no more messages saying things are unmatched I remove the last pattern (or just leave it in). In a published program the last part is most often removed, it was necessary during development but can be removed once you have a correct program. What you see in published examples is the final result - during development it's helpful to add print statements so you can see what's happeing and remove them later. I think if you'd done this you'd have immediately seen what was wrong. Cheers /Joe On Tue, Mar 15, 2016 at 1:28 PM, Pietro wrote: > dmkolesnikov@REDACTED writes: > >> Hello, >> >> You are using atoms instead of variables in your program. Erlang >> variables starts with capital letter. >> >> The receive loop cannot match your message: >> >> {From, {insert, key, value}} -> >> >> It expects one triple of atoms but you send different one. >> >> test:interact(Pid, {insert, testkey, testvalue}). >> >> Dmitry >> >> Sent from my iPhone > Ouch ! Thank you very much, I knew I were missing something crucial > here, would you help me in my next step ? > > loop(Table) -> > receive > {From, {insert, Key, Value}} -> > io:format("Here insert"), > Return = ets:insert(Table, {Key, Value}), > From ! Return, > loop(Table); > {From, {lookup, Key}} -> > io:format("Here lookup"), > Return = ets:lookup(Table, Key), > From ! Return, > loop(Table); > {From, Any} -> > From ! {self(), {error,Any}}, > loop(Table) > end. > > This code results in : > > 1> c(test). > {ok,test} > 2> Pid = test:start(). > <0.40.0> > 3> Pid ! {self(), {insert, a, 1}}. > Here insert{<0.33.0>,{insert,a,1}} > 4> Pid ! {self(), {lookup, a}}. > Here lookup{<0.33.0>,{lookup,a}} > > > It does not return the the stored values even though it clearly matches > the patterns, where is the rub here ? > > >> >> On 15 Mar 2016, at 00:53, Pietro wrote: >> >> Hi all, >> >> I have recently started to implement a small academic project in >> Erlang >> and I have bumped into a problem which seems to show to myself I >> haven't >> grasped something important about the technology itself. >> >> That's my code : >> >> -module(test). >> -export([start/0, interact/2]). >> >> >> start() -> >> spawn (fun() -> startloop() end). >> >> >> interact(Pid, Request) -> >> Pid ! {self(), Request}, >> receive >> {Pid, Response} -> Response >> end. >> >> >> startloop() -> >> TableId = ets:new(dictionary, [set]), >> loop(TableId). >> >> loop(Table) -> >> receive >> {From, {insert, key, value}} -> >> From ! ets:insert(Table, {key, value}), >> loop(Table); >> {From, {lookup, key}} -> >> From ! ets:lookup(Table, key), >> loop(Table); >> Any -> >> Any >> >> end. >> >> >> The problem happens when I try to interact with the server I start >> using >> the command: >> >> Pid = test:start(). >> >> Then I run : >> >> test:interact(Pid, {insert, testkey, testvalue}). >> >> My ershell at this point hangs and nothing happens ... what is >> happening >> ? Please feel free to redirect me to a more appropriate newsgroup >> if my >> question is not appropriate or if there is a better one. >> >> Thanks in advance. >> >> Pietro. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pravin.gohite@REDACTED Tue Mar 15 23:05:46 2016 From: pravin.gohite@REDACTED (Pravin Gohite) Date: Tue, 15 Mar 2016 15:05:46 -0700 Subject: [erlang-questions] Beginner question In-Reply-To: <87a8lzpjtt.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> <87a8lzpjtt.fsf@posteo.net> Message-ID: Hi, Your receive for interact function is trying to match Pid along with Response: interact(Pid, Request) -> Pid ! {self(), Request}, receive *{Pid, Response} -> Response* end. loop(Table) -> receive {From, {insert, key, value}} -> From ! ets:insert(Table, {Key, Value}), %% << you are not sending something to match with Pid loop(Table); {From, {lookup, key}} -> From ! ets:lookup(Table, Key), %% << you are not sending something to match with Pid loop(Table); Any -> Any end. So I think this correct send code should be: >From ! {*self(),* ets:insert(Table, {Key, Value})} and >From ! {*self(), *ets:lookup(Table, Key)}, Thanks, Pravin On Tue, Mar 15, 2016 at 1:28 PM, Pietro wrote: > dmkolesnikov@REDACTED writes: > > > Hello, > > > > You are using atoms instead of variables in your program. Erlang > > variables starts with capital letter. > > > > The receive loop cannot match your message: > > > > {From, {insert, key, value}} -> > > > > It expects one triple of atoms but you send different one. > > > > test:interact(Pid, {insert, testkey, testvalue}). > > > > Dmitry > > > > Sent from my iPhone > Ouch ! Thank you very much, I knew I were missing something crucial > here, would you help me in my next step ? > > loop(Table) -> > receive > {From, {insert, Key, Value}} -> > io:format("Here insert"), > Return = ets:insert(Table, {Key, Value}), > From ! Return, > loop(Table); > {From, {lookup, Key}} -> > io:format("Here lookup"), > Return = ets:lookup(Table, Key), > From ! Return, > loop(Table); > {From, Any} -> > From ! {self(), {error,Any}}, > loop(Table) > end. > > This code results in : > > 1> c(test). > {ok,test} > 2> Pid = test:start(). > <0.40.0> > 3> Pid ! {self(), {insert, a, 1}}. > Here insert{<0.33.0>,{insert,a,1}} > 4> Pid ! {self(), {lookup, a}}. > Here lookup{<0.33.0>,{lookup,a}} > > > It does not return the the stored values even though it clearly matches > the patterns, where is the rub here ? > > > > > > On 15 Mar 2016, at 00:53, Pietro wrote: > > > > Hi all, > > > > I have recently started to implement a small academic project in > > Erlang > > and I have bumped into a problem which seems to show to myself I > > haven't > > grasped something important about the technology itself. > > > > That's my code : > > > > -module(test). > > -export([start/0, interact/2]). > > > > > > start() -> > > spawn (fun() -> startloop() end). > > > > > > interact(Pid, Request) -> > > Pid ! {self(), Request}, > > receive > > {Pid, Response} -> Response > > end. > > > > > > startloop() -> > > TableId = ets:new(dictionary, [set]), > > loop(TableId). > > > > loop(Table) -> > > receive > > {From, {insert, key, value}} -> > > From ! ets:insert(Table, {key, value}), > > loop(Table); > > {From, {lookup, key}} -> > > From ! ets:lookup(Table, key), > > loop(Table); > > Any -> > > Any > > > > end. > > > > > > The problem happens when I try to interact with the server I start > > using > > the command: > > > > Pid = test:start(). > > > > Then I run : > > > > test:interact(Pid, {insert, testkey, testvalue}). > > > > My ershell at this point hangs and nothing happens ... what is > > happening > > ? Please feel free to redirect me to a more appropriate newsgroup > > if my > > question is not appropriate or if there is a better one. > > > > Thanks in advance. > > > > Pietro. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 249505968@REDACTED Wed Mar 16 09:05:45 2016 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Wed, 16 Mar 2016 16:05:45 +0800 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? Message-ID: erlang pattern match file could be written like : -module(config). get(1) -> 11, get(2) -> 22, ..... get(1000001) -> 12312, get(_) -> none. When I need get some data, ets:lookup(some_table, some_value) & config:get(some_value) Which one is more efficiency? How much different between them? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Wed Mar 16 10:05:09 2016 From: vances@REDACTED (Vance Shipley) Date: Wed, 16 Mar 2016 14:35:09 +0530 Subject: [erlang-questions] Beginner question In-Reply-To: <87d1qwk6xf.fsf@posteo.net> References: <87d1qwk6xf.fsf@posteo.net> Message-ID: On Tue, Mar 15, 2016 at 4:23 AM, Pietro wrote: > > loop(Table) -> > receive > {From, {insert, key, value}} -> > From ! ets:insert(Table, {key, value}), > loop(Table); > {From, {lookup, key}} -> > From ! ets:lookup(Table, key), > loop(Table); > Any -> > Any > > end. I recall when beginning with Erlang, many years ago, my first struggle was with the concept of functional programming. Where are my global variables? Yours is not an example of functional style so I am compelled to rewrite your loop/1 function in a couple ways to demonstrate a side effect free functional implementation. First the simplest case where we have a server manage one value: start() -> spawn (fun() -> loop(undefined) end). loop(Value) -> receive {From, {set, NewValue}} -> From ! {self(), {ok, NewValue}}, loop(NewValue); {From, get} -> From ! {self(), {ok, Value}}, loop(Value) end. 1> Pid = a:start(). <0.35.0> 2> a:interact(Pid, get). {ok,undefined} 3> a:interact(Pid, {set, 42}). {ok,42} 4> a:interact(Pid, get). {ok,42} To handle more values use more arguments (e.g. loop/3): start() -> spawn (fun() -> loop(undefined, undefined, undefined) end). loop(A, B, C) -> receive {From, {set, a, Value}} -> From ! {self(), {ok, Value}}, loop(Value, B, C); {From, {set, b, Value}} -> From ! {self(), {ok, Value}}, loop(A, Value, C); {From, {set, c, Value}} -> From ! {self(), {ok, Value}}, loop(A, Value, C); {From, {get, a}} -> From ! {self(), {ok, A}}, loop(A, B, C); {From, {get, b}} -> From ! {self(), {ok, B}}, loop(A, B, C); {From, {get, c}} -> From ! {self(), {ok, C}}, loop(A, B, C); end. To handle many values replace the argument to loop/1 with a data structure chosen to match our use case: start() -> spawn (fun() -> loop(dict:new()) end). loop(Dict) -> receive {From, {set, Key, Value}} -> NewDict = dict:store(Key, Value, Dict), From ! {self(), {ok, Value}}, loop(NewDict); {From, {get, Key}} -> From ! dict:fetch(Key, Dict), loop(Dict) end. ... or: start() -> spawn (fun() -> loop(gb_trees:empty()) end). loop(Tree) -> receive {From, {set, Key, Value}} -> NewTree = gb_trees:enter(Key, Value, Tree), From ! {self(), {ok, Value}}, loop(NewTree); {From, {get, Key}} -> From ! gb_trees:get(Key, Tree), loop(Tree) end. There's nothing wrong with using the erlang term storage (ets) module. I just think it's important to understand and embrace the functional style first or you risk writing terrible imperative code by using ets, or worse the process dictionary, to accomplish global variables. -- -Vance From vychodil.hynek@REDACTED Wed Mar 16 10:05:33 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 16 Mar 2016 10:05:33 +0100 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: Message-ID: If your key is numbers and mostly continuous sequence element(X, {11, 22, ..., 12312}) is the most efficient solution. Use X+K for more different values. When your keys are general terms, try use maps:get/2,3. It is important to have the map or the tuple as literal constant in the module. (Then it's shared between processes.) Always try and measure for your use pattern. On Wed, Mar 16, 2016 at 9:05 AM, ?? <249505968@REDACTED> wrote: > erlang pattern match file could be written like : > > -module(config). > get(1) -> 11, > get(2) -> 22, > ..... > get(1000001) -> 12312, > get(_) -> none. > > When I need get some data, > ets:lookup(some_table, some_value) > & > config:get(some_value) > > Which one is more efficiency? > How much different between them? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Mar 16 10:38:12 2016 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 16 Mar 2016 10:38:12 +0100 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: Message-ID: Your example suggests that the keys are strictly from 1..N If this is the case then the array module *should be* the most efficient. If the array module is not the most efficient and it turned out that (say ets) is more efficient then parts of the array module should be rewritten using ets to reflect this. My point here is not only should you ask what is the most efficient way to do X, but that the solution should be in the place where you expect to find it. If the array indices are in gappy integer sequences then you should use sparce_arrays.erl - but this hasn't been written, so go write it :-) If the array indices are unpredictable use maps or ets - you'd have to measure to find the best way. Cheers /Joe On Wed, Mar 16, 2016 at 9:05 AM, ?? <249505968@REDACTED> wrote: > erlang pattern match file could be written like : > > -module(config). > get(1) -> 11, > get(2) -> 22, > ..... > get(1000001) -> 12312, > get(_) -> none. > > When I need get some data, > ets:lookup(some_table, some_value) > & > config:get(some_value) > > Which one is more efficiency? > How much different between them? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bchesneau@REDACTED Wed Mar 16 10:51:22 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 16 Mar 2016 09:51:22 +0000 Subject: [erlang-questions] some help with hackney and multipart In-Reply-To: References: Message-ID: Sorry for the late answer, between traveling, recovering and welcome back meetings I have been busy :) The way the file is given is the standard way on the web. If you want to pass another name, why not using the form: ({Name, Bin, ExtraHeaders}, If you really need to stream the file, a new pattern can be added I think. Let me know what's you're trying to do. - benoit On Sat, Mar 12, 2016 at 9:02 PM Caragea Silviu wrote: > I tested and what I see is that hackeny also use for files the name > "file". I don't see how I can customize this as time mailgun search for > attachment. > > Also looking to > https://github.com/benoitc/hackney/blob/de35c1ec93dca89568b8b32cf053d82277a90762/src/hackney_multipart.erl > > line 228 this seems hard-coded: > > Disposition = {<<"form-data">>, [{<<"name">>, <<"\"file\"">>}, {<<" > filename">>, <<"\"", FName/binary, "\"">>}]}, > Silviu > > On Wed, Mar 9, 2016 at 12:55 PM, Hynek Vychodil > wrote: > >> Hi, >> Try compare what is sent in body of request made by example with your one >> sent by hackney >> >> curl -s --user 'api:YOUR_API_KEY' \ >> https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages \ >> -F from='Excited User ' \ >> -F to='foo@REDACTED' \ >> -F cc='bar@REDACTED' \ >> -F bcc='baz@REDACTED' \ >> -F subject='Hello' \ >> -F text='Testing some Mailgun awesomness!' \ >> --form-string html='HTML version of the body' \ >> -F attachment=@files/cartman.jpg \ >> -F attachment=@files/cartman.png >> >> Your {multipart, PayloadBase ++ [{file, Attachment}]} doesn't make much >> sense to me. Try something like {multipart, [{form, PayloadBase},{file, >> Attachment}]} (untested). >> >> Hynek >> >> On Wed, Mar 9, 2016 at 11:31 AM, Caragea Silviu >> wrote: >> >>> >>> >>> Hello, >>> >>> I'm trying to send an email via mailgun.com using the hackney and I >>> have some issues sending attachments (which requires multipart). >>> >>> https://documentation.mailgun.com/api-sending.html#sending >>> >>> Basically my interest fields are: >>> >>> from >>> to >>> subject >>> text >>> attachment File attachment. You can post multiple attachment values. >>> Important: You must use multipart/form-data encoding when sending >>> attachments. >>> >>> I tried the following: >>> >>> PayloadBase =[ >>> {<<"from">>, From}, >>> {<<"to">>, To}, >>> {<<"subject">>, Subject}, >>> {<<"text">>, TextBody}, >>> {<<"html">>, HtmlBody} >>> ], >>> >>> Payload = case Attachment of >>> null -> >>> {form, PayloadBase}; >>> _-> >>> {multipart, PayloadBase ++ [{file, Attachment}]} >>> end, >>> >>> But for some reason the attachment is not sent.. Everything else works >>> as expected. >>> I don't see how I can set the filed name to "attachment" as required by >>> mailgun .. at this this is what I suspect beeing wrong >>> >>> Silviu >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Mar 16 11:18:16 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 16 Mar 2016 11:18:16 +0100 Subject: [erlang-questions] [ANN] Syn 1.1.0 - Now with PubSub Message-ID: Dear All, Syn 1.1.0 has been released. On top of being a global Process Registry, Syn is now also able to handle Process Groups and has classical mechanisms that allow Publish / Subscribe patterns. To add a process to a group: syn:join(Name, Pid) -> ok | {error, Error}. Types: Name = any() Pid = pid() Error = pid_already_in_group To remove a process from a group: syn:leave(Name, Pid) -> ok | {error, Error}. Types: Name = any() Pid = pid() Error = undefined | pid_not_in_group To publish a message to all group members: syn:publish(Name, Message) -> {ok, RecipientCount}. Types: Name = any() Message = any() RecipientCount = non_neg_integer() Processes are monitored and removed from groups if they die. Also, the automated resolution of net splits has also been extended to Process Groups. You can find Syn here: https://github.com/ostinelli/syn Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Wed Mar 16 12:12:31 2016 From: kenneth@REDACTED (Kenneth Lundin) Date: Wed, 16 Mar 2016 12:12:31 +0100 Subject: [erlang-questions] Erlang/OTP 18.3 has been released Message-ID: Erlang/OTP 18.3 is a service release on the 18 track with mostly bug fixes, but is does contain a number of new features and characteristics improvements as well. Some highlights of the release are: - New statistics info about runnable and active processes & ports. Call erlang:statistics with: total_run_queue_lengths | run_queue_lengths | total_active_tasks | active_tasks. - Time warp improvements: dbg:p/2 and erlang:trace/3 with monotonic_timestamp |strict_monotonic_timestamp. - Introduced a validation callback for heart. - The module overload in sasl has been deprecated. - ~90 contributions since 18.2 You can find the Release Notes with more detailed info at http://www.erlang.org/download/otp_src_18.3.readme You can download the full source distribution from http://www.erlang.org/download/otp_src_18.3.tar.gz Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README that is part of the distribution. You can also find the source code at github.com in the official Erlang repository. Git tag OTP-18.3 https://github.com/erlang/otp/tree/OTP-18.3 The Windows binary distributions can be downloaded from http://www.erlang.org/download/otp_win32_18.3.exe http://www.erlang.org/download/otp_win64_18.3.exe You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_18.3.tar.gz http://www.erlang.org/download/otp_doc_man_18.3.tar.gz You can also read the documentation on-line here: (see the Release Notes mentioned above for release notes which are not updated in the doc, but the new functionality is) http://www.erlang.org/doc/ We also want to thank those that sent us patches, suggestions and bug reports. If you find bugs in Erlang/OTP report them via the public issue tracker at http://bugs.erlang.org The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From itshowlertime@REDACTED Wed Mar 16 10:43:07 2016 From: itshowlertime@REDACTED (Alex Howle) Date: Wed, 16 Mar 2016 09:43:07 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: <56E7F2C8.1000006@ericsson.com> References: <56E7F2C8.1000006@ericsson.com> Message-ID: Assuming that when you say "win" you mean that ets:lookup should be more efficient (and less CPU intensive) then I'm seeing the opposite. On 15 Mar 2016 11:32, "Sverker Eriksson" wrote: > Each successful ets:lookup call is a copy operation of the entire term > from ETS to the process heap. > > If you are comparing ets:lookup of big map > to sending big map in message then I would expect > ets:lookup to win, as copy_shallow (used by ets:lookup) > is optimized to be faster than copy_struct (used by send). > > > /Sverker, Erlang/OTP > > > On 03/15/2016 09:52 AM, Alex Howle wrote: > > I've been experiencing an issue and was wondering if anyone else has any > experience in this area. I've stripped back the problem to its bare bones > for the purposes of this mail. > > > > I have an Erlang 18.1 application that uses ETS to store an Erlang map > structure. Using erts_debug:flat_size/1 I can approximate the map's size to > be 1MB. Upon the necessary activity trigger the application spawns about 25 > short-lived processes to perform the main work of the application. This > activity trigger is fired roughly 9 times a second under normal operating > conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to > read from the map. > > > > What I've found is that the above implementation has a CPU profile that is > quite "expensive" - each of the CPU cores (40 total comprised of 2 > Processors with 10 hyperthreaded cores) frequently runs at 100%. The > machine in question also has 32GB RAM of which about 9GB is used at peak. > There is no swap usage whatsoever. Examination shows that copy_shallow is > performing the most work. > > > > After changing the implementation so that the 25 spawned processes no > longer read from the ETS table to retrieve the map structure and, instead > the map is passed to the processes on spawn, the CPU usage on the server is > considerably lower. > > > > Can anyone offer advice as to why I'm seeing the differing CPU profiles? > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 249505968@REDACTED Wed Mar 16 12:37:01 2016 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Wed, 16 Mar 2016 19:37:01 +0800 Subject: [erlang-questions] =?gb18030?b?u9i4tKO6ICB3aGljaCBvbmUgaXMgbW9y?= =?gb18030?q?e_efficiency=3F_ets_or_pattern_match=3F?= In-Reply-To: References: Message-ID: thank you for the advise~ The whole thing is that: My team use the ets way before (load a csv file to ets). And now we are thinking about to use erlang beam file to replace it(pattern match),because it's easy to hot code load. But someone is afraid about the efficiency of pattern match.(because we are not very sure about the efficiency of pattern match,just heard that if the code is good, it will be a binary search). And the keys in pattern match will be all the same type (all of it in the same file will be either number or atom, sometime maybe string). If both way is very efficiency, we will choose the pattern match solution(translate the csv file to erlang file). So if the efficiency still pretty good for the pattern match, if they are not strictly from 1..N (and it will always be ascending?like 1, 3,5,6,7...N, or one, two, three)? thank you again~ ------------------ ???? ------------------ ???: "Joe Armstrong";; ????: 2016?3?16?(???) ??5:38 ???: "??"<249505968@REDACTED>; ??: "erlang-questions"; ??: Re: [erlang-questions] which one is more efficiency? ets or pattern match? Your example suggests that the keys are strictly from 1..N If this is the case then the array module *should be* the most efficient. If the array module is not the most efficient and it turned out that (say ets) is more efficient then parts of the array module should be rewritten using ets to reflect this. My point here is not only should you ask what is the most efficient way to do X, but that the solution should be in the place where you expect to find it. If the array indices are in gappy integer sequences then you should use sparce_arrays.erl - but this hasn't been written, so go write it :-) If the array indices are unpredictable use maps or ets - you'd have to measure to find the best way. Cheers /Joe On Wed, Mar 16, 2016 at 9:05 AM, ?? <249505968@REDACTED> wrote: > erlang pattern match file could be written like : > > -module(config). > get(1) -> 11, > get(2) -> 22, > ..... > get(1000001) -> 12312, > get(_) -> none. > > When I need get some data, > ets:lookup(some_table, some_value) > & > config:get(some_value) > > Which one is more efficiency? > How much different between them? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Wed Mar 16 12:58:07 2016 From: vances@REDACTED (Vance Shipley) Date: Wed, 16 Mar 2016 17:28:07 +0530 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaIHdoaWNoIG9uZSBpcyBtb3Jl?= =?utf-8?q?_efficiency=3F_ets_or_pattern_match=3F?= In-Reply-To: References: Message-ID: On Wed, Mar 16, 2016 at 5:07 PM, ?? <249505968@REDACTED> wrote: > But someone is afraid about the efficiency of pattern match.(because we are > not very sure about the efficiency of pattern match,just heard that if the > code is good, it will be a binary search). Pattern matching is quite efficient up to some point of design intent. Some guidance is in the Efficiency Guide here: http://erlang.org/doc/efficiency_guide/functions.html#id68205 What you need to worry about however is in how many clauses heads you have. I once benchmarked this and found that it worked well up to something on the order of thousands but wouldn't compile beyond that. This is the sort of thing which could change for the better or worse with any Erlang release because they've never expressed a guarantee. I haven't tried it lately. -- -Vance From Catenacci@REDACTED Wed Mar 16 13:00:32 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Wed, 16 Mar 2016 08:00:32 -0400 Subject: [erlang-questions] Beginner question Message-ID: > > Message: 6 > Date: Tue, 15 Mar 2016 21:22:30 -0700 > From: Joe Armstrong > To: Pietro > Cc: Erlang > Subject: Re: [erlang-questions] Beginner question > Message-ID: > < > CAANBt-r0K4j5jEG4CsWrYnnNNOTDy0+s6MFfmx4yuUmt6N6RZQ@REDACTED> > Content-Type: text/plain; charset=UTF-8 > > Here's a tip > > I *always* start writing message processing lops like this: > > loop(State) -> > receive > Any -> > io:format("***unexpected ~p~n",[Any]), > loop(State) > end. > > Then I compile and run this with my problem. > This is a great idea Joe! I plan to steal this one! :) -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Wed Mar 16 13:16:34 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 16 Mar 2016 12:16:34 +0000 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaIHdoaWNoIG9uZSBpcyBtb3Jl?= =?utf-8?q?_efficiency=3F_ets_or_pattern_match=3F?= In-Reply-To: References: Message-ID: You can build large tuples and use element/2 on them as well, for example see functions in the bottom of rand.erl. On Wed, Mar 16, 2016 at 12:58 PM Vance Shipley wrote: > On Wed, Mar 16, 2016 at 5:07 PM, ?? <249505968@REDACTED> wrote: > > But someone is afraid about the efficiency of pattern match.(because we > are > > not very sure about the efficiency of pattern match,just heard that if > the > > code is good, it will be a binary search). > > Pattern matching is quite efficient up to some point of design intent. > Some guidance is in the Efficiency Guide here: > http://erlang.org/doc/efficiency_guide/functions.html#id68205 > > What you need to worry about however is in how many clauses heads you > have. I once benchmarked this and found that it worked well up to > something on the order of thousands but wouldn't compile beyond that. > This is the sort of thing which could change for the better or worse > with any Erlang release because they've never expressed a guarantee. > I haven't tried it lately. > > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Mar 16 13:32:54 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 16 Mar 2016 13:32:54 +0100 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaIHdoaWNoIG9uZSBpcyBtb3Jl?= =?utf-8?q?_efficiency=3F_ets_or_pattern_match=3F?= In-Reply-To: References: Message-ID: It is pretty good for lookup tables. Another such implementation: https://github.com/jlouis/eministat/blob/master/src/eministat_analysis.erl#L85-L190 On Wed, Mar 16, 2016 at 1:16 PM, Dan Gudmundsson wrote: > You can build large tuples and use element/2 on them as well, for example > see functions in the bottom of rand.erl. > > On Wed, Mar 16, 2016 at 12:58 PM Vance Shipley wrote: > >> On Wed, Mar 16, 2016 at 5:07 PM, ?? <249505968@REDACTED> wrote: >> > But someone is afraid about the efficiency of pattern match.(because we >> are >> > not very sure about the efficiency of pattern match,just heard that if >> the >> > code is good, it will be a binary search). >> >> Pattern matching is quite efficient up to some point of design intent. >> Some guidance is in the Efficiency Guide here: >> http://erlang.org/doc/efficiency_guide/functions.html#id68205 >> >> What you need to worry about however is in how many clauses heads you >> have. I once benchmarked this and found that it worked well up to >> something on the order of thousands but wouldn't compile beyond that. >> This is the sort of thing which could change for the better or worse >> with any Erlang release because they've never expressed a guarantee. >> I haven't tried it lately. >> >> >> >> -- >> -Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Mar 16 14:53:46 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 16 Mar 2016 09:53:46 -0400 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: Message-ID: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> On 03/16, Joe Armstrong wrote: >If the array module is not the most efficient and it turned out that >(say ets) is more >efficient then parts of the array module should be rewritten using ets >to reflect this. Oh oh oh, this is a dangerous thing! You would be switching the purely functional semantics of the array module into destructive ones using ETS tables! This is what sofs do for sets, and the usage patterns may change entirely! Moreover, passing the data type over to different processes could stop working or have unintended consequences. The array module should *not* be rewritten if ETS is faster, simply because they would behave entirely differently! > >If the array indices are in gappy integer sequences then you should use >sparce_arrays.erl - but this hasn't been written, so go write it :-) > It has been written in that the current `array' module does support sparse arrays, and can also deal with fixed or resizable arrays: http://erlang.org/doc/man/array.html You will find functions for: - initializing sparse arrays - doing sparse folds - doing sparse maps - converting sparse arrays to lists - converting sparse arrays to orddicts The internal representation uses branching trees of tuples and you can test for yourself that the representation may be sparse: 11> array:from_list(lists:zip(lists:seq(5,10)++lists:seq(995,1000), lists:seq(1,12))). {array,12,100,undefined, {{{5,1}, {6,2}, {7,3}, {8,4}, {9,5}, {10,6}, {995,7}, {996,8}, {997,9}, {998,10}}, {{999,11}, {1000,12}, undefined,undefined,undefined,undefined,undefined,undefined, undefined,undefined}, 10,10,10,10,10,10,10,10,10}} There is no need for a second array module, since the current one handles both sparse and non-sparse arrays fine. Regards, Fred. From mikpelinux@REDACTED Wed Mar 16 16:45:38 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Wed, 16 Mar 2016 16:45:38 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> Message-ID: <22249.32674.198390.541898@gargle.gargle.HOWL> Torben Hoffmann writes: > > > Mikael Pettersson writes: > > > [ text/plain ] > > Zachary Kessin writes: > > > I for one would love to see an ML-family language on the beam! > > > > I'm working on a Standard ML (SML'97) compiler targeting the > > Erlang/OTP VM. I hope to have it finished enough to make it > > public sometime this year. > > I have been thinking about this a bit, so I'll throw in some questions. > > What kind of type system are you using? The SML'97 one, with minimal extensions for interfacing with Erlang (see below), and possibly fixes from the Successor-ML effort. > I have been told that a type-and-effect system would be the way to > capture the effects of statements on the mailboxes. > > Have you given any thought to upgrades? > > The problem is that an upgrade can change the types, so that leads to > some thinking about versioning the types. Tricky business. > Without some attention to this it will be hard to evolve a distributed > system. Yes. Basically, we'd need to redo type checking before allowing calls to or from a dynamically upgraded module. Unfortunately the VM doesn't give us a hook for that. I'm still thinking about the best way to address this. Initially I'll ignore the problem. In fact, there isn't even a cast-in-stone mapping between SML and Erlang modules. While I could compile each functor or top-level structure to a module (Erlang-like), I could also compile entire applications to single modules (at least two SML implementations already do that). > And then there is interfacing to the external world. > My instinct tells me that it would make sense to attach some sort of > adapter code that will have the type signature > val adapt: binary -> some_type My plan is to support something like typecase on values of type dynamic, as I've seen other functional languages do before. That's enough to enable safe import of data from the dynamically typed world. > Which leads me to... how will you describe binary pattern matching? Binary matching is already statically typed, so I don't see any problem there. binary_to_term would have result type dynamic. From vychodil.hynek@REDACTED Wed Mar 16 17:18:53 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 16 Mar 2016 17:18:53 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: Just to be sure, when you wrote that erts_debug:flat_size/1 approximate the map's size to be 1MB, you mean something around this value > erts_debug:flat_size(Map) 131072 Because if you have something like > erts_debug:flat_size(Map) 1048576 It is 8MB and your memory IO is like 25*9/s*8MB = 1800MB/s. It is still manageable by your HW but should be considered in your design. On Wed, Mar 16, 2016 at 10:43 AM, Alex Howle wrote: > Assuming that when you say "win" you mean that ets:lookup should be more > efficient (and less CPU intensive) then I'm seeing the opposite. > On 15 Mar 2016 11:32, "Sverker Eriksson" > wrote: > >> Each successful ets:lookup call is a copy operation of the entire term >> from ETS to the process heap. >> >> If you are comparing ets:lookup of big map >> to sending big map in message then I would expect >> ets:lookup to win, as copy_shallow (used by ets:lookup) >> is optimized to be faster than copy_struct (used by send). >> >> >> /Sverker, Erlang/OTP >> >> >> On 03/15/2016 09:52 AM, Alex Howle wrote: >> >> I've been experiencing an issue and was wondering if anyone else has any >> experience in this area. I've stripped back the problem to its bare bones >> for the purposes of this mail. >> >> >> >> I have an Erlang 18.1 application that uses ETS to store an Erlang map >> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >> be 1MB. Upon the necessary activity trigger the application spawns about 25 >> short-lived processes to perform the main work of the application. This >> activity trigger is fired roughly 9 times a second under normal operating >> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >> read from the map. >> >> >> >> What I've found is that the above implementation has a CPU profile that >> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >> machine in question also has 32GB RAM of which about 9GB is used at peak. >> There is no swap usage whatsoever. Examination shows that copy_shallow is >> performing the most work. >> >> >> >> After changing the implementation so that the 25 spawned processes no >> longer read from the ETS table to retrieve the map structure and, instead >> the map is passed to the processes on spawn, the CPU usage on the server is >> considerably lower. >> >> >> >> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From itshowlertime@REDACTED Wed Mar 16 17:20:32 2016 From: itshowlertime@REDACTED (Alex Howle) Date: Wed, 16 Mar 2016 16:20:32 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: I'm on a 64-bit machine, and flat_size returns the number of words. So I took the result and multiplied by 8 to get the number of bytes. On 16 Mar 2016 16:18, "Hynek Vychodil" wrote: > Just to be sure, when you wrote that erts_debug:flat_size/1 approximate > the map's size to be 1MB, you mean something around this value > > > erts_debug:flat_size(Map) > 131072 > > Because if you have something like > > erts_debug:flat_size(Map) > 1048576 > > It is 8MB and your memory IO is like 25*9/s*8MB = 1800MB/s. It is still > manageable by your HW but should be considered in your design. > > On Wed, Mar 16, 2016 at 10:43 AM, Alex Howle > wrote: > >> Assuming that when you say "win" you mean that ets:lookup should be more >> efficient (and less CPU intensive) then I'm seeing the opposite. >> On 15 Mar 2016 11:32, "Sverker Eriksson" >> wrote: >> >>> Each successful ets:lookup call is a copy operation of the entire term >>> from ETS to the process heap. >>> >>> If you are comparing ets:lookup of big map >>> to sending big map in message then I would expect >>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>> is optimized to be faster than copy_struct (used by send). >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>> >>> I've been experiencing an issue and was wondering if anyone else has any >>> experience in this area. I've stripped back the problem to its bare bones >>> for the purposes of this mail. >>> >>> >>> >>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>> short-lived processes to perform the main work of the application. This >>> activity trigger is fired roughly 9 times a second under normal operating >>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>> read from the map. >>> >>> >>> >>> What I've found is that the above implementation has a CPU profile that >>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>> performing the most work. >>> >>> >>> >>> After changing the implementation so that the 25 spawned processes no >>> longer read from the ETS table to retrieve the map structure and, instead >>> the map is passed to the processes on spawn, the CPU usage on the server is >>> considerably lower. >>> >>> >>> >>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Wed Mar 16 17:20:24 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Wed, 16 Mar 2016 17:20:24 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> Message-ID: <56E987C8.7080500@ericsson.com> Well, I would expect copy_shallow (from ETS) to be less CPU intensive than copy_struct (from process). However, as indicated by others, ets:lookup on such a big map will probably trigger a garbage collection on the process, which will lead to yet another copy of the big map. The spawn(fun() -> do_something(BigMap) end) on the other hand will allocate a big enough heap for the process form the start and only do one copy of the big map. /Sverker, Erlang/OTP On 03/16/2016 10:43 AM, Alex Howle wrote: > > Assuming that when you say "win" you mean that ets:lookup should be > more efficient (and less CPU intensive) then I'm seeing the opposite. > > On 15 Mar 2016 11:32, "Sverker Eriksson" > > > wrote: > > Each successful ets:lookup call is a copy operation of the entire term > from ETS to the process heap. > > If you are comparing ets:lookup of big map > to sending big map in message then I would expect > ets:lookup to win, as copy_shallow (used by ets:lookup) > is optimized to be faster than copy_struct (used by send). > > > /Sverker, Erlang/OTP > > > On 03/15/2016 09:52 AM, Alex Howle wrote: >> >> I've been experiencing an issue and was wondering if anyone else >> has any experience in this area. I've stripped back the problem >> to its bare bones for the purposes of this mail. >> >> I have an Erlang 18.1 application that uses ETS to store an >> Erlang map structure. Using erts_debug:flat_size/1 I can >> approximate the map's size to be 1MB. Upon the necessary activity >> trigger the application spawns about 25 short-lived processes to >> perform the main work of the application. This activity trigger >> is fired roughly 9 times a second under normal operating >> conditions. Each of these 25 processes performs 1 x ets:lookup/2 >> calls to read from the map. >> >> What I've found is that the above implementation has a CPU >> profile that is quite "expensive" - each of the CPU cores (40 >> total comprised of 2 Processors with 10 hyperthreaded cores) >> frequently runs at 100%. The machine in question also has 32GB >> RAM of which about 9GB is used at peak. There is no swap usage >> whatsoever. Examination shows that copy_shallow is performing the >> most work. >> >> After changing the implementation so that the 25 spawned >> processes no longer read from the ETS table to retrieve the map >> structure and, instead the map is passed to the processes on >> spawn, the CPU usage on the server is considerably lower. >> >> Can anyone offer advice as to why I'm seeing the differing CPU >> profiles? >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 16 18:33:50 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 16 Mar 2016 18:33:50 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: <56E987C8.7080500@ericsson.com> References: <56E7F2C8.1000006@ericsson.com> <56E987C8.7080500@ericsson.com> Message-ID: I was curious enough to try it: -module(ets_vs_msg). -export([start/1]). -export([ets/2, ets_h/2, msg/2, arg/2]). -define(Tab, ?MODULE). -define(MapSize, 100000). %% 100000 is 2.87 MB start(N) -> Map = gen_map(), ets_init(Map), [[{X, element(1, timer:tc(fun ?MODULE:X/2, [N, Map]))/N} || X <- [ets_h, ets, msg, arg]] || _ <- lists:seq(1, 3)]. gen_map() -> gen_map(?MapSize). gen_map(N) -> maps:from_list([{X, []} || X <- lists:seq(1, N)]). ets_init(Map) -> (catch ets:new(?Tab, [named_table])), ets:insert(?Tab, {foo, Map}). ets(N, _Msg) -> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], [ Pid ! {ets, self()} || Pid <- Pids], [ receive {ok, Pid} -> ok end || Pid <- Pids ]. ets_h(N, Msg) -> Size = 2*erts_debug:flat_size(Msg), Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- lists:seq(1, N) ], [ Pid ! {ets, self()} || Pid <- Pids], [ receive {ok, Pid} -> ok end || Pid <- Pids ]. msg(N, Msg) -> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], [ Pid ! {msg, self(), Msg} || Pid <- Pids], [ receive {ok, Pid} -> ok end || Pid <- Pids ]. arg(N, Msg) -> Pids = [ spawn_link(fun() -> init(Msg) end) || _ <- lists:seq(1, N) ], [ Pid ! {do, self()} || Pid <- Pids], [ receive {ok, Pid} -> ok end || Pid <- Pids ]. init(_) -> loop(). loop() -> receive {ets, From} -> ets:lookup(?Tab, foo), From; {msg, From, _Msg} -> From; {do, From} -> From end ! {ok, self()}. Reading from ets with prepared heap is clear winner: 40> ets_vs_msg:start(1000). [[{ets_h,805.83},{ets,2383.31},{msg,4492.15},{arg,3957.693}], [{ets_h,918.221}, {ets,2379.459}, {msg,4651.258}, {arg,4028.799}], [{ets_h,927.538}, {ets,2370.421}, {msg,4519.885}, {arg,4057.264}]] But there is a catch. If I look to CPU utilisation, only ets_h and ets uses all cores/schedulers (i7 with 4 HT in my case) which indicate that both msg and arg version copy the map from the single process. In my case sending a message from more processes would lead to max 4x speed up for msg and arg version. On Wed, Mar 16, 2016 at 5:20 PM, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > Well, I would expect copy_shallow (from ETS) to be less CPU intensive > than copy_struct (from process). > > However, as indicated by others, ets:lookup on such a big map will probably > trigger a garbage collection on the process, which will lead to > yet another copy of the big map. > > The spawn(fun() -> do_something(BigMap) end) on the other hand will > allocate a big enough heap for the process form the start and only do > one copy of the big map. > > /Sverker, Erlang/OTP > > > > On 03/16/2016 10:43 AM, Alex Howle wrote: > > Assuming that when you say "win" you mean that ets:lookup should be more > efficient (and less CPU intensive) then I'm seeing the opposite. > On 15 Mar 2016 11:32, "Sverker Eriksson" > wrote: > >> Each successful ets:lookup call is a copy operation of the entire term >> from ETS to the process heap. >> >> If you are comparing ets:lookup of big map >> to sending big map in message then I would expect >> ets:lookup to win, as copy_shallow (used by ets:lookup) >> is optimized to be faster than copy_struct (used by send). >> >> >> /Sverker, Erlang/OTP >> >> >> On 03/15/2016 09:52 AM, Alex Howle wrote: >> >> I've been experiencing an issue and was wondering if anyone else has any >> experience in this area. I've stripped back the problem to its bare bones >> for the purposes of this mail. >> >> >> >> I have an Erlang 18.1 application that uses ETS to store an Erlang map >> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >> be 1MB. Upon the necessary activity trigger the application spawns about 25 >> short-lived processes to perform the main work of the application. This >> activity trigger is fired roughly 9 times a second under normal operating >> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >> read from the map. >> >> >> >> What I've found is that the above implementation has a CPU profile that >> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >> machine in question also has 32GB RAM of which about 9GB is used at peak. >> There is no swap usage whatsoever. Examination shows that copy_shallow is >> performing the most work. >> >> >> >> After changing the implementation so that the 25 spawned processes no >> longer read from the ETS table to retrieve the map structure and, instead >> the map is passed to the processes on spawn, the CPU usage on the server is >> considerably lower. >> >> >> >> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 16 18:58:07 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 16 Mar 2016 18:58:07 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> <56E987C8.7080500@ericsson.com> Message-ID: I have tried parallel version of msg and arg msg_p(N, Msg) -> do_p(fun msg/2, N, Msg). arg_p(N, Msg) -> do_p(fun arg/2, N, Msg). do_p(F, N, Msg) -> Schedulers = erlang:system_info(schedulers), Parent = self(), N2 = N div Schedulers, Pids = [spawn_link(fun() -> F(N2, Msg), Parent ! {ok, self()} end) || _ <- lists:seq(1, Schedulers) ], [ receive {ok, Pid} -> ok end || Pid <- Pids]. and it performs better but still worse than ets but I don't know how it would behave on HW with 40 CPUs/schedulers [[{ets_h,787.688}, {ets,2215.42}, {msg_p,2525.365}, {msg,4964.156}, {arg_p,2780.5}, {arg,4248.214}], [{ets_h,901.369}, {ets,2343.145}, {msg_p,2368.203}, {msg,5062.984}, {arg_p,2073.172}, {arg,4260.998}], [{ets_h,906.705}, {ets,2423.889}, {msg_p,3135.662}, {msg,5069.39}, {arg_p,2186.49}, {arg,4268.753}]] Setting initial heap size in msg helps little bit msg(N, Msg) -> Size = 2*erts_debug:flat_size(Msg), Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- lists:seq(1, N) ], [ Pid ! {msg, self(), Msg} || Pid <- Pids], [ receive {ok, Pid} -> ok end || Pid <- Pids ]. [[{ets_h,823.901}, {ets,2200.168}, {msg_p,1974.292}, {msg,4678.855}, {arg_p,2082.779}, {arg,4666.294}], [{ets_h,906.677}, {ets,2033.719}, {msg_p,2092.892}, {msg,4665.692}, {arg_p,2005.953}, {arg,4707.86}], [{ets_h,902.813}, {ets,2290.883}, {msg_p,2041.713}, {msg,4655.373}, {arg_p,2011.422}, {arg,4659.18}]] So I think sending message could be reasonably faster than ets version on HW with 40 CPUs. Anyway storing or sending map this big doesn't seem good design. On Wed, Mar 16, 2016 at 6:33 PM, Hynek Vychodil wrote: > I was curious enough to try it: > > -module(ets_vs_msg). > > -export([start/1]). > > -export([ets/2, ets_h/2, msg/2, arg/2]). > > -define(Tab, ?MODULE). > > -define(MapSize, 100000). %% 100000 is 2.87 MB > > start(N) -> > Map = gen_map(), > ets_init(Map), > [[{X, element(1, timer:tc(fun ?MODULE:X/2, [N, Map]))/N} > || X <- [ets_h, ets, msg, arg]] > || _ <- lists:seq(1, 3)]. > > gen_map() -> > gen_map(?MapSize). > > gen_map(N) -> > maps:from_list([{X, []} || X <- lists:seq(1, N)]). > > ets_init(Map) -> > (catch ets:new(?Tab, [named_table])), > ets:insert(?Tab, {foo, Map}). > > ets(N, _Msg) -> > Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], > [ Pid ! {ets, self()} || Pid <- Pids], > [ receive {ok, Pid} -> ok end || Pid <- Pids ]. > > ets_h(N, Msg) -> > Size = 2*erts_debug:flat_size(Msg), > Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- > lists:seq(1, N) ], > [ Pid ! {ets, self()} || Pid <- Pids], > [ receive {ok, Pid} -> ok end || Pid <- Pids ]. > > msg(N, Msg) -> > Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], > [ Pid ! {msg, self(), Msg} || Pid <- Pids], > [ receive {ok, Pid} -> ok end || Pid <- Pids ]. > > arg(N, Msg) -> > Pids = [ spawn_link(fun() -> init(Msg) end) || _ <- lists:seq(1, N) ], > [ Pid ! {do, self()} || Pid <- Pids], > [ receive {ok, Pid} -> ok end || Pid <- Pids ]. > > init(_) -> > loop(). > > loop() -> > receive > {ets, From} -> > ets:lookup(?Tab, foo), > From; > {msg, From, _Msg} -> > From; > {do, From} -> > From > end ! {ok, self()}. > > Reading from ets with prepared heap is clear winner: > > 40> ets_vs_msg:start(1000). > [[{ets_h,805.83},{ets,2383.31},{msg,4492.15},{arg,3957.693}], > [{ets_h,918.221}, > {ets,2379.459}, > {msg,4651.258}, > {arg,4028.799}], > [{ets_h,927.538}, > {ets,2370.421}, > {msg,4519.885}, > {arg,4057.264}]] > > But there is a catch. If I look to CPU utilisation, only ets_h and ets > uses all cores/schedulers (i7 with 4 HT in my case) which indicate that > both msg and arg version copy the map from the single process. In my case > sending a message from more processes would lead to max 4x speed up for msg > and arg version. > > On Wed, Mar 16, 2016 at 5:20 PM, Sverker Eriksson < > sverker.eriksson@REDACTED> wrote: > >> Well, I would expect copy_shallow (from ETS) to be less CPU intensive >> than copy_struct (from process). >> >> However, as indicated by others, ets:lookup on such a big map will >> probably >> trigger a garbage collection on the process, which will lead to >> yet another copy of the big map. >> >> The spawn(fun() -> do_something(BigMap) end) on the other hand will >> allocate a big enough heap for the process form the start and only do >> one copy of the big map. >> >> /Sverker, Erlang/OTP >> >> >> >> On 03/16/2016 10:43 AM, Alex Howle wrote: >> >> Assuming that when you say "win" you mean that ets:lookup should be more >> efficient (and less CPU intensive) then I'm seeing the opposite. >> On 15 Mar 2016 11:32, "Sverker Eriksson" >> wrote: >> >>> Each successful ets:lookup call is a copy operation of the entire term >>> from ETS to the process heap. >>> >>> If you are comparing ets:lookup of big map >>> to sending big map in message then I would expect >>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>> is optimized to be faster than copy_struct (used by send). >>> >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>> >>> I've been experiencing an issue and was wondering if anyone else has any >>> experience in this area. I've stripped back the problem to its bare bones >>> for the purposes of this mail. >>> >>> >>> >>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>> short-lived processes to perform the main work of the application. This >>> activity trigger is fired roughly 9 times a second under normal operating >>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>> read from the map. >>> >>> >>> >>> What I've found is that the above implementation has a CPU profile that >>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>> performing the most work. >>> >>> >>> >>> After changing the implementation so that the 25 spawned processes no >>> longer read from the ETS table to retrieve the map structure and, instead >>> the map is passed to the processes on spawn, the CPU usage on the server is >>> considerably lower. >>> >>> >>> >>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From itshowlertime@REDACTED Wed Mar 16 21:44:21 2016 From: itshowlertime@REDACTED (Alex Howle) Date: Wed, 16 Mar 2016 20:44:21 +0000 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> <56E987C8.7080500@ericsson.com> Message-ID: Thank you very much for taking the time to experiment with this! Does that mean that splitting the 1MB map into smaller maps would somehow be better, do you think? All parts of the map are required for the processing to be successful. On Wed, Mar 16, 2016 at 5:58 PM, Hynek Vychodil wrote: > I have tried parallel version of msg and arg > > msg_p(N, Msg) -> > do_p(fun msg/2, N, Msg). > > arg_p(N, Msg) -> > do_p(fun arg/2, N, Msg). > > do_p(F, N, Msg) -> > Schedulers = erlang:system_info(schedulers), > Parent = self(), > N2 = N div Schedulers, > Pids = [spawn_link(fun() -> F(N2, Msg), Parent ! {ok, self()} end) > || _ <- lists:seq(1, Schedulers) ], > [ receive {ok, Pid} -> ok end || Pid <- Pids]. > > and it performs better but still worse than ets but I don't know how it > would behave on HW with 40 CPUs/schedulers > > [[{ets_h,787.688}, > {ets,2215.42}, > {msg_p,2525.365}, > {msg,4964.156}, > {arg_p,2780.5}, > {arg,4248.214}], > [{ets_h,901.369}, > {ets,2343.145}, > {msg_p,2368.203}, > {msg,5062.984}, > {arg_p,2073.172}, > {arg,4260.998}], > [{ets_h,906.705}, > {ets,2423.889}, > {msg_p,3135.662}, > {msg,5069.39}, > {arg_p,2186.49}, > {arg,4268.753}]] > > Setting initial heap size in msg helps little bit > > msg(N, Msg) -> > Size = 2*erts_debug:flat_size(Msg), > Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- > lists:seq(1, N) ], > [ Pid ! {msg, self(), Msg} || Pid <- Pids], > [ receive {ok, Pid} -> ok end || Pid <- Pids ]. > > [[{ets_h,823.901}, > {ets,2200.168}, > {msg_p,1974.292}, > {msg,4678.855}, > {arg_p,2082.779}, > {arg,4666.294}], > [{ets_h,906.677}, > {ets,2033.719}, > {msg_p,2092.892}, > {msg,4665.692}, > {arg_p,2005.953}, > {arg,4707.86}], > [{ets_h,902.813}, > {ets,2290.883}, > {msg_p,2041.713}, > {msg,4655.373}, > {arg_p,2011.422}, > {arg,4659.18}]] > > So I think sending message could be reasonably faster than ets version on > HW with 40 CPUs. Anyway storing or sending map this big doesn't seem good > design. > > > On Wed, Mar 16, 2016 at 6:33 PM, Hynek Vychodil > wrote: > >> I was curious enough to try it: >> >> -module(ets_vs_msg). >> >> -export([start/1]). >> >> -export([ets/2, ets_h/2, msg/2, arg/2]). >> >> -define(Tab, ?MODULE). >> >> -define(MapSize, 100000). %% 100000 is 2.87 MB >> >> start(N) -> >> Map = gen_map(), >> ets_init(Map), >> [[{X, element(1, timer:tc(fun ?MODULE:X/2, [N, Map]))/N} >> || X <- [ets_h, ets, msg, arg]] >> || _ <- lists:seq(1, 3)]. >> >> gen_map() -> >> gen_map(?MapSize). >> >> gen_map(N) -> >> maps:from_list([{X, []} || X <- lists:seq(1, N)]). >> >> ets_init(Map) -> >> (catch ets:new(?Tab, [named_table])), >> ets:insert(?Tab, {foo, Map}). >> >> ets(N, _Msg) -> >> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], >> [ Pid ! {ets, self()} || Pid <- Pids], >> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >> >> ets_h(N, Msg) -> >> Size = 2*erts_debug:flat_size(Msg), >> Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- >> lists:seq(1, N) ], >> [ Pid ! {ets, self()} || Pid <- Pids], >> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >> >> msg(N, Msg) -> >> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], >> [ Pid ! {msg, self(), Msg} || Pid <- Pids], >> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >> >> arg(N, Msg) -> >> Pids = [ spawn_link(fun() -> init(Msg) end) || _ <- lists:seq(1, N) ], >> [ Pid ! {do, self()} || Pid <- Pids], >> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >> >> init(_) -> >> loop(). >> >> loop() -> >> receive >> {ets, From} -> >> ets:lookup(?Tab, foo), >> From; >> {msg, From, _Msg} -> >> From; >> {do, From} -> >> From >> end ! {ok, self()}. >> >> Reading from ets with prepared heap is clear winner: >> >> 40> ets_vs_msg:start(1000). >> [[{ets_h,805.83},{ets,2383.31},{msg,4492.15},{arg,3957.693}], >> [{ets_h,918.221}, >> {ets,2379.459}, >> {msg,4651.258}, >> {arg,4028.799}], >> [{ets_h,927.538}, >> {ets,2370.421}, >> {msg,4519.885}, >> {arg,4057.264}]] >> >> But there is a catch. If I look to CPU utilisation, only ets_h and ets >> uses all cores/schedulers (i7 with 4 HT in my case) which indicate that >> both msg and arg version copy the map from the single process. In my case >> sending a message from more processes would lead to max 4x speed up for msg >> and arg version. >> >> On Wed, Mar 16, 2016 at 5:20 PM, Sverker Eriksson < >> sverker.eriksson@REDACTED> wrote: >> >>> Well, I would expect copy_shallow (from ETS) to be less CPU intensive >>> than copy_struct (from process). >>> >>> However, as indicated by others, ets:lookup on such a big map will >>> probably >>> trigger a garbage collection on the process, which will lead to >>> yet another copy of the big map. >>> >>> The spawn(fun() -> do_something(BigMap) end) on the other hand will >>> allocate a big enough heap for the process form the start and only do >>> one copy of the big map. >>> >>> /Sverker, Erlang/OTP >>> >>> >>> >>> On 03/16/2016 10:43 AM, Alex Howle wrote: >>> >>> Assuming that when you say "win" you mean that ets:lookup should be more >>> efficient (and less CPU intensive) then I'm seeing the opposite. >>> On 15 Mar 2016 11:32, "Sverker Eriksson" >>> wrote: >>> >>>> Each successful ets:lookup call is a copy operation of the entire term >>>> from ETS to the process heap. >>>> >>>> If you are comparing ets:lookup of big map >>>> to sending big map in message then I would expect >>>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>>> is optimized to be faster than copy_struct (used by send). >>>> >>>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>>> >>>> I've been experiencing an issue and was wondering if anyone else has >>>> any experience in this area. I've stripped back the problem to its bare >>>> bones for the purposes of this mail. >>>> >>>> >>>> >>>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>>> short-lived processes to perform the main work of the application. This >>>> activity trigger is fired roughly 9 times a second under normal operating >>>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>>> read from the map. >>>> >>>> >>>> >>>> What I've found is that the above implementation has a CPU profile that >>>> is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>>> performing the most work. >>>> >>>> >>>> >>>> After changing the implementation so that the 25 spawned processes no >>>> longer read from the ETS table to retrieve the map structure and, instead >>>> the map is passed to the processes on spawn, the CPU usage on the server is >>>> considerably lower. >>>> >>>> >>>> >>>> Can anyone offer advice as to why I'm seeing the differing CPU profiles? >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 16 22:22:09 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 16 Mar 2016 22:22:09 +0100 Subject: [erlang-questions] ETS and CPU In-Reply-To: References: <56E7F2C8.1000006@ericsson.com> <56E987C8.7080500@ericsson.com> Message-ID: It's hard to tell because we still don't know much about your use case. Where the data in map comes from? How often do they change? How big portion of keys change? How big portion of keys is read in each process? For example, if data in map change less than like once per quarter of an hour and is read heavily I would consider compile module with literal constant. If the data in map changes often but a small portion of keys and also a small portion of keys are read I would definitely store data in ets one key per record which is what Jesper Louis Andersen suggested and so on. It depends heavily on your exact use case. One thing we can tell you for sure, each time you call ets:lookup/2 the whole record from ets is copied in memory. Each time you send a message, the whole message is copied in memory. Each time you spawn process the whole initial data is copied in memory. I made some more measurements. When I use 2 schedulers, sending messages or using spawn are 96% slower than ets:lookup/2. When I use 4 chedulers the difference is only 70% so there is a possibility that wich much more schedulers it could be even faster. I don't have enough data. But it is just out of my curiosity because for your real use case there could be a way better solution. On Wed, Mar 16, 2016 at 9:44 PM, Alex Howle wrote: > Thank you very much for taking the time to experiment with this! > > Does that mean that splitting the 1MB map into smaller maps would somehow > be better, do you think? All parts of the map are required for the > processing to be successful. > > On Wed, Mar 16, 2016 at 5:58 PM, Hynek Vychodil > wrote: > >> I have tried parallel version of msg and arg >> >> msg_p(N, Msg) -> >> do_p(fun msg/2, N, Msg). >> >> arg_p(N, Msg) -> >> do_p(fun arg/2, N, Msg). >> >> do_p(F, N, Msg) -> >> Schedulers = erlang:system_info(schedulers), >> Parent = self(), >> N2 = N div Schedulers, >> Pids = [spawn_link(fun() -> F(N2, Msg), Parent ! {ok, self()} end) >> || _ <- lists:seq(1, Schedulers) ], >> [ receive {ok, Pid} -> ok end || Pid <- Pids]. >> >> and it performs better but still worse than ets but I don't know how it >> would behave on HW with 40 CPUs/schedulers >> >> [[{ets_h,787.688}, >> {ets,2215.42}, >> {msg_p,2525.365}, >> {msg,4964.156}, >> {arg_p,2780.5}, >> {arg,4248.214}], >> [{ets_h,901.369}, >> {ets,2343.145}, >> {msg_p,2368.203}, >> {msg,5062.984}, >> {arg_p,2073.172}, >> {arg,4260.998}], >> [{ets_h,906.705}, >> {ets,2423.889}, >> {msg_p,3135.662}, >> {msg,5069.39}, >> {arg_p,2186.49}, >> {arg,4268.753}]] >> >> Setting initial heap size in msg helps little bit >> >> msg(N, Msg) -> >> Size = 2*erts_debug:flat_size(Msg), >> Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- >> lists:seq(1, N) ], >> [ Pid ! {msg, self(), Msg} || Pid <- Pids], >> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >> >> [[{ets_h,823.901}, >> {ets,2200.168}, >> {msg_p,1974.292}, >> {msg,4678.855}, >> {arg_p,2082.779}, >> {arg,4666.294}], >> [{ets_h,906.677}, >> {ets,2033.719}, >> {msg_p,2092.892}, >> {msg,4665.692}, >> {arg_p,2005.953}, >> {arg,4707.86}], >> [{ets_h,902.813}, >> {ets,2290.883}, >> {msg_p,2041.713}, >> {msg,4655.373}, >> {arg_p,2011.422}, >> {arg,4659.18}]] >> >> So I think sending message could be reasonably faster than ets version on >> HW with 40 CPUs. Anyway storing or sending map this big doesn't seem good >> design. >> >> >> On Wed, Mar 16, 2016 at 6:33 PM, Hynek Vychodil > > wrote: >> >>> I was curious enough to try it: >>> >>> -module(ets_vs_msg). >>> >>> -export([start/1]). >>> >>> -export([ets/2, ets_h/2, msg/2, arg/2]). >>> >>> -define(Tab, ?MODULE). >>> >>> -define(MapSize, 100000). %% 100000 is 2.87 MB >>> >>> start(N) -> >>> Map = gen_map(), >>> ets_init(Map), >>> [[{X, element(1, timer:tc(fun ?MODULE:X/2, [N, Map]))/N} >>> || X <- [ets_h, ets, msg, arg]] >>> || _ <- lists:seq(1, 3)]. >>> >>> gen_map() -> >>> gen_map(?MapSize). >>> >>> gen_map(N) -> >>> maps:from_list([{X, []} || X <- lists:seq(1, N)]). >>> >>> ets_init(Map) -> >>> (catch ets:new(?Tab, [named_table])), >>> ets:insert(?Tab, {foo, Map}). >>> >>> ets(N, _Msg) -> >>> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], >>> [ Pid ! {ets, self()} || Pid <- Pids], >>> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >>> >>> ets_h(N, Msg) -> >>> Size = 2*erts_debug:flat_size(Msg), >>> Pids = [ spawn_opt(fun loop/0, [link, {min_heap_size,Size}]) || _ <- >>> lists:seq(1, N) ], >>> [ Pid ! {ets, self()} || Pid <- Pids], >>> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >>> >>> msg(N, Msg) -> >>> Pids = [ spawn_link(fun loop/0) || _ <- lists:seq(1, N) ], >>> [ Pid ! {msg, self(), Msg} || Pid <- Pids], >>> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >>> >>> arg(N, Msg) -> >>> Pids = [ spawn_link(fun() -> init(Msg) end) || _ <- lists:seq(1, N) >>> ], >>> [ Pid ! {do, self()} || Pid <- Pids], >>> [ receive {ok, Pid} -> ok end || Pid <- Pids ]. >>> >>> init(_) -> >>> loop(). >>> >>> loop() -> >>> receive >>> {ets, From} -> >>> ets:lookup(?Tab, foo), >>> From; >>> {msg, From, _Msg} -> >>> From; >>> {do, From} -> >>> From >>> end ! {ok, self()}. >>> >>> Reading from ets with prepared heap is clear winner: >>> >>> 40> ets_vs_msg:start(1000). >>> [[{ets_h,805.83},{ets,2383.31},{msg,4492.15},{arg,3957.693}], >>> [{ets_h,918.221}, >>> {ets,2379.459}, >>> {msg,4651.258}, >>> {arg,4028.799}], >>> [{ets_h,927.538}, >>> {ets,2370.421}, >>> {msg,4519.885}, >>> {arg,4057.264}]] >>> >>> But there is a catch. If I look to CPU utilisation, only ets_h and ets >>> uses all cores/schedulers (i7 with 4 HT in my case) which indicate that >>> both msg and arg version copy the map from the single process. In my case >>> sending a message from more processes would lead to max 4x speed up for msg >>> and arg version. >>> >>> On Wed, Mar 16, 2016 at 5:20 PM, Sverker Eriksson < >>> sverker.eriksson@REDACTED> wrote: >>> >>>> Well, I would expect copy_shallow (from ETS) to be less CPU intensive >>>> than copy_struct (from process). >>>> >>>> However, as indicated by others, ets:lookup on such a big map will >>>> probably >>>> trigger a garbage collection on the process, which will lead to >>>> yet another copy of the big map. >>>> >>>> The spawn(fun() -> do_something(BigMap) end) on the other hand will >>>> allocate a big enough heap for the process form the start and only do >>>> one copy of the big map. >>>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>>> >>>> On 03/16/2016 10:43 AM, Alex Howle wrote: >>>> >>>> Assuming that when you say "win" you mean that ets:lookup should be >>>> more efficient (and less CPU intensive) then I'm seeing the opposite. >>>> On 15 Mar 2016 11:32, "Sverker Eriksson" >>>> wrote: >>>> >>>>> Each successful ets:lookup call is a copy operation of the entire term >>>>> from ETS to the process heap. >>>>> >>>>> If you are comparing ets:lookup of big map >>>>> to sending big map in message then I would expect >>>>> ets:lookup to win, as copy_shallow (used by ets:lookup) >>>>> is optimized to be faster than copy_struct (used by send). >>>>> >>>>> >>>>> /Sverker, Erlang/OTP >>>>> >>>>> >>>>> On 03/15/2016 09:52 AM, Alex Howle wrote: >>>>> >>>>> I've been experiencing an issue and was wondering if anyone else has >>>>> any experience in this area. I've stripped back the problem to its bare >>>>> bones for the purposes of this mail. >>>>> >>>>> >>>>> >>>>> I have an Erlang 18.1 application that uses ETS to store an Erlang map >>>>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to >>>>> be 1MB. Upon the necessary activity trigger the application spawns about 25 >>>>> short-lived processes to perform the main work of the application. This >>>>> activity trigger is fired roughly 9 times a second under normal operating >>>>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to >>>>> read from the map. >>>>> >>>>> >>>>> >>>>> What I've found is that the above implementation has a CPU profile >>>>> that is quite "expensive" - each of the CPU cores (40 total comprised of 2 >>>>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The >>>>> machine in question also has 32GB RAM of which about 9GB is used at peak. >>>>> There is no swap usage whatsoever. Examination shows that copy_shallow is >>>>> performing the most work. >>>>> >>>>> >>>>> >>>>> After changing the implementation so that the 25 spawned processes no >>>>> longer read from the ETS table to retrieve the map structure and, instead >>>>> the map is passed to the processes on spawn, the CPU usage on the server is >>>>> considerably lower. >>>>> >>>>> >>>>> >>>>> Can anyone offer advice as to why I'm seeing the differing CPU >>>>> profiles? >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Mar 16 23:51:37 2016 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Wed, 16 Mar 2016 23:51:37 +0100 Subject: [erlang-questions] [ANN] geas 2.0.6 Message-ID: <56E9E379.7000708@wanadoo.fr> Hi, Geas 2.0.6 has been released. Geas is a tool that will detect the runnable official Erlang release window for your project. https://github.com/crownedgrouse/geas Main changes since last release : - 18.3 Erlang version : geas database update and detection. - Geas will automatically exclude some notoriously buggy Erlang version if some module/function/arity are concerned. For now only R16B03 with syntax_tools. To disable this feature, simply export GEAS_DISC_RELS=0 environment variable. - Environment variable GEAS_LOG allow to display analyze logs. - Some process dictionary entries are now available after compat run. - Some bug fixes. https://github.com/crownedgrouse/geas/wiki/API-changelog Cheers, Eric From steven.charles.davis@REDACTED Thu Mar 17 01:31:43 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 16 Mar 2016 19:31:43 -0500 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) Message-ID: I do believe that Erlang?s syntax is concise and beautiful. You can?t deny the power of the bit syntax, right? Also, how else do you express ideas in maybe 1/3 of the lines you?d need in any imperative language. I do have an issue that ?strings are just lists of integers? was maybe a wrong path that led to many criticisms. Pattern matching a decode from a binary message is simplified greatly were ?strings? defined to be binaries not lists (as I have spoken on before). I do think maps are really handy (although slightly clumsy), and suspect that frames as proposed by ROK may have been more powerful. I do believe that Garrett has a great handle on appropriate decomposition and maintainabllity. Elixir is great to get Ruby-type ppl on board, but I wish they?d write their libraries in Erlang! Maybe one day they?ll ?see the light?. I wish I was able to provide real solutions to any critique above; but they are trivial compared to the power of the platform, because (as we all know) beyond the syntax wrangles there?s OTP and truly outrageous practical applicability enabled by BEAM. ?OK - got a few things off my chest. Fair assessment? /s From ok@REDACTED Thu Mar 17 01:46:00 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 17 Mar 2016 13:46:00 +1300 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: Message-ID: <61c5f689-23f9-6d8c-df93-c79e287d54a9@cs.otago.ac.nz> On 17/03/16 1:31 pm, Steve Davis wrote: > I do have an issue that ?strings are just lists of integers? was maybe a wrong path that led to many criticisms. Pattern matching a decode from a binary message is simplified greatly were ?strings? defined to be binaries not lists (as I have spoken on before). Anything unfamiliar, whether right or wrong, is going to "lead to many criticisms". Many functional languages have represented strings as lists of atoms or integers, and there are two important reasons for this: (1) lists have a very large number of operations defined on them and are extremely easy to process, so this made strings easy to process (2) the space argument (you mean you take 8 bytes per character shock horror) had some force in the days of 4 MB machines; in the days of 21-bit characters and 16 GB machines, it has much less force. But if space was an issue, strings were the wrong data type anyway. (In fact strings are wrong any time you have structure in the contents that you need to be aware of.) I don't understand your statement about pattern matching from binary messages. It would be absurd to think that ANY abstract data type had to be represented in one and ONLY one way. The classical pointer-to-NUL-terminated-array-of-char is not the only and not always the best representation for strings in C, and the classical list-of-character-codes is not the only and not always the best representation of strings in Erlang. Yawn. From essen@REDACTED Thu Mar 17 02:26:46 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 17 Mar 2016 02:26:46 +0100 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: Message-ID: <56EA07D6.3090801@ninenines.eu> On 03/17/2016 01:31 AM, Steve Davis wrote: > Fair assessment? No. The key point is that Erlang is unfamiliar and so people are not drawn to it. Erlang is unfamiliar because of syntax. Erlang is unfamiliar because it's a concurrent environment. Erlang is unfamiliar because it's not OO. Erlang is unfamiliar because it's used for writing unfamiliar applications (how many people are programming databases or distributed systems? And I don't mean deploying Redis). I suspect Erlang is unfamiliar also because it comes from Sweden, and Swedish people are too humble for their own good. It doesn't really matter that something is the best if you don't make any claims to that effect. To be noticed you need to make a fool of yourself and exaggerate how good your product is (and how bad the other products are). And to make your product familiar you have to do this a lot. And with confidence. MongoDB is a great example. Even if you never used it you are probably familiar with it. You probably also know it will lose your data. Your manager doesn't, though, and that's why you end up using it. Or you don't believe it will affect you (humans have a hard time accepting that the worst could happen; software error, computer crash, or even death). Perhaps you're also just batshit crazy. None of the details matter. Erlang can be the best, or just good enough, it will make very little difference. Syntax details only start to matter when you already decided to learn Erlang, and even then maybe only 5% of people will give up because of it (that's my hope that only 5% of developers are that superficial). All that matters is reaching out to people and trying to appeal to them. But even the latter is not that important. Bad publicity is also good publicity. The best thing that could happen to Erlang right now is for WhatsApp to have a worldwide outage and to publicly blame it on a flaw in Erlang/OTP itself. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From lloyd@REDACTED Thu Mar 17 03:48:53 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 16 Mar 2016 22:48:53 -0400 Subject: [erlang-questions] Erlang in education --- back from the dead Message-ID: Hello, A year or so ago there was on this list a wonderful spirited discussion of how to get kids and educators interested in Erlang. We have now low-cost low-power 64-bit hardware platforms that open boundless opportunities: https://www.raspberrypi.org/blog/raspberry-pi-3-on-sale/ http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438 https://www.pine64.com/product http://www.techspot.com/news/64113-western-digital-creates-314gb-hard-drive-specifically-raspberry.html So, I'm wondering if the Erlang community, particularly our corporate members, would be interested in mounting a programming challenge to students of all ages to develop innovative applications on one or another of these platforms? All entries would be released as open-source. Incentives might range from hardware to cash to internships to jobs. I, in my Writers Glen incarnation, would be willing to seed the program with $500 and devote some time if we can secure enough interest and resources to mount a credible program. Or, short of such a grandiose scheme, I'm wondering if there are folks in the community interested in cooperating off-list in exploring and publishing on-line or in print under Creative Commons license how-tos and tutorials designed to inspire kids and push these platforms to their limits. Best wishes, LRP Sent from my iPad From ok@REDACTED Thu Mar 17 04:28:39 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 17 Mar 2016 16:28:39 +1300 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: <56EA07D6.3090801@ninenines.eu> References: <56EA07D6.3090801@ninenines.eu> Message-ID: <3077c005-df2a-4473-1e3a-bbf883e882fe@cs.otago.ac.nz> On 17/03/16 2:26 pm, Lo?c Hoguin wrote: > Erlang is unfamiliar because it's a concurrent environment. I sometimes think you don't appreciate Erlang until you've struggled with concurrency yourself. Last week I was working on some notes for a course I'm teaching next semester, and needed a concurrent component with a certain interface. I struggled for two *days* trying to make it work. I learned more GDB commands than I ever wished to learn. Eventually I got there. To make sure I wasn't crazy, along the way I implemented in Erlang: 24 line module, 10 line core, first thing I thought of worked, and it was *obviously* right. I keep on having this kind of experience. There are all sorts of things where I'd rather use some other language than Erlang, but for concurrency, give me Erlang every time. From steven.charles.davis@REDACTED Thu Mar 17 11:14:12 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 17 Mar 2016 05:14:12 -0500 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: <56EA07D6.3090801@ninenines.eu> References: <56EA07D6.3090801@ninenines.eu> Message-ID: <29B9FFD3-22FF-43D8-A155-F19F57032A81@gmail.com> > On Mar 16, 2016, at 8:26 PM, Lo?c Hoguin wrote: > > The best thing that could happen to Erlang right now is for WhatsApp to have a worldwide outage and to publicly blame it on a flaw in Erlang/OTP itself. Given the maturity and coherence of the platform, I suspect that outcome would never happen. However, I have been able to use WhatsApp to succinctly ?sell" Erlang/OTP. q: but will it scale? a: WhatsApp q: WhatsApp? a: same platform q: Ah... OK /s From steven.charles.davis@REDACTED Thu Mar 17 11:53:17 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 17 Mar 2016 05:53:17 -0500 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) Message-ID: > ROK said: > Yawn. (What am I doing trying to argue with ROK??? Am I MAD?) 1) Why is it people rant about "string handling" in Erlang? 2) Principle of least surprise: 1> [H|T] = [22,87,65,84,33]. [22,87,65,84,33] 2> H. 22 3> T. "WAT!? 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). Without tagging, merely parsing out a string as a list is not perfectly reversible. 4) What is the right way to implement the function is_string(List) correctly? *ducks* /s -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Thu Mar 17 12:47:12 2016 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Thu, 17 Mar 2016 07:47:12 -0400 Subject: [erlang-questions] [ANN] jwalk 1.1.2 - Helper module for working with Erlang representations of JSON: eep-18, Maps, mochijson-style and proplists representations Message-ID: Jwalk is a helper module for working with Erlang representations of JSON: eep-18, maps, mochijson-style and proplists representations. It is very similar to ej (https://github.com/seth/ej). I have been using jwalk in production for about 4 months without issue. Having said that, I would still consider it to NOT have had robust, in-the-field vetting. Ej is great, but it did not have Map support, and I wasn't smart enough to add it to that code base (plus it has been forked a lot, and I couldn't decide whose fork was most likely to be maintained). Like ej, jwalk implements delete, get, set and set_p functions. I'm not in the class of Erlang developer that many (most) of you are, but I have been using this library successfully, and if it can help anyone else, I would be happy. https://github.com/jr0senblum/jwalk https://hex.pm/packages/jwalk Any feedback is welcome jr0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Mar 17 13:24:30 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 17 Mar 2016 08:24:30 -0400 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: Message-ID: <20160317122428.GM31558@fhebert-ltm2.internal.salesforce.com> On 03/17, Steve Davis wrote: >3) A codec should be perfectly reversible i.e. X = encode(decode(X)). >Without tagging, merely parsing out a string as a list is not perfectly >reversible. > >4) What is the right way to implement the function is_string(List) correctly? > Those two are kind of funny tricky because 'String' as a standalone type does not convery enough information yet. What you need to be aware of, for example, is encoding: is it ISO-8859-1 (latin1), a flavor of unicode (UTF-8, UTF-16, UTF-32, UCS-* etc.), Other variants such as CP-1252, plain ASCII, and so on. Erlang lists also let you specify strings as raw unicode codepoint sequences rather than under any specific encoding. Many of these will share the same basic format, such that "Hello, World!" shows up the same in most of these (if you omit byte-order marks) such that you cannot *detect* that information from the data, it has to be carried from the input or specification in most cases. Then, once it's in place, you can tag it appropriately or make sure you know the meaning. is_string(List) may tell you true or false, but the result there does not tell you whether you can do anything with it in your libraries, merge them together, or make sure they have been normalized to fixed point. Strings are trickier than that, sadly. From bjorn@REDACTED Thu Mar 17 13:47:34 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 17 Mar 2016 13:47:34 +0100 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> References: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On Wed, Mar 16, 2016 at 2:53 PM, Fred Hebert wrote: > Oh oh oh, this is a dangerous thing! You would be switching the purely > functional semantics of the array module into destructive ones using ETS > tables! This is what sofs do for sets, and the usage patterns may change > entirely! Not sure what you mean with the comment about sofs. sofs is purely functional and does not use ets. It has a few functions that convert to and from digraphs (and the digraph uses ets), but there is no direct use of ets. /Bj?rn P.S. As one of the biggest fans of the sofs module, I thought I had to defend sofs. :-) -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Thu Mar 17 13:56:30 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 17 Mar 2016 13:56:30 +0100 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On Thu, Mar 17, 2016 at 1:47 PM, Bj?rn Gustavsson wrote: > P.S. As one of the biggest fans of the sofs module, > I thought I had to defend sofs. :-) > I think the sofs module is one I should cover in a blog post. It is one of the best modules we have. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Thu Mar 17 14:01:52 2016 From: garry@REDACTED (Garry Hodgson) Date: Thu, 17 Mar 2016 09:01:52 -0400 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: <3077c005-df2a-4473-1e3a-bbf883e882fe@cs.otago.ac.nz> References: <56EA07D6.3090801@ninenines.eu> <3077c005-df2a-4473-1e3a-bbf883e882fe@cs.otago.ac.nz> Message-ID: <56EAAAC0.90704@research.att.com> On 3/16/16 11:28 PM, Richard A. O'Keefe wrote: > I sometimes think you don't appreciate Erlang until you've struggled > with concurrency > yourself. Last week I was working on some notes for a course I'm > teaching next > semester, and needed a concurrent component with a certain interface. > I struggled > for two *days* trying to make it work. I learned more GDB commands > than I ever > wished to learn. Eventually I got there. To make sure I wasn't > crazy, along the > way I implemented in Erlang: 24 line module, 10 line core, first thing > I thought of > worked, and it was *obviously* right. I keep on having this kind of > experience. I can relate. While some of our core components are in Erlang, we use Java (shudder) and Python in other parts. Among those other parts are api proxies that translate between our standard api's and vendor-specific ones for various security appliances. Building those proxies in Python is a joy. They're easy, simple, succinct. Until you need concurrency, and scale. The threading and multiprocessing modules are simple, but you find yourself doing klunky things to solve basic problems, and even when it works you worry about what isn't being handled that will bite you when things get dodgy. I have confidence that our Erlang code will handle Bad Things with grace. Our Python code? Probably/maybe/hope for the best. From vances@REDACTED Thu Mar 17 14:33:56 2016 From: vances@REDACTED (Vance Shipley) Date: Thu, 17 Mar 2016 19:03:56 +0530 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On Thu, Mar 17, 2016 at 6:26 PM, Jesper Louis Andersen wrote: > On Thu, Mar 17, 2016 at 1:47 PM, Bj?rn Gustavsson wrote: >> >> P.S. As one of the biggest fans of the sofs module, >> I thought I had to defend sofs. :-) > > > I think the sofs module is one I should cover in a blog post. It is one of > the best modules we have. That module vexes me. I wanted a set of sets data structure in my HSS project but I was never able to figure out how to use the sofs module. Reading that modules documentation humbles me to the point where I feel I am not qualified to be a software engineer. If either of you could show e how to use it here I'd be thankful: https://github.com/vances/hss/blob/master/src/hss.erl hss:add_implicit_registration_set/2 -- -Vance From bchesneau@REDACTED Thu Mar 17 14:45:29 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 17 Mar 2016 13:45:29 +0000 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: Message-ID: in somewhat related i studied recently the impact of each for a simple lookup in erlang-idna. The main goal was to find the best write-once read-only K/V data structure to embed in a beam without the cost of ets or a gen_server: https://github.com/benoitc/erlang-idna/issues/12 between pattern matching and maps on a string with 29915 entries maps were faster and - not sure why - it creates a smaller beam. Maps are also faster to compile and use less memory in the process. anyway you should probably test the right one for your case. - beno?t On Wed, 16 Mar 2016 at 09:05, ?? <249505968@REDACTED> wrote: > erlang pattern match file could be written like : > > -module(config). > get(1) -> 11, > get(2) -> 22, > ..... > get(1000001) -> 12312, > get(_) -> none. > > When I need get some data, > ets:lookup(some_table, some_value) > & > config:get(some_value) > > Which one is more efficiency? > How much different between them? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Thu Mar 17 16:07:05 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 17 Mar 2016 10:07:05 -0500 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: <20160317122428.GM31558@fhebert-ltm2.internal.salesforce.com> References: <20160317122428.GM31558@fhebert-ltm2.internal.salesforce.com> Message-ID: Indeed! Which is, I think, why they are best left as the ?opaque" binaries that they are when parsing? and leaving the final decision about their content to the presentation layer that they are implicitly targeting. /s > On Mar 17, 2016, at 7:24 AM, Fred Hebert wrote: > > On 03/17, Steve Davis wrote: >> 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). >> Without tagging, merely parsing out a string as a list is not perfectly >> reversible. >> >> 4) What is the right way to implement the function is_string(List) correctly? >> > > Those two are kind of funny tricky because 'String' as a standalone type does not convery enough information yet. > > What you need to be aware of, for example, is encoding: is it ISO-8859-1 (latin1), a flavor of unicode (UTF-8, UTF-16, UTF-32, UCS-* etc.), Other variants such as CP-1252, plain ASCII, and so on. Erlang lists also let you specify strings as raw unicode codepoint sequences rather than under any specific encoding. > > Many of these will share the same basic format, such that "Hello, World!" shows up the same in most of these (if you omit byte-order marks) such that you cannot *detect* that information from the data, it has to be carried from the input or specification in most cases. Then, once it's in place, you can tag it appropriately or make sure you know the meaning. > > is_string(List) may tell you true or false, but the result there does not tell you whether you can do anything with it in your libraries, merge them together, or make sure they have been normalized to fixed point. > > Strings are trickier than that, sadly. From mononcqc@REDACTED Thu Mar 17 18:42:19 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 17 Mar 2016 13:42:19 -0400 Subject: [erlang-questions] which one is more efficiency? ets or pattern match? In-Reply-To: References: <20160316135345.GJ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: <20160317174218.GP31558@fhebert-ltm2.internal.salesforce.com> On 03/17, Bj?rn Gustavsson wrote: > >Not sure what you mean with the comment about sofs. > >sofs is purely functional and does not use ets. >It has a few functions that convert to and from >digraphs (and the digraph uses ets), but there is >no direct use of ets. > You're right, I did mean digraph. Sofs is a great module with the worst documentation available *unless* you already know set theory pretty well, then it translates pretty directly to the mathematical concepts. But pick a random wikipedia page for a mathematical concept or theorem you know little about, and it will be likely as readable -- or less so -- than sofs is for people not in the know for set theory. From khitai.pang@REDACTED Thu Mar 17 19:05:26 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Fri, 18 Mar 2016 02:05:26 +0800 Subject: [erlang-questions] gproc registration 'partitioning'? In-Reply-To: <53EECA26-265A-4AE6-AB64-78FC4525E262@wiger.net> References: <53EECA26-265A-4AE6-AB64-78FC4525E262@wiger.net> Message-ID: The problem is also observed when registering the same property again: (node1)3> gproc:reg({p, g, mytopic}). true ..send a message to this property on a remote node.. (node1)4> flush(). Shell got {mytopic,hello} ok (node1)5> gproc:reg({p, g, mytopic}). ** exception error: bad argument in function gproc:reg/1 called as gproc:reg({p,g,mytopic}) ..send a message to this property on a remote node.. (node1)6> flush(). ok Thanks Khitai On 2016/3/16 3:24, Ulf Wiger wrote: > I have made a PR for this: > > https://github.com/uwiger/gproc/pull/108 > > I?m not 100% happy with it, nor does it have a test case for the > previously failing case. > However, the original test suite passes, and the manual test you > describe also works for me (and didn?t before). > > BR, > Ulf W -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Mar 17 19:09:24 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 17 Mar 2016 12:09:24 -0600 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: References: Message-ID: On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin wrote: > Erlang/OTP 18.3 is a service release on the 18 track with mostly bug fixes, > but is does contain a number of new features and characteristics > improvements > as well. > > I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. I then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in gzip format. I'm on centos so I double-clicked it and was able to view and extract in the ark tool. How was it compressed? -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Thu Mar 17 19:46:20 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 17 Mar 2016 19:46:20 +0100 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: References: Message-ID: tar xzf worked for me. Have you checked the checksum? Cheers, -- Pierre Fenoll On 17 March 2016 at 19:09, Wes James wrote: > > > On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin > wrote: > >> Erlang/OTP 18.3 is a service release on the 18 track with mostly bug >> fixes, >> but is does contain a number of new features and characteristics >> improvements >> as well. >> >> > I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. I > then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in gzip > format. I'm on centos so I double-clicked it and was able to view and > extract in the ark tool. How was it compressed? > > -wes > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Mar 17 19:58:48 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 17 Mar 2016 19:58:48 +0100 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: References: Message-ID: <56EAFE68.6060506@ninenines.eu> People have reported issues recently with downloading using certain clients (like browsers) where the file is apparently sent uncompressed, if I understood right. On 03/17/2016 07:09 PM, Wes James wrote: > > > On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin > wrote: > > Erlang/OTP 18.3 is a service release on the 18 track with mostly bug > fixes, > but is does contain a number of new features and characteristics > improvements > as well. > > > I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. > I then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in > gzip format. I'm on centos so I double-clicked it and was able to view > and extract in the ark tool. How was it compressed? > > -wes > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From comptekki@REDACTED Thu Mar 17 20:11:39 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 17 Mar 2016 13:11:39 -0600 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: <56EAFE68.6060506@ninenines.eu> References: <56EAFE68.6060506@ninenines.eu> Message-ID: On Thu, Mar 17, 2016 at 12:58 PM, Lo?c Hoguin wrote: > > People have reported issues recently with downloading using certain clients (like browsers) where the file is apparently sent uncompressed, if I understood right. > > On 03/17/2016 07:09 PM, Wes James wrote: >> >> >> >> On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin > > wrote: >> >> Erlang/OTP 18.3 is a service release on the 18 track with mostly bug >> fixes, >> but is does contain a number of new features and characteristics >> improvements >> as well. >> >> >> I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. >> I then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in >> gzip format. I'm on centos so I double-clicked it and was able to view >> and extract in the ark tool. How was it compressed? >> >> -wes >> That was it. I did tar -xvf and it worked. Chrome on centos must have done the gzip decompression for me. It hasn't done that before. I actually don't want it too. I'm need to check how to stop that. Thanks for the tip Lo?c! -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Mar 17 20:13:26 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 17 Mar 2016 13:13:26 -0600 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: References: <56EAFE68.6060506@ninenines.eu> Message-ID: On Thu, Mar 17, 2016 at 1:11 PM, Wes James wrote: > > > On Thu, Mar 17, 2016 at 12:58 PM, Lo?c Hoguin wrote: > > > > People have reported issues recently with downloading using certain > clients (like browsers) where the file is apparently sent uncompressed, if > I understood right. > > > > On 03/17/2016 07:09 PM, Wes James wrote: > >> > >> > >> > >> On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin >> > wrote: > >> > >> Erlang/OTP 18.3 is a service release on the 18 track with mostly bug > >> fixes, > >> but is does contain a number of new features and characteristics > >> improvements > >> as well. > >> > >> > >> I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. > >> I then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in > >> gzip format. I'm on centos so I double-clicked it and was able to view > >> and extract in the ark tool. How was it compressed? > >> > >> -wes > >> > > I just tried on my mac and it worked like I expected. > > -wes > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Mar 17 20:15:25 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 17 Mar 2016 13:15:25 -0600 Subject: [erlang-questions] Erlang/OTP 18.3 has been released In-Reply-To: References: <56EAFE68.6060506@ninenines.eu> Message-ID: On Thu, Mar 17, 2016 at 1:13 PM, Wes James wrote: > > > On Thu, Mar 17, 2016 at 1:11 PM, Wes James wrote: > >> >> >> On Thu, Mar 17, 2016 at 12:58 PM, Lo?c Hoguin wrote: >> > >> > People have reported issues recently with downloading using certain >> clients (like browsers) where the file is apparently sent uncompressed, if >> I understood right. >> > >> > On 03/17/2016 07:09 PM, Wes James wrote: >> >> >> >> >> >> >> >> On Wed, Mar 16, 2016 at 5:12 AM, Kenneth Lundin > >> > wrote: >> >> >> >> Erlang/OTP 18.3 is a service release on the 18 track with mostly >> bug >> >> fixes, >> >> but is does contain a number of new features and characteristics >> >> improvements >> >> as well. >> >> >> >> >> >> I tried the standard tar -zxf otp_src_18.3.tar.gz and it had an error. >> >> I then tried gzip -dc otp_src_18.3.tar.gz and it said it was not in >> >> gzip format. I'm on centos so I double-clicked it and was able to view >> >> and extract in the ark tool. How was it compressed? >> >> >> >> -wes >> >> >> >> Well that's kind of irritating :( http://superuser.com/questions/985870/prevent-chrome-to-unzip-files-while-downloading -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Mar 18 00:50:35 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 18 Mar 2016 12:50:35 +1300 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: Message-ID: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> On 17/03/16 11:53 pm, Steve Davis wrote: > > ROK said: > > Yawn. > (What am I doing trying to argue with ROK??? Am I MAD?) > > 1) Why is it people rant about "string handling" in Erlang? Because it is not the same as Java. > > 2) Principle of least surprise: > 1> [H|T] = [22,87,65,84,33]. > [22,87,65,84,33] > 2> H. > 22 > 3> T. > "WAT!? This is a legitimate complaint, but it confuses two things. There is *STRING HANDLING*, which is fine, and there is *LIST PRINTING*, which causes the confusion. For comparison, DEC-10 Prolog, PDP-11 Prolog, C-Prolog, and Quintus Prolog all did STRING HANDLING as lists of character codes, but all did LIST PRINTING without ever converting lists of numbers to strings. The answer was that there was a library procedure to print a list of integers as a string and you could call that whenever you wanted to, such as in a user-defined pretty-printing procedure. Here's a transcript from SICStus Prolog: | ?- write([65,66,67]). [65,66,67] yes | ?- write("ABC"). [65,66,67] yes The heuristic used by the debugger in some Prologs was that a list of integers between 32 and 126 inclusive was printed as a string; that broke down with Latin 1, and broke harder with Unicode. The simple behaviour mandated by the standard that lists of integers print as lists of integers confuses people once, then they learn that string quotes are an input notation, not an output notation, and if they want string notation in output, they have to call a special procedure to get it. The ISO Prolog committee introduced a horrible alternative which the DEC-10 Prolog designers had experienced in some Lisp systems and learned to hate: flip a switch and "ABC" is read as ['A','B','C']. The principal reason given for that was that the output was semi-readable. One of my arguments against it was that this required every Prolog system to be able to hold 17*2**16 atoms, and I new for a fact that many would struggle to do so. The retort was "they must be changed to make a special case for one-character atoms". Oh well, no silver bullet. That does serve as a reminder, though, that using [a,b,c] instead of [$a,$b,$c] is *possible* in Erlang. Just to repeat the basic point: the printing of (some) integer lists as strings is SEPARABLE from the issue of how strings are represented and processed; that could be changed without anything else in the language changing. > > 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). > Without tagging, merely parsing out a string as a list is not > perfectly reversible. Here you are making a demand that very few other programming languages can support. For example, take JavaScript. "\u0041" is read as "A", and you are not going to get "\u0041" back from "A". You're not even going to get "\x41" back from it, even though "\x41" == "A". Or take Erlang, where 1> 'foo bar'. 'foo bar' 2> 'foobar'. foobar with the same kind of thing happening in Prolog. And of COURSE reading [1 /* one */, 2 /* deux */, 4 /* kvar */] in JavaScript preserves the comments so that re-encoding the data structure restores the input perfectly. Or for that matter consider floating point numbers, where even the languages that produce the best possible conversions cannot promise that encode(decode(x)) == x. No, I'm sorry, this "perfectly reversible codec" requirement sets up a standard that NO programming language I'm aware of satisfies. It is, in fact, a straw man. What you *can* ask, and what some language designers and implementers strive to give you, is decode(encode(decode(x))) == decode(x). But to repeat the point made earlier, the way that lists of plausible character codes is printed is SEPARABLE from the way strings are represented and handled and in an ancestral language is SEPARATE. > > 4) What is the right way to implement the function is_string(List) > correctly? > > *ducks* That really is a "have you stopped beating your wife, answer yes or no" sort of question. It depends on the semantics you *want* it to have. The Quintus library didn't provide any such predicate, but it did provide plausible_chars(Term) when Term is a sequence of integers satisfying is_graphic(C) or is_space(C), possibly ending with a tail that is a variable or a variable bound by numbervars/3. Notice the careful choice of name: not IS (certainly) a string, but is a PLAUSIBLE list of characters. It was good enough for paying customers to be happy with the module it was part of (which was the one offering the non-usual portray_chars(Term) command). One of the representations Quintus used for strings (again, a library feature, not a core language feature) was in Erlang notation {external_string,FileName,Offset,Length}, and idea that struck the customer I developed it for as a great innovation, when I'd simply stolen it from Smalltalk! The thing is that STRINGS ARE WRONG for most things, however represented. For example, when Java changed the representation of String so that slicing became a costly operation, I laughed, because I had my own representation of strings that provided O(1) concatenation as well as cheap slicing. (Think Erlang iolists and you won't be far wrong.) The Pop2 language developed and used at Edinburgh represented file names as lists, e.g., [/dev/null] was in Erlang notation ['/',dev.'/',null]. This made file name manipulation easier than representing them as strings. Any time there is internal structure, any time there is scope for sharing substructure, any time you need to process the parts of a string, strings are wrong. The PERL lesson is that regular expressions are a fantastic tool for doing the wrong thing quite simply. From spanemangalore@REDACTED Fri Mar 18 05:53:06 2016 From: spanemangalore@REDACTED (Sachin Panemangalore) Date: Thu, 17 Mar 2016 21:53:06 -0700 Subject: [erlang-questions] Dialyzer type specifications for macros with bitstrings makes erlc barf Message-ID: 1. The following compiles fine in Erlang. -define(MY_INT_MACRO, 1). -define(MY_INT_MACRO, 2). -type my_type() :: ?MY_INT_MACRO1 | ?MY_INT_ MACRO2 . 2. The following doesn't -define(MY_BITSTRING_MACRO, <<"01">>). -define(MY_BITSTRING_MACRO, <<"02">>). -type my_type() :: ?MY_BITSTRING_MACRO1 | ?MY_BITSTRING_ MACRO2 . Why ? and is there a way to make this work ? Regards sachin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Mar 18 06:52:21 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 18 Mar 2016 14:52:21 +0900 Subject: [erlang-questions] Dialyzer type specifications for macros with bitstrings makes erlc barf In-Reply-To: References: Message-ID: <5860244.2BDD7qnhcW@changa> On 2016?3?17? ??? 21:53:06 Sachin Panemangalore wrote: > 1. The following compiles fine in Erlang. > > -define(MY_INT_MACRO1, 1). > -define(MY_INT_MACRO2, 2). > -type my_type() :: ?MY_INT_MACRO1 | ?MY_INT_MACRO2 . > > > 2. The following doesn't > > -define(MY_BITSTRING_MACRO1, <<"01">>). > -define(MY_BITSTRING_MACRO2, <<"02">>). > -type my_type() :: ?MY_BITSTRING_MACRO1 | ?MY_BITSTRING_ MACRO2 . > > Why ? and is there a way to make this work ? (Changed the names above to say what I think you meant.) Type declarations have a few limitations with regard to defining what is inside. You can declare the size of a binary, but not what it contains, hence these sort of type definitions: https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl#L47-L56 In the same way, you can declare a dictionary or map's k/v types: -type my_dict() :: dict:dict(atom(), integer()). -type my_map() :: maps:map(atom(), string()). You'll never be able to provide a complete, fixed shape of a dict or map, though, because that would actually be a schema, and that's what using tuple or records as types already does. This only turns out to be the same limitation as we have defining lists of things: -type my_tuple_list() :: [{atom(), some_type()}]. This is why the following is not legal: -type some_phrase() :: "I am a string literal.". Actually, I think you can't even declare a list of some specific length, which leaves binary type definitions slightly more rigorous than lists. But if you *knew* the length of a list why wouldn't you be using a tuple? It would be pretty convenient to be able to define types based on binary literals, since their use case differs from binaries. But on the other hand, if you want binary strings defined this way you usually actually want atoms that you can derive from some external binary data you receive -- and atoms are (usually) an excellent way to clean up your type defs in self-documenting ways. -Craig From Catenacci@REDACTED Fri Mar 18 12:56:08 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Fri, 18 Mar 2016 07:56:08 -0400 Subject: [erlang-questions] [ANN] Erlang Chocolatey NuGet Package Updated Message-ID: For the five folks besides me that use Erlang on Windows :) I just wanted to let you know that the Chocolatey NuGet Package for Erlang is up to 18.3. https://chocolatey.org/packages/erlang/18.3 -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From emil@REDACTED Fri Mar 18 17:30:02 2016 From: emil@REDACTED (Emil Holmstrom) Date: Fri, 18 Mar 2016 16:30:02 +0000 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> References: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> Message-ID: I am probably repeating what someone else already have said in some other similar thread. The confusion between strings and [integer()] would have been greatly reduced if char() existed, $a wouldn't have to be syntactic sugar for 97 but would actually be "character a". You would have to explicitly convert char() -> integer() and wise versa. This is how strings are implemented in ML and Haskell. Regarding character encoding: inside Erlang Unicode could always be assumed, converson between different character encodings could be done on I/O. /emil On Fri, 18 Mar 2016 at 00:51, Richard A. O'Keefe wrote: > > > On 17/03/16 11:53 pm, Steve Davis wrote: > > > ROK said: > > > Yawn. > > (What am I doing trying to argue with ROK??? Am I MAD?) > > > > 1) Why is it people rant about "string handling" in Erlang? > > Because it is not the same as Java. > > > > 2) Principle of least surprise: > > 1> [H|T] = [22,87,65,84,33]. > > [22,87,65,84,33] > > 2> H. > > 22 > > 3> T. > > "WAT!? > This is a legitimate complaint, but it confuses two things. > There is *STRING HANDLING*, which is fine, and > there is *LIST PRINTING*, which causes the confusion. > > For comparison, DEC-10 Prolog, PDP-11 Prolog, C-Prolog, and Quintus Prolog > all did STRING HANDLING as lists of character codes, but > all did LIST PRINTING without ever converting lists of numbers to strings. > The answer was that there was a library procedure to print a list of > integers as a string and you could call that whenever you wanted to, > such as in a user-defined pretty-printing procedure. Here's a transcript > from SICStus Prolog: > | ?- write([65,66,67]). > [65,66,67] > yes > | ?- write("ABC"). > [65,66,67] > yes > > The heuristic used by the debugger in some Prologs was that a list of > integers between 32 and 126 inclusive was printed as a string; that > broke down with Latin 1, and broke harder with Unicode. The simple > behaviour mandated by the standard that lists of integers print as > lists of integers confuses people once, then they learn that string > quotes are an input notation, not an output notation, and if they want > string notation in output, they have to call a special procedure to get it. > > The ISO Prolog committee introduced a horrible alternative which the > DEC-10 Prolog designers had experienced in some Lisp systems and > learned to hate: flip a switch and "ABC" is read as ['A','B','C']. The > principal reason given for that was that the output was semi-readable. > One of my arguments against it was that this required every Prolog > system to be able to hold 17*2**16 atoms, and I new for a fact that > many would struggle to do so. The retort was "they must be changed > to make a special case for one-character atoms". Oh well, no silver > bullet. > > That does serve as a reminder, though, that using [a,b,c] instead of > [$a,$b,$c] is *possible* in Erlang. > > Just to repeat the basic point: the printing of (some) integer lists as > strings is SEPARABLE from the issue of how strings are represented and > processed; that could be changed without anything else in the language > changing. > > > > 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). > > Without tagging, merely parsing out a string as a list is not > > perfectly reversible. > Here you are making a demand that very few other programming languages > can support. For example, take JavaScript. "\u0041" is read as "A", > and you are not going to get "\u0041" back from "A". You're not even > going to get "\x41" back from it, even though "\x41" == "A". > > Or take Erlang, where > 1> 'foo bar'. > 'foo bar' > 2> 'foobar'. > foobar > with the same kind of thing happening in Prolog. > > And of COURSE reading [1 /* one */, 2 /* deux */, 4 /* kvar */] > in JavaScript preserves the comments so that re-encoding the > data structure restores the input perfectly. > > Or for that matter consider floating point numbers, where > even the languages that produce the best possible conversions > cannot promise that encode(decode(x)) == x. > > No, I'm sorry, this "perfectly reversible codec" requirement sets up > a standard that NO programming language I'm aware of satisfies. > It is, in fact, a straw man. What you *can* ask, and what some > language designers and implementers strive to give you, is > decode(encode(decode(x))) == decode(x). > > But to repeat the point made earlier, the way that lists of plausible > character codes is printed is SEPARABLE from the way strings are > represented and handled and in an ancestral language is SEPARATE. > > > > 4) What is the right way to implement the function is_string(List) > > correctly? > > > > *ducks* > > That really is a "have you stopped beating your wife, answer yes or no" > sort of question. > > It depends on the semantics you *want* it to have. The Quintus > library didn't provide any such predicate, but it did provide > > plausible_chars(Term) > when Term is a sequence of integers satisfying > is_graphic(C) or is_space(C), > possibly ending with a tail that is a variable or > a variable bound by numbervars/3. > > Notice the careful choice of name: not IS (certainly) a string, > but is a PLAUSIBLE list of characters. > > It was good enough for paying customers to be happy with the > module it was part of (which was the one offering the > non-usual portray_chars(Term) command). > > One of the representations Quintus used for strings (again, a > library feature, not a core language feature) was in Erlang > notation {external_string,FileName,Offset,Length}, and idea > that struck the customer I developed it for as a great > innovation, when I'd simply stolen it from Smalltalk! > > The thing is that STRINGS ARE WRONG for most things, > however represented. For example, when Java changed > the representation of String so that slicing became a > costly operation, I laughed, because I had my own representation > of strings that provided O(1) concatenation as well as cheap > slicing. (Think Erlang iolists and you won't be far wrong.) > The Pop2 language developed and used at Edinburgh > represented file names as lists, e.g., [/dev/null] was in > Erlang notation ['/',dev.'/',null]. This made file name > manipulation easier than representing them as strings. > Any time there is internal structure, any time there is scope > for sharing substructure, any time you need to process > the parts of a string, strings are wrong. > > The PERL lesson is that regular expressions are a fantastic > tool for doing the wrong thing quite simply. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Fri Mar 18 18:15:22 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Fri, 18 Mar 2016 18:15:22 +0100 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> Message-ID: A superlative suggestion sir, with only two minor drawbacks: one, Erlang is dynamically typed language and two, Erlang is dynamically typed language. I know that technically that?s only one drawback, but I thought it was such a big one it was worth mentioning twice. Hynek On Fri, Mar 18, 2016 at 5:30 PM, Emil Holmstrom wrote: > I am probably repeating what someone else already have said in some other > similar thread. > > The confusion between strings and [integer()] would have been greatly > reduced if char() existed, $a wouldn't have to be syntactic sugar for 97 > but would actually be "character a". You would have to explicitly convert > char() -> integer() and wise versa. This is how strings are implemented in > ML and Haskell. > > Regarding character encoding: inside Erlang Unicode could always be > assumed, converson between different character encodings could be done on > I/O. > > /emil > > On Fri, 18 Mar 2016 at 00:51, Richard A. O'Keefe > wrote: > >> >> >> On 17/03/16 11:53 pm, Steve Davis wrote: >> > > ROK said: >> > > Yawn. >> > (What am I doing trying to argue with ROK??? Am I MAD?) >> > >> > 1) Why is it people rant about "string handling" in Erlang? >> >> Because it is not the same as Java. >> > >> > 2) Principle of least surprise: >> > 1> [H|T] = [22,87,65,84,33]. >> > [22,87,65,84,33] >> > 2> H. >> > 22 >> > 3> T. >> > "WAT!? >> This is a legitimate complaint, but it confuses two things. >> There is *STRING HANDLING*, which is fine, and >> there is *LIST PRINTING*, which causes the confusion. >> >> For comparison, DEC-10 Prolog, PDP-11 Prolog, C-Prolog, and Quintus Prolog >> all did STRING HANDLING as lists of character codes, but >> all did LIST PRINTING without ever converting lists of numbers to strings. >> The answer was that there was a library procedure to print a list of >> integers as a string and you could call that whenever you wanted to, >> such as in a user-defined pretty-printing procedure. Here's a transcript >> from SICStus Prolog: >> | ?- write([65,66,67]). >> [65,66,67] >> yes >> | ?- write("ABC"). >> [65,66,67] >> yes >> >> The heuristic used by the debugger in some Prologs was that a list of >> integers between 32 and 126 inclusive was printed as a string; that >> broke down with Latin 1, and broke harder with Unicode. The simple >> behaviour mandated by the standard that lists of integers print as >> lists of integers confuses people once, then they learn that string >> quotes are an input notation, not an output notation, and if they want >> string notation in output, they have to call a special procedure to get >> it. >> >> The ISO Prolog committee introduced a horrible alternative which the >> DEC-10 Prolog designers had experienced in some Lisp systems and >> learned to hate: flip a switch and "ABC" is read as ['A','B','C']. The >> principal reason given for that was that the output was semi-readable. >> One of my arguments against it was that this required every Prolog >> system to be able to hold 17*2**16 atoms, and I new for a fact that >> many would struggle to do so. The retort was "they must be changed >> to make a special case for one-character atoms". Oh well, no silver >> bullet. >> >> That does serve as a reminder, though, that using [a,b,c] instead of >> [$a,$b,$c] is *possible* in Erlang. >> >> Just to repeat the basic point: the printing of (some) integer lists as >> strings is SEPARABLE from the issue of how strings are represented and >> processed; that could be changed without anything else in the language >> changing. >> > >> > 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). >> > Without tagging, merely parsing out a string as a list is not >> > perfectly reversible. >> Here you are making a demand that very few other programming languages >> can support. For example, take JavaScript. "\u0041" is read as "A", >> and you are not going to get "\u0041" back from "A". You're not even >> going to get "\x41" back from it, even though "\x41" == "A". >> >> Or take Erlang, where >> 1> 'foo bar'. >> 'foo bar' >> 2> 'foobar'. >> foobar >> with the same kind of thing happening in Prolog. >> >> And of COURSE reading [1 /* one */, 2 /* deux */, 4 /* kvar */] >> in JavaScript preserves the comments so that re-encoding the >> data structure restores the input perfectly. >> >> Or for that matter consider floating point numbers, where >> even the languages that produce the best possible conversions >> cannot promise that encode(decode(x)) == x. >> >> No, I'm sorry, this "perfectly reversible codec" requirement sets up >> a standard that NO programming language I'm aware of satisfies. >> It is, in fact, a straw man. What you *can* ask, and what some >> language designers and implementers strive to give you, is >> decode(encode(decode(x))) == decode(x). >> >> But to repeat the point made earlier, the way that lists of plausible >> character codes is printed is SEPARABLE from the way strings are >> represented and handled and in an ancestral language is SEPARATE. >> > >> > 4) What is the right way to implement the function is_string(List) >> > correctly? >> > >> > *ducks* >> >> That really is a "have you stopped beating your wife, answer yes or no" >> sort of question. >> >> It depends on the semantics you *want* it to have. The Quintus >> library didn't provide any such predicate, but it did provide >> >> plausible_chars(Term) >> when Term is a sequence of integers satisfying >> is_graphic(C) or is_space(C), >> possibly ending with a tail that is a variable or >> a variable bound by numbervars/3. >> >> Notice the careful choice of name: not IS (certainly) a string, >> but is a PLAUSIBLE list of characters. >> >> It was good enough for paying customers to be happy with the >> module it was part of (which was the one offering the >> non-usual portray_chars(Term) command). >> >> One of the representations Quintus used for strings (again, a >> library feature, not a core language feature) was in Erlang >> notation {external_string,FileName,Offset,Length}, and idea >> that struck the customer I developed it for as a great >> innovation, when I'd simply stolen it from Smalltalk! >> >> The thing is that STRINGS ARE WRONG for most things, >> however represented. For example, when Java changed >> the representation of String so that slicing became a >> costly operation, I laughed, because I had my own representation >> of strings that provided O(1) concatenation as well as cheap >> slicing. (Think Erlang iolists and you won't be far wrong.) >> The Pop2 language developed and used at Edinburgh >> represented file names as lists, e.g., [/dev/null] was in >> Erlang notation ['/',dev.'/',null]. This made file name >> manipulation easier than representing them as strings. >> Any time there is internal structure, any time there is scope >> for sharing substructure, any time you need to process >> the parts of a string, strings are wrong. >> >> The PERL lesson is that regular expressions are a fantastic >> tool for doing the wrong thing quite simply. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spanemangalore@REDACTED Fri Mar 18 19:22:43 2016 From: spanemangalore@REDACTED (Sachin Panemangalore) Date: Fri, 18 Mar 2016 11:22:43 -0700 Subject: [erlang-questions] Dialyzer type specifications for macros with bitstrings makes erlc barf In-Reply-To: <5860244.2BDD7qnhcW@changa> References: <5860244.2BDD7qnhcW@changa> Message-ID: Thanks a lot Craig Amazing , detailed information . However I do feel there is an inconsistency in the behavior of erlang compiler vs dialyzer. Inconsistencies ============= 1. Although a type-spec doesn't allow you to specify a "set" of objects, it lets us specify a set of integers e.g 0..12 etc. 2. You can specify definitions as bitstring() or string() instead of a set of values, but dialyzer complains that the return type is a super-type of the set of values being returned . Erlang has a powerful pattern matcher, hence potentially we can afford to return sets of arbitrary unique objects from functions , hence we do need the ability to specify a unique set of any data-type , e.g strings, bitstrings, tuples . just my opinion, thanks again for the help!. Regards sachin On Thu, Mar 17, 2016 at 10:52 PM, zxq9 wrote: > On 2016?3?17? ??? 21:53:06 Sachin Panemangalore wrote: > > 1. The following compiles fine in Erlang. > > > > -define(MY_INT_MACRO1, 1). > > -define(MY_INT_MACRO2, 2). > > -type my_type() :: ?MY_INT_MACRO1 | ?MY_INT_MACRO2 . > > > > > > 2. The following doesn't > > > > -define(MY_BITSTRING_MACRO1, <<"01">>). > > -define(MY_BITSTRING_MACRO2, <<"02">>). > > -type my_type() :: ?MY_BITSTRING_MACRO1 | ?MY_BITSTRING_ MACRO2 . > > > > Why ? and is there a way to make this work ? > > (Changed the names above to say what I think you meant.) > > Type declarations have a few limitations with regard to defining what is > inside. You can declare the size of a binary, but not what it contains, > hence these sort of type definitions: > https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl#L47-L56 > > In the same way, you can declare a dictionary or map's k/v types: > > -type my_dict() :: dict:dict(atom(), integer()). > -type my_map() :: maps:map(atom(), string()). > > You'll never be able to provide a complete, fixed shape of a dict or > map, though, because that would actually be a schema, and that's what > using tuple or records as types already does. This only turns out to > be the same limitation as we have defining lists of things: > > -type my_tuple_list() :: [{atom(), some_type()}]. > > This is why the following is not legal: > > -type some_phrase() :: "I am a string literal.". > > Actually, I think you can't even declare a list of some specific length, > which leaves binary type definitions slightly more rigorous than lists. But > if you *knew* the length of a list why wouldn't you be using a tuple? > > It would be pretty convenient to be able to define types based on binary > literals, since their use case differs from binaries. But on the other > hand, > if you want binary strings defined this way you usually actually want atoms > that you can derive from some external binary data you receive -- and atoms > are (usually) an excellent way to clean up your type defs in > self-documenting ways. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Fri Mar 18 21:20:39 2016 From: andre@REDACTED (Andre Graf) Date: Fri, 18 Mar 2016 21:20:39 +0100 Subject: [erlang-questions] [ANN] VerneMQ 0.12.5 released Message-ID: <56EC6317.9040505@dergraf.org> Hello everyone, I am happy to announce the new VerneMQ 0.12.5 release today! For those of you who haven't heard of VerneMQ [1], VerneMQ is a scalable MQTT [2] broker implemented in Erlang, licensed under the Apache 2 license. Please feel free to check it out [3] or get in touch over email if this sounds interesting. The new release includes several bug fixes as well as (finally) Erlang 18 support. Moreover we started to use hex.pm packages for the 3d-party dependencies where an up-to-date package was available. We plan to further leverage the Rebar3/hex.pm infrastructure in the future. Thank a lot to the community members, especially the folks behind Rebar3, for the kind support. Best, Andre [1] https://verne.mq [2] https://en.wikipedia.org/wiki/MQTT [2] https://github.com/erlio/vernemq From roberto.ostinelli@REDACTED Fri Mar 18 23:40:40 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 18 Mar 2016 23:40:40 +0100 Subject: [erlang-questions] rebar3 dependencies Message-ID: Dear list, Does someone know if there's a way to vendor dependencies with rebar3, just like you do with ruby's bundler? I always have the habit to commit dependencies too, and I've done so with rebar2 quite easily by committing the /deps folder too. However to my understanding with rebar3 all dependencies go into _build, which isn't supposed to be committed. Any input welcome, I'm new to rebar3, am considering the switch but would like to tackle this first. Thanks, r. From t@REDACTED Sat Mar 19 00:22:07 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 18:22:07 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: Message-ID: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> There is no built-in support for vendoring. What is the use case? Is it for developers to be able to checkout a single repo and begin working on the project without anything additional needing to be fetched? -- Tristan Sloughter t@REDACTED On Fri, Mar 18, 2016, at 05:40 PM, Roberto Ostinelli wrote: > Dear list, > Does someone know if there's a way to vendor dependencies with rebar3, > just like you do with ruby's bundler? > > I always have the habit to commit dependencies too, and I've done so with > rebar2 quite easily by committing the /deps folder too. However to my > understanding with rebar3 all dependencies go into _build, which isn't > supposed to be committed. > > Any input welcome, I'm new to rebar3, am considering the switch but would > like to tackle this first. > > Thanks, > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Sat Mar 19 01:26:09 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Mar 2016 17:26:09 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> Message-ID: <56EC9CA1.80901@gmail.com> On 03/18/2016 04:22 PM, Tristan Sloughter wrote: > There is no built-in support for vendoring. > > What is the use case? Is it for developers to be able to checkout a > single repo and begin working on the project without anything additional > needing to be fetched? > This use-case of rebar is an important one due to Erlang's focus on fault-tolerance. To make sure your built is robust and stable, it requires having all your dependencies at a known state. I understand the lock file is a method to rely on remote repositories at a specific commit and the commit should change if the repositories history is rewritten. However, repositories can disappear or be inaccessible (in a deployment environment), and it is more dependable to have the dependency stored along with the dependent source code (to avoid a reliance on the remote repository, due to its potential to fail in unexpected ways). For pursuing fault-tolerance (seriously) this should be a way of using rebar that is required functionality. I have setup rebar2 to do this in https://github.com/CloudI/CloudI but this is one area that prevents me from adopting rebar3 unless this is easily done with using |'||{project_app_dirs, ["external", "lib"]}' instead of '||lib_dirs' ('||lib_dirs' works with rebar2), or some other method. | -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 01:32:47 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 19:32:47 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56EC9CA1.80901@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> Message-ID: <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> > This use-case of rebar is an important one due to Erlang's focus on > fault-tolerance.? To make sure your built is robust and stable, it requires having all your dependencies at a known state. Your build artifact for deployment is a release which does "bundle" all dependencies at a certain point in time. For reproducibility in the event of losing the original release is, as you mention, what the lock files are for. When hex gains easy mirroring there will also be a quick way to ensure you have your own backups of alls your dependencies. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Mar 19 01:53:52 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Mar 2016 17:53:52 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> Message-ID: <56ECA320.8050601@gmail.com> On 03/18/2016 05:32 PM, Tristan Sloughter wrote: > > This use-case of rebar is an important one due to Erlang's focus on fault-tolerance. To make sure your built is robust and stable, it requires having all your dependencies at a known state. > > Your build artifact for deployment is a release which does "bundle" all dependencies at a certain point in time. > For reproducibility in the event of losing the original release is, as you mention, what the lock files are for. When hex gains easy mirroring there will also be a quick way to ensure you have your own backups of alls your dependencies. Yes, what I am talking about is having all the source code dependencies at a known state. hex attempts to do this, but as you mentioned, is unable to be hosted privately and the public hex site may go down at anytime for any length of time. You appear to be avoiding the topic of putting all the source code dependencies into a single repository and having them work with rebar3. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 01:58:53 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 19:58:53 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56ECA320.8050601@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> Message-ID: <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> > You appear to be avoiding the topic of putting all the source > code dependencies into a single repository and having them work > with rebar3. I think it is unnecessary and poor form. Especially if you application is public and others want to depend on it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 02:00:38 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 20:00:38 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> Message-ID: <1458349238.2179672.553552258.3C081A80@webmail.messagingengine.com> That said, there are custom resources and plugins which make it possible to create a sort of "bundler" I'm sure http://www.rebar3.org/docs/custom-dep-resources -- Tristan Sloughter t@REDACTED On Fri, Mar 18, 2016, at 07:58 PM, Tristan Sloughter wrote: >> You appear to be avoiding the topic of putting all the source code >> dependencies into a single repository and having them work with >> rebar3. > > I think it is unnecessary and poor form. Especially if you application > is public and others want to depend on it. > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Mar 19 02:37:48 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Mar 2016 18:37:48 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> Message-ID: <56ECAD6C.7040905@gmail.com> On 03/18/2016 05:58 PM, Tristan Sloughter wrote: > > You appear to be avoiding the topic of putting all the source code dependencies into a single repository and having them work with rebar3. > > I think it is unnecessary and poor form. Especially if you application is public and others want to depend on it. While your opinion is that the approach is "bad", I see it as required for stable and repeatable builds. So we reach an impasse where it appears that rebar3 is not meant to support this, unless by chance without it intentionally being a use-case, which is unfortunate. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sat Mar 19 03:05:24 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 19 Mar 2016 11:05:24 +0900 Subject: [erlang-questions] Dialyzer type specifications for macros with bitstrings makes erlc barf In-Reply-To: References: <5860244.2BDD7qnhcW@changa> Message-ID: <6259401.7zDxaInSDF@changa> Hi, Sachin. On 2016?3?18? ??? 11:22:43 you wrote: > Thanks a lot Craig > Amazing , detailed information . However I do feel there is an > inconsistency in the behavior of erlang compiler vs dialyzer. > Inconsistencies > ============= > 1. Although a type-spec doesn't allow you to specify a "set" of objects, > it lets us specify a set of integers e.g 0..12 etc. But this isn't really a set definition, it is only specifying a *range*. -type zero_through_five() :: 0..5. The above declaration is effectively shorthand for: -type zero_through_five() :: 0 | 1 | 2 | 3 | 4 | 5. And that, when viewed as a union, is not so different from: -type odd_union() :: zero_through_five() | blah | tuple(). As for declaring an actual set... I suppose you can do this: -type odd_set() :: set:set(odd_union()). But I *think* this is declaring that the set or any subset of odd_union() members, not a fixed set that is all members of odd_union(). In any case, it will be an actual set (no duplicates) where the union types themselves are just union types and have nothing to say about how many of each member type is involved. Neither, for that matter, do list declarations: -type odd_list() :: [odd_union()]. I don't think I've actually had any call for a set declaration like this. But I'm sure there is some case out there where it would be useful. Anyway, Erlang's type system and Dialyzer are permissive in nature, only telling you when something is definitely wrong, instead of a stricter Haskell sort of type system where anything that isn't clearly correct defaults to being wrong. Just part of the tradeoff of this particular environment. > 2. You can specify definitions as bitstring() or string() instead of a set > of values, but dialyzer complains that the return type > is a super-type of the set of values being returned . By "set of values" do you mean "union of types"? I'm not quite sure I'm following you, as I don't think I've ever seen this warning or error. Can you show me an example? Some warnings/errors are a bit cryptic, so I want to know what this one is before I run into it in my own code. (Its only a matter of time...) From eric.meadows.jonsson@REDACTED Sat Mar 19 03:03:23 2016 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Sat, 19 Mar 2016 03:03:23 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56ECAD6C.7040905@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> Message-ID: > > Yes, what I am talking about is having all the source code dependencies at > a known state. > hex attempts to do this, but as you mentioned, is unable to be hosted > privately and the public > hex site may go down at anytime for any length of time. You can host Hex privately, there is nothing prohibiting you from hosting your own Hex repo or mirroring Hex.pm. On Sat, Mar 19, 2016 at 2:37 AM, Michael Truog wrote: > On 03/18/2016 05:58 PM, Tristan Sloughter wrote: > > You appear to be avoiding the topic of putting all the source code > dependencies into a single repository and having them work with rebar3. > > > I think it is unnecessary and poor form. Especially if you application is > public and others want to depend on it. > > > While your opinion is that the approach is "bad", I see it as required for > stable and repeatable builds. So we reach an impasse where it appears that > rebar3 is not meant to support this, unless by chance without it > intentionally being a use-case, which is unfortunate. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Mar 19 03:35:49 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Mar 2016 19:35:49 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> Message-ID: <56ECBB05.8010702@gmail.com> On 03/18/2016 07:03 PM, Eric Meadows-J?nsson wrote: > > Yes, what I am talking about is having all the source code dependencies at a known state. > hex attempts to do this, but as you mentioned, is unable to be hosted privately and the public > hex site may go down at anytime for any length of time. > > > You can host Hex privately, there is nothing prohibiting you from hosting your own Hex repo or mirroring Hex.pm. While this may be possible, it is undocumented functionality, so it doesn't exist in a useable way. The same was true of sinan/faxien in the past. > > On Sat, Mar 19, 2016 at 2:37 AM, Michael Truog > wrote: > > On 03/18/2016 05:58 PM, Tristan Sloughter wrote: >> >> You appear to be avoiding the topic of putting all the source code dependencies into a single repository and having them work with rebar3. >> >> I think it is unnecessary and poor form. Especially if you application is public and others want to depend on it. > > While your opinion is that the approach is "bad", I see it as required for > stable and repeatable builds. So we reach an impasse where it appears that > rebar3 is not meant to support this, unless by chance without it > intentionally being a use-case, which is unfortunate. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Eric Meadows-J?nsson > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Mar 19 03:43:33 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 18 Mar 2016 22:43:33 -0400 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56ECAD6C.7040905@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> Message-ID: <20160319024332.GH53561@ferdmbp.local> On 03/18, Michael Truog wrote: > >While your opinion is that the approach is "bad", I see it as required for >stable and repeatable builds. So we reach an impasse where it appears that >rebar3 is not meant to support this, unless by chance without it >intentionally being a use-case, which is unfortunate. Just so I'm clear, the feature you're after here is that rebar3 should be able to fetch your dependencies and build them, but you just want them to be fetched and built in a different location so that you do not have to go fetch them yourself? Because you could just go and place them yourself in libs/ (and put your own in apps/, or else use {project_app_dirs, ["deps/*", "apps/*", "lib/*", "."]} to have a deps/ directory) and then check this in. This would split your code from where they are built, and would tell rebar3 "don't handle these, I'll do it myself", which I assume is kind of the point here, isn't it? So the only thing missing there is fetching and upgrading, although is this still a thing you want since you don't necessarily control the order it is done in? Is this the thing not being handled that would be critical? From t@REDACTED Sat Mar 19 04:00:19 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 22:00:19 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <20160319024332.GH53561@ferdmbp.local> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <20160319024332.GH53561@ferdmbp.local> Message-ID: <1458356419.2207860.553599450.7FB18A6B@webmail.messagingengine.com> Putting them under project_app_dirs is a bad idea. It'll mean running tests for the project will run all those tests, breaks the ability to use it as a dependency of another apps, you can't have the same app in the rebar.config and expect it to know the project_app of the same name is the actual dep, etc... To correctly do this a plugin should be created. -- Tristan Sloughter t@REDACTED On Fri, Mar 18, 2016, at 09:43 PM, Fred Hebert wrote: > On 03/18, Michael Truog wrote: > > > >While your opinion is that the approach is "bad", I see it as required for > >stable and repeatable builds. So we reach an impasse where it appears that > >rebar3 is not meant to support this, unless by chance without it > >intentionally being a use-case, which is unfortunate. > > Just so I'm clear, the feature you're after here is that rebar3 should > be able to fetch your dependencies and build them, but you just want > them to be fetched and built in a different location so that you do not > have to go fetch them yourself? > > Because you could just go and place them yourself in libs/ (and put your > own in apps/, or else use {project_app_dirs, ["deps/*", "apps/*", > "lib/*", "."]} to have a deps/ directory) and then check this in. > > This would split your code from where they are built, and would tell > rebar3 "don't handle these, I'll do it myself", which I assume is kind > of the point here, isn't it? > > So the only thing missing there is fetching and upgrading, although is > this still a thing you want since you don't necessarily control the > order it is done in? Is this the thing not being handled that would be > critical? From mononcqc@REDACTED Sat Mar 19 04:14:11 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 18 Mar 2016 23:14:11 -0400 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <07AFCF93-4A55-4384-9900-D457DEF1B4D8@spam.mx.lisp.geek.nz> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <20160319024332.GH53561@ferdmbp.local> <1458356419.2207860.553599450.7FB18A6B@webmail.messagingengine.com> <07AFCF93-4A55-4384-9900-D457DEF1B4D8@spam.mx.lisp.geek.nz> Message-ID: <20160319031410.GI53561@ferdmbp.local> On 03/18, Geoff Cant wrote: >Wouldn't checkouts work for this case? > Yes and no. _checkouts work fine for local development, but it still carries that pesky ebin/. Tristan and I are discussing plugins for this. You could really have `rebar3 vendor store' which copies from `_build//lib' to `deps/' (minus the ebins), and `rebar3 vendor apply' which moves them back to `_build//lib'. The plugin would only need to copy/move files around, and could still use the rest of all rebar3's providers, without having us maintain it in core. That way you can experiment with upgrades, blowing stuff up, crushing it, and so on, and still vendor your stuff in. There's simple solutions for it, and we're hoping people can use the plugin interface for that. From t@REDACTED Sat Mar 19 04:35:13 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 18 Mar 2016 22:35:13 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <20160319031410.GI53561@ferdmbp.local> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <20160319024332.GH53561@ferdmbp.local> <1458356419.2207860.553599450.7FB18A6B@webmail.messagingengine.com> <07AFCF93-4A55-4384-9900-D457DEF1B4D8@spam.mx.lisp.geek.nz> <20160319031410.GI53561@ferdmbp.local> Message-ID: <1458358513.2221718.553613898.683690E1@webmail.messagingengine.com> Yea, don't use _checkouts, that should be for local development and removes the checkout app from the lock file. People can try this https://github.com/tsloughter/rebar3_vendor It has issues if you have git deps... but I think it is a start for those who need this. Hopefully those who need it can improve it from here :) -- Tristan Sloughter t@REDACTED On Fri, Mar 18, 2016, at 10:14 PM, Fred Hebert wrote: > On 03/18, Geoff Cant wrote: > >Wouldn't checkouts work for this case? > > > > Yes and no. _checkouts work fine for local development, but it still > carries that pesky ebin/. > > Tristan and I are discussing plugins for this. You could really have > `rebar3 vendor store' which copies from `_build//lib' to > `deps/' (minus the ebins), and `rebar3 vendor apply' which moves them > back to `_build//lib'. > > The plugin would only need to copy/move files around, and could still > use the rest of all rebar3's providers, without having us maintain it in > core. > > That way you can experiment with upgrades, blowing stuff up, crushing > it, and so on, and still vendor your stuff in. > > There's simple solutions for it, and we're hoping people can use the > plugin interface for that. From mjtruog@REDACTED Sat Mar 19 05:28:28 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Mar 2016 21:28:28 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458358513.2221718.553613898.683690E1@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <20160319024332.GH53561@ferdmbp.local> <1458356419.2207860.553599450.7FB18A6B@webmail.messagingengine.com> <07AFCF93-4A55-4384-9900-D457DEF1B4D8@spam.mx.lisp.geek.nz> <20160319031410.GI53561@ferdmbp.local> <1458358513.2221718.553613898.683690E1@webmail.messagingengine.com> Message-ID: <56ECD56C.7050701@gmail.com> On 03/18/2016 08:35 PM, Tristan Sloughter wrote: > Yea, don't use _checkouts, that should be for local development and > removes the checkout app from the lock file. > > People can try this https://github.com/tsloughter/rebar3_vendor > > It has issues if you have git deps... but I think it is a start for > those who need this. Hopefully those who need it can improve it from > here :) > Thank you for providing the start of a plugin for this. I will use this when doing a conversion to rebar3 and may have changes if any are required. Thanks, Michael From michael.eugene.turner@REDACTED Sat Mar 19 06:28:25 2016 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 19 Mar 2016 14:28:25 +0900 Subject: [erlang-questions] Erlang in education --- back from the dead In-Reply-To: References: Message-ID: Some years back, I mentioned that Erlang seemed a lot closer to what Alan Kay originally meant by "object oriented" (which at the time did not include inheritance) than most O-O languages since. He has long been occupied with the issues of education through writing software, even in the early grades. His foundation's software work seems a bit moribund (I guess Open Cobalt is what it has come down to, not a wildly active project) but what they've done over the years might provide some ideas. Regards, Michael Turner Executive Director Project Persephone K-1 bldg 3F 7-2-6 Nishishinjuku Shinjuku-ku Tokyo 160-0023 Tel: +81 (3) 6890-1140 Fax: +81 (3) 6890-1158 Mobile: +81 (90) 5203-8682 turner@REDACTED http://www.projectpersephone.org/ "Love does not consist in gazing at each other, but in looking outward together in the same direction." -- Antoine de Saint-Exup?ry On Thu, Mar 17, 2016 at 11:48 AM, Lloyd R. Prentice wrote: > Hello, > > A year or so ago there was on this list a wonderful spirited discussion of > how to get kids and educators interested in Erlang. > > We have now low-cost low-power 64-bit hardware platforms that open > boundless opportunities: > > https://www.raspberrypi.org/blog/raspberry-pi-3-on-sale/ > http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438 > https://www.pine64.com/product > > > http://www.techspot.com/news/64113-western-digital-creates-314gb-hard-drive-specifically-raspberry.html > > So, I'm wondering if the Erlang community, particularly our corporate > members, would be interested in mounting a programming challenge to > students of all ages to develop innovative applications on one or another > of these platforms? All entries would be released as open-source. > Incentives might range from hardware to cash to internships to jobs. I, in > my Writers Glen incarnation, would be willing to seed the program with $500 > and devote some time if we can secure enough interest and resources to > mount a credible program. > > Or, short of such a grandiose scheme, I'm wondering if there are folks in > the community interested in cooperating off-list in exploring and > publishing on-line or in print under Creative Commons license how-tos and > tutorials designed to inspire kids and push these platforms to their limits. > > Best wishes, > > LRP > > > > > > > > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alin.popa@REDACTED Sat Mar 19 07:43:11 2016 From: alin.popa@REDACTED (Alin Popa) Date: Sat, 19 Mar 2016 06:43:11 +0000 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> Message-ID: <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> On 19 Mar 2016, at 02:03, Eric Meadows-J?nsson wrote: >> Yes, what I am talking about is having all the source code dependencies at a known state. >> hex attempts to do this, but as you mentioned, is unable to be hosted privately and the public >> hex site may go down at anytime for any length of time. > > You can host Hex privately, there is nothing prohibiting you from hosting your own Hex repo or mirroring Hex.pm. > Could you please give us more details on how to do this, or even some unofficial/yet to be released documentation? Thanks, Alin >> On Sat, Mar 19, 2016 at 2:37 AM, Michael Truog wrote: >>> On 03/18/2016 05:58 PM, Tristan Sloughter wrote: >>> You appear to be avoiding the topic of putting all the source code dependencies into a single repository and having them work with rebar3. >>> >>> I think it is unnecessary and poor form. Especially if you application is public and others want to depend on it. >> >> While your opinion is that the approach is "bad", I see it as required for >> stable and repeatable builds. So we reach an impasse where it appears that >> rebar3 is not meant to support this, unless by chance without it >> intentionally being a use-case, which is unfortunate. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Eric Meadows-J?nsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sat Mar 19 08:30:44 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 19 Mar 2016 08:30:44 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: Tristan, Fred, Eric First of all, let me very clear on this: I want to thank you for the work you're all doing. It is a tremendous amount of work, and I can only be grateful that you are willing to share it with me and the rest of the community. I know how hard it can be working in open source and receiving a variety of random requests, sometimes coming from people that "expect" things to get done by you. So: thank you. Now, to get back to my original question: Tristan, you say that releases "bundle all dependencies at a certain point in time", and that wanting to vendor dependencies is "unnecessary and poor form". I would like for you to consider that there are real cases for which vendoring is necessary, and not poor form. Let me give you some examples. 1. Even in the Ruby community, gems disappear - whatever the reason. It has happened before, it will happen again. Hex.pm, being smaller and way more recent, is also probably (at least now) less reliable than rubygems, and in general relying on github repositories for those libraries not yet packaged is even worse. It is therefore understandable that some may feel better knowing that they just have to rely on their own repository, where all dependencies have been vendored. 2. Releases "bundle" all dependencies at a certain point in time, as you say, but only with whatever exists at that time. I'd like to be able to release the same code on newer Erlang releases, sometimes years after my first release. Or with different operating systems. See point 1: I need to ensure to have all the dependencies in my code, whatever happens to the rest of the world. 3. I often compile releases on private clouds, which do not have access to internet. Yes, there are workarounds, but the point her is that it makes my life easier not to have to find one. Rebase 2 currently satisfies these requirements, and since I'd like to move to Rebase3 I'd like to find working alternatives. Eric, sure, I can host Hex privately. Though, you'll probably agree that it is way more easier to just go with the vendoring thing. :) That being said, I'd like for you, Tristan, Fred, Eric, to try and welcome my little feedback here, and take it for what it is. I'd like it to be easy and possible to discuss these ideas serenely, because relying on a system that considers (or better yet, values) the community input such as mine would make me sleep better at night :) All the best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sat Mar 19 09:16:53 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 19 Mar 2016 09:16:53 +0100 Subject: [erlang-questions] rebar3 & hex: how to install? Message-ID: I'm on OSX. I've Erlang and Elixir installed. rebar3 is in /usr/local/bin. However: $ rebar3 hex user register ===> Command hex not found But: $ mix local mix hex # Prints Hex help information mix hex.build # Builds a new package version locally mix hex.config # Reads or updates Hex config mix hex.docs # Publishes docs for package mix hex.info # Prints Hex information mix hex.key # Hex API key tasks mix hex.outdated # Shows outdated Hex deps for the current project mix hex.owner # Hex package ownership tasks mix hex.public_keys # Manages Hex public keys mix hex.publish # Publishes a new package version mix hex.registry # Hex registry tasks mix hex.search # Searches for package names mix hex.user # Hex user tasks What am I missing? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Sat Mar 19 09:46:23 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sat, 19 Mar 2016 16:46:23 +0800 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? Message-ID: Hi, I have been using the esl-erlang pacakge from https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for Trusty. I have these in my bashrc: export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in the cloud) export ERL_EPMD_PORT="5678" Everything worked fine with the 18.2 package. After upgrading to esl-erlang 18.3, epmd stopped to work: > epmd -debug epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot assign requested address There is a workaround: if I remove the ERL_EPMD_ADDRESS environment variable, epms works: tcp 0 0 0.0.0.0:5678 0.0.0.0:* LISTEN 3328/epmd tcp6 0 0 :::5678 :::* LISTEN 3328/epmd After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. Thanks Khitai From bchesneau@REDACTED Sat Mar 19 09:57:04 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 19 Mar 2016 08:57:04 +0000 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: On Sat, Mar 19, 2016 at 8:30 AM Roberto Ostinelli wrote: > Tristan, Fred, Eric > First of all, let me very clear on this: I want to thank you for the work > you're all doing. It is a tremendous amount of work, and I can only be > grateful that you are willing to share it with me and the rest of the > community. I know how hard it can be working in open source and receiving a > variety of random requests, sometimes coming from people that "expect" > things to get done by you. So: thank you. > > Now, to get back to my original question: Tristan, you say that releases > "bundle all dependencies at a certain point in time", and that wanting to > vendor dependencies is "unnecessary and poor form". I would like for you to > consider that there are real cases for which vendoring is necessary, and > not poor form. Let me give you some examples. > > 1. Even in the Ruby community, gems disappear - whatever the reason. It > has happened before, it will happen again. Hex.pm, being smaller and way > more recent, is also probably (at least now) less reliable than rubygems, > and in general relying on github repositories for those libraries not yet > packaged is even worse. It is therefore understandable that some may feel > better knowing that they just have to rely on their own repository, where > all dependencies have been vendored. > > 2. Releases "bundle" all dependencies at a certain point in time, as you > say, but only with whatever exists at that time. I'd like to be able to > release the same code on newer Erlang releases, sometimes years after my > first release. Or with different operating systems. See point 1: I need to > ensure to have all the dependencies in my code, whatever happens to the > rest of the world. > > 3. I often compile releases on private clouds, which do not have access to > internet. Yes, there are workarounds, but the point her is that it makes my > life easier not to have to find one. > > Rebase 2 currently satisfies these requirements, and since I'd like to > move to Rebase3 I'd like to find working alternatives. > > Eric, sure, I can host Hex privately. Though, you'll probably agree that > it is way more easier to just go with the vendoring thing. :) > > > That being said, I'd like for you, Tristan, Fred, Eric, to try and welcome > my little feedback here, and take it for what it is. I'd like it to be easy > and possible to discuss these ideas serenely, because relying on a system > that considers (or better yet, values) the community input such as mine > would make me sleep better at night :) > > All the best, > r. > > Just to add the discussion, "vendors" dependencies or at least a way to have them in the source archive is a good way to ship the source code to customers. Doing it remove any dependency to an external source that could disappear or change over the time. It provides a final source product to customers that can still be edited/modified in future without to worry much. (they just need to make sure to archive correctly the deliverables). I had a look at the vendor plugin and it's a good start. But I think the "rebar3 vendor apply" should be automatic when you launch a build. Says if you have a _vendor dir or something like it it would be used to deploy the deps inside. Then we could have the following commands: rebar3 vendor fetch rebar3 vendor upgrade What do you think about it? Can we right now hooks the way the dependency is retrieved? Or do we need to provide some kind of custom dep resource [1] that would proxy each deps? Seem like right now that _checkouts is an hack and something like it can't be done externally. Anyway any hint is appreciated. I also need to have such feature on a day basis :) - benoit [1] http://www.rebar3.org/docs/custom-dep-resources -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sat Mar 19 10:07:38 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 19 Mar 2016 10:07:38 +0100 Subject: [erlang-questions] rebar3 and CT's main node Message-ID: All, When I manually run CT with: ct_run -dir test -pa ebin and print out the node the tests are running on with a simple: ct:pal("node = ~p", [node()]). I get something similar to: 'ct@REDACTED' However, when I run CT with rebar3 by issuing: rebar3 ct Then the same command prints: 'nonode@REDACTED' This is ok for most tests, but since I'm testing distributed nodes with ct_slave [1] this impedes a slave to be started, as the main node is a 'nonode@REDACTED' and hence the slave node cannot connect to this 'nonode'. If anyone wants to try this just run Syn's suite, it will fail when using rebar3 here: https://github.com/ostinelli/syn/blob/1.1.0/test/syn_test_suite_helper.erl#L62 With a {{badmatch,{error,not_alive,nonode@REDACTED}}. Any ideas? Thank you, r. [1] http://erlang.org/doc/man/ct_slave.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sat Mar 19 10:36:24 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 19 Mar 2016 10:36:24 +0100 Subject: [erlang-questions] rebar3 & hex: how to install? In-Reply-To: References: Message-ID: For anyone wandering the same thing: just add to your global rebar3 config in ~/.config/rebar3/rebar.config: {plugins, [rebar3_hex]}. This is documented here: https://hex.pm/docs/rebar3_publish Best, r. On Sat, Mar 19, 2016 at 9:16 AM, Roberto Ostinelli wrote: > I'm on OSX. I've Erlang and Elixir installed. > rebar3 is in /usr/local/bin. > > However: > > $ rebar3 hex user register > ===> Command hex not found > > But: > > $ mix local > mix hex # Prints Hex help information > mix hex.build # Builds a new package version locally > mix hex.config # Reads or updates Hex config > mix hex.docs # Publishes docs for package > mix hex.info # Prints Hex information > mix hex.key # Hex API key tasks > mix hex.outdated # Shows outdated Hex deps for the current project > mix hex.owner # Hex package ownership tasks > mix hex.public_keys # Manages Hex public keys > mix hex.publish # Publishes a new package version > mix hex.registry # Hex registry tasks > mix hex.search # Searches for package names > mix hex.user # Hex user tasks > > What am I missing? > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.meadows.jonsson@REDACTED Sat Mar 19 11:32:50 2016 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Sat, 19 Mar 2016 11:32:50 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: > > While this may be possible, it is undocumented functionality, so it > doesn't exist in a useable way. The same was true of sinan/faxien in the past. But it's not undocumented. In fact everything that makes up Hex is specified and documented. Hex.pm is simply an implementation of those specifications. Could you please give us more details on how to do this, or even some > unofficial/yet to be released documentation? Here you have all the specifications for Hex https://github.com/hexpm/specifications. To run your own repository or hex.pm mirror you simply have to implement the repository endpoints https://github.com/hexpm/specifications/blob/master/endpoints.md#repository. If you find anything that is not documented it's a bug. On Sat, Mar 19, 2016 at 9:57 AM, Benoit Chesneau wrote: > > > On Sat, Mar 19, 2016 at 8:30 AM Roberto Ostinelli > wrote: > >> Tristan, Fred, Eric >> First of all, let me very clear on this: I want to thank you for the work >> you're all doing. It is a tremendous amount of work, and I can only be >> grateful that you are willing to share it with me and the rest of the >> community. I know how hard it can be working in open source and receiving a >> variety of random requests, sometimes coming from people that "expect" >> things to get done by you. So: thank you. >> >> Now, to get back to my original question: Tristan, you say that releases >> "bundle all dependencies at a certain point in time", and that wanting to >> vendor dependencies is "unnecessary and poor form". I would like for you to >> consider that there are real cases for which vendoring is necessary, and >> not poor form. Let me give you some examples. >> >> 1. Even in the Ruby community, gems disappear - whatever the reason. It >> has happened before, it will happen again. Hex.pm, being smaller and way >> more recent, is also probably (at least now) less reliable than rubygems, >> and in general relying on github repositories for those libraries not yet >> packaged is even worse. It is therefore understandable that some may feel >> better knowing that they just have to rely on their own repository, where >> all dependencies have been vendored. >> >> 2. Releases "bundle" all dependencies at a certain point in time, as you >> say, but only with whatever exists at that time. I'd like to be able to >> release the same code on newer Erlang releases, sometimes years after my >> first release. Or with different operating systems. See point 1: I need to >> ensure to have all the dependencies in my code, whatever happens to the >> rest of the world. >> >> 3. I often compile releases on private clouds, which do not have access >> to internet. Yes, there are workarounds, but the point her is that it makes >> my life easier not to have to find one. >> >> Rebase 2 currently satisfies these requirements, and since I'd like to >> move to Rebase3 I'd like to find working alternatives. >> >> Eric, sure, I can host Hex privately. Though, you'll probably agree that >> it is way more easier to just go with the vendoring thing. :) >> >> >> That being said, I'd like for you, Tristan, Fred, Eric, to try and >> welcome my little feedback here, and take it for what it is. I'd like it to >> be easy and possible to discuss these ideas serenely, because relying on a >> system that considers (or better yet, values) the community input such as >> mine would make me sleep better at night :) >> >> All the best, >> r. >> >> > > Just to add the discussion, "vendors" dependencies or at least a way to > have them in the source archive is a good way to ship the source code to > customers. Doing it remove any dependency to an external source that could > disappear or change over the time. It provides a final source product to > customers that can still be edited/modified in future without to worry > much. (they just need to make sure to archive correctly the deliverables). > > I had a look at the vendor plugin and it's a good start. But I think the "rebar3 > vendor apply" should be automatic when you launch a build. Says if you > have a _vendor dir or something like it it would be used to deploy the deps > inside. Then we could have the following commands: > > rebar3 vendor fetch > rebar3 vendor upgrade > > What do you think about it? Can we right now hooks the way the dependency > is retrieved? Or do we need to provide some kind of custom dep resource [1] > that would proxy each deps? Seem like right now that _checkouts is an hack > and something like it can't be done externally. Anyway any hint is > appreciated. I also need to have such feature on a day basis :) > > - benoit > > > [1] http://www.rebar3.org/docs/custom-dep-resources > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Sat Mar 19 12:45:50 2016 From: michael.santos@REDACTED (Michael Santos) Date: Sat, 19 Mar 2016 07:45:50 -0400 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: Message-ID: On Mar 18, 2016 10:46 PM, "Khitai Pang" wrote: > > Hi, > > I have been using the esl-erlang pacakge from https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for Trusty. > > I have these in my bashrc: > > export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in the cloud) > export ERL_EPMD_PORT="5678" Just tested and it worked for me. Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > > Everything worked fine with the 18.2 package. After upgrading to esl-erlang 18.3, epmd stopped to work: > > > epmd -debug > epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot assign requested address > > There is a workaround: if I remove the ERL_EPMD_ADDRESS environment variable, epms works: > > tcp 0 0 0.0.0.0:5678 0.0.0.0:* LISTEN 3328/epmd > tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > > After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Mar 19 13:14:58 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 19 Mar 2016 08:14:58 -0400 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: If the default mode isn't enough, the easiest solution is always: ct_run -dir test -pa `rebar3 path` Which makes your whole app available to the tests. You can bypass rebar3 for the run, but still use it for the rest. You may want to try: rebar3 shell --sname=mynode ... r3:do(ct). To run tests from the interactive shell with a named node, but I have never tried that and don't know how it would go. On Mar 19, 2016 5:07 AM, "Roberto Ostinelli" wrote: > All, > When I manually run CT with: > > ct_run -dir test -pa ebin > > and print out the node the tests are running on with a simple: > > ct:pal("node = ~p", [node()]). > > I get something similar to: > > 'ct@REDACTED' > > > However, when I run CT with rebar3 by issuing: > > rebar3 ct > > Then the same command prints: > > 'nonode@REDACTED' > > This is ok for most tests, but since I'm testing distributed nodes with > ct_slave [1] this impedes a slave to be started, as the main node is a > 'nonode@REDACTED' and hence the slave node cannot connect to this 'nonode'. > > If anyone wants to try this just run Syn's suite, it will fail when using > rebar3 here: > > https://github.com/ostinelli/syn/blob/1.1.0/test/syn_test_suite_helper.erl#L62 > > With a {{badmatch,{error,not_alive,nonode@REDACTED}}. > > Any ideas? > > Thank you, > r. > > > [1] http://erlang.org/doc/man/ct_slave.html > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Mar 19 13:24:27 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 19 Mar 2016 08:24:27 -0400 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: <20160319122425.GJ53561@ferdmbp.local> On 03/19, Roberto Ostinelli wrote: >Tristan, Fred, Eric >That being said, I'd like for you, Tristan, Fred, Eric, to try and >welcome >my little feedback here, and take it for what it is. I'd like it to be easy >and possible to discuss these ideas serenely, because relying on a system >that considers (or better yet, values) the community input such as mine >would make me sleep better at night :) > We do, Tristan even started a toy plugin here for you :) It would certainly have been easier for us if these concerns had been brought to us in the last year and a half we have spent in beta, alpha and pre-alpha, though, rather than week 1 after stable release. It's now too late to change the core, so we must think in plugins. Hopefully what we can come up with will prove satisfactory. From mononcqc@REDACTED Sat Mar 19 13:32:56 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 19 Mar 2016 08:32:56 -0400 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: <20160319123255.GK53561@ferdmbp.local> On 03/19, Benoit Chesneau wrote: > >What do you think about it? Can we right now hooks the way the >dependency is retrieved? Or do we need to provide some kind of custom >dep resource [1] that would proxy each deps? http://www.rebar3.org/docs/configuration#section-hookable-points-in-providers Currently those are the only hookable providers. Either we'd need to add hookable providers [1], or you could use aliases: http://www.rebar3.org/docs/using-available-plugins#alias Or run your build as hooks of your vendoring provider. >Seem like right now that _checkouts is an hack >and something like it can't be done externally. It is. The plugin is a better solution. [1] probably https://github.com/erlang/rebar3/blob/master/src/rebar_app_discover.erl From Oliver.Korpilla@REDACTED Sat Mar 19 12:54:53 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 19 Mar 2016 12:54:53 +0100 Subject: [erlang-questions] Problems with efficiently parsing ASN.1 S1AP decoder output Message-ID: An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 15:55:49 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 19 Mar 2016 09:55:49 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <20160319123255.GK53561@ferdmbp.local> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319123255.GK53561@ferdmbp.local> Message-ID: <1458399349.2344374.553865306.31584C7B@webmail.messagingengine.com> Yea, I'd change it to hook before install_deps. No need to do it for each individual dependency, just copy them all from the vendor dir to the build dir once before rebar3 tries to resolve dependencies. -- Tristan Sloughter t@REDACTED On Sat, Mar 19, 2016, at 07:32 AM, Fred Hebert wrote: > On 03/19, Benoit Chesneau wrote: > > > >What do you think about it? Can we right now hooks the way the > >dependency is retrieved? Or do we need to provide some kind of custom > >dep resource [1] that would proxy each deps? > > http://www.rebar3.org/docs/configuration#section-hookable-points-in-providers > > Currently those are the only hookable providers. Either we'd need to add > hookable providers [1], or you could use aliases: > > http://www.rebar3.org/docs/using-available-plugins#alias > > Or run your build as hooks of your vendoring provider. > > >Seem like right now that _checkouts is an hack > >and something like it can't be done externally. > > It is. The plugin is a better solution. > > [1] probably > https://github.com/erlang/rebar3/blob/master/src/rebar_app_discover.erl > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t@REDACTED Sat Mar 19 15:58:20 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 19 Mar 2016 09:58:20 -0500 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: <1458399500.2344933.553865882.44C41776@webmail.messagingengine.com> Additionally if you use `rebar3 local install` you are able to set the os var $REBAR3_ERL_ARGS which you can set to `-sname mynode`. -- Tristan Sloughter t@REDACTED On Sat, Mar 19, 2016, at 07:14 AM, Fred Hebert wrote: > If the default mode isn't enough, the easiest solution is always: > ct_run -dir test -pa `rebar3 path` > Which makes your whole app available to the tests. You can bypass > rebar3 for the run, but still use it for the rest. > You may want to try: > rebar3 shell --sname=mynode > ... > r3:do(ct). > To run tests from the interactive shell with a named node, but I have > never tried that and don't know how it would go. > On Mar 19, 2016 5:07 AM, "Roberto Ostinelli" wrote: >> All, >> When I manually run CT with: >> >> ct_run -dir test?-pa ebin >> >> and print out the node the tests are running on with a simple: >> >> ct:pal("node = ~p", [node()]). >> >> I get something similar to: >> >> 'ct@REDACTED' >> >> >> However, when I run CT with rebar3 by issuing: >> >> rebar3 ct >> >> Then the same command prints: >> >> 'nonode@REDACTED' >> >> This is ok for most tests, but since I'm testing distributed nodes >> with ct_slave [1] this impedes a slave to be started, as the main >> node is a 'nonode@REDACTED' and hence the slave node cannot connect to >> this 'nonode'. >> >> If anyone wants to try this just run Syn's suite, it will fail when using rebar3 here: >> https://github.com/ostinelli/syn/blob/1.1.0/test/syn_test_suite_helper.erl#L62 >> >> With a?{{badmatch,{error,not_alive,nonode@REDACTED}}. >> >> Any ideas? >> >> Thank you, >> r. >> >> >> [1]?http://erlang.org/doc/man/ct_slave.html >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 16:03:00 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 19 Mar 2016 10:03:00 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> Message-ID: <1458399780.2345777.553866778.705748E7@webmail.messagingengine.com> Roberto, Benoit, understandable. I hated vendoring as a package managing solution and came to even more when working on Go projects for a period of time :). But you are right, it makes sense to have some support for these limited cases. But I do hope a plugin is sufficient to fulfil the needs and it not requiring any core changes.. -- Tristan Sloughter t@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Sat Mar 19 16:47:49 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sat, 19 Mar 2016 23:47:49 +0800 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: Message-ID: On 2016/3/19 19:45, Michael Santos wrote: > > On Mar 18, 2016 10:46 PM, "Khitai Pang" > wrote: > > > > Hi, > > > > I have been using the esl-erlang pacakge from > https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for Trusty. > > > > I have these in my bashrc: > > > > export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in the > cloud) > > export ERL_EPMD_PORT="5678" > > Just tested and it worked for me. > > Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > Yes it does, it's the IP of the LAN in the clound, and it all works fine with 18.2. I believe it's possible that the issue is caused by the configuration in the operating system (Ubuntu 14.04.4 x86_64) but currently I have no idea how to further debug this issue. With esl-erlang 18.2 and the following configs in bashrc: export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" export ERL_EPMD_PORT="5678" epmd works fine: tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd Thanks Khitai > > > > Everything worked fine with the 18.2 package. After upgrading to > esl-erlang 18.3, epmd stopped to work: > > > > > epmd -debug > > epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > > epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot assign > requested address > > > > There is a workaround: if I remove the ERL_EPMD_ADDRESS environment > variable, epms works: > > > > tcp 0 0 0.0.0.0:5678 0.0.0.0:* > LISTEN 3328/epmd > > tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > > > > After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > > > > > > Thanks > > Khitai > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emil@REDACTED Sat Mar 19 17:34:47 2016 From: emil@REDACTED (Emil Holmstrom) Date: Sat, 19 Mar 2016 16:34:47 +0000 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> Message-ID: I fail to see the significance of the type system in this case, it doesn't stop Erlang to have a char() type? It has float(), integer(), atom(), etc... Too force lists to have the same element type is still possible even if Erlang is dynamically typed. Unfortunately iolist() would have to go. Maybe I am missing something obvious? /emil On Fri, 18 Mar 2016 at 18:15, Hynek Vychodil wrote: > A superlative suggestion sir, with only two minor drawbacks: one, Erlang > is dynamically typed language and two, Erlang is dynamically typed > language. I know that technically that?s only one drawback, but I thought > it was such a big one it was worth mentioning twice. > > Hynek > > On Fri, Mar 18, 2016 at 5:30 PM, Emil Holmstrom wrote: > >> I am probably repeating what someone else already have said in some other >> similar thread. >> >> The confusion between strings and [integer()] would have been greatly >> reduced if char() existed, $a wouldn't have to be syntactic sugar for 97 >> but would actually be "character a". You would have to explicitly convert >> char() -> integer() and wise versa. This is how strings are implemented in >> ML and Haskell. >> >> Regarding character encoding: inside Erlang Unicode could always be >> assumed, converson between different character encodings could be done on >> I/O. >> >> /emil >> >> On Fri, 18 Mar 2016 at 00:51, Richard A. O'Keefe >> wrote: >> >>> >>> >>> On 17/03/16 11:53 pm, Steve Davis wrote: >>> > > ROK said: >>> > > Yawn. >>> > (What am I doing trying to argue with ROK??? Am I MAD?) >>> > >>> > 1) Why is it people rant about "string handling" in Erlang? >>> >>> Because it is not the same as Java. >>> > >>> > 2) Principle of least surprise: >>> > 1> [H|T] = [22,87,65,84,33]. >>> > [22,87,65,84,33] >>> > 2> H. >>> > 22 >>> > 3> T. >>> > "WAT!? >>> This is a legitimate complaint, but it confuses two things. >>> There is *STRING HANDLING*, which is fine, and >>> there is *LIST PRINTING*, which causes the confusion. >>> >>> For comparison, DEC-10 Prolog, PDP-11 Prolog, C-Prolog, and Quintus >>> Prolog >>> all did STRING HANDLING as lists of character codes, but >>> all did LIST PRINTING without ever converting lists of numbers to >>> strings. >>> The answer was that there was a library procedure to print a list of >>> integers as a string and you could call that whenever you wanted to, >>> such as in a user-defined pretty-printing procedure. Here's a transcript >>> from SICStus Prolog: >>> | ?- write([65,66,67]). >>> [65,66,67] >>> yes >>> | ?- write("ABC"). >>> [65,66,67] >>> yes >>> >>> The heuristic used by the debugger in some Prologs was that a list of >>> integers between 32 and 126 inclusive was printed as a string; that >>> broke down with Latin 1, and broke harder with Unicode. The simple >>> behaviour mandated by the standard that lists of integers print as >>> lists of integers confuses people once, then they learn that string >>> quotes are an input notation, not an output notation, and if they want >>> string notation in output, they have to call a special procedure to get >>> it. >>> >>> The ISO Prolog committee introduced a horrible alternative which the >>> DEC-10 Prolog designers had experienced in some Lisp systems and >>> learned to hate: flip a switch and "ABC" is read as ['A','B','C']. The >>> principal reason given for that was that the output was semi-readable. >>> One of my arguments against it was that this required every Prolog >>> system to be able to hold 17*2**16 atoms, and I new for a fact that >>> many would struggle to do so. The retort was "they must be changed >>> to make a special case for one-character atoms". Oh well, no silver >>> bullet. >>> >>> That does serve as a reminder, though, that using [a,b,c] instead of >>> [$a,$b,$c] is *possible* in Erlang. >>> >>> Just to repeat the basic point: the printing of (some) integer lists as >>> strings is SEPARABLE from the issue of how strings are represented and >>> processed; that could be changed without anything else in the language >>> changing. >>> > >>> > 3) A codec should be perfectly reversible i.e. X = encode(decode(X)). >>> > Without tagging, merely parsing out a string as a list is not >>> > perfectly reversible. >>> Here you are making a demand that very few other programming languages >>> can support. For example, take JavaScript. "\u0041" is read as "A", >>> and you are not going to get "\u0041" back from "A". You're not even >>> going to get "\x41" back from it, even though "\x41" == "A". >>> >>> Or take Erlang, where >>> 1> 'foo bar'. >>> 'foo bar' >>> 2> 'foobar'. >>> foobar >>> with the same kind of thing happening in Prolog. >>> >>> And of COURSE reading [1 /* one */, 2 /* deux */, 4 /* kvar */] >>> in JavaScript preserves the comments so that re-encoding the >>> data structure restores the input perfectly. >>> >>> Or for that matter consider floating point numbers, where >>> even the languages that produce the best possible conversions >>> cannot promise that encode(decode(x)) == x. >>> >>> No, I'm sorry, this "perfectly reversible codec" requirement sets up >>> a standard that NO programming language I'm aware of satisfies. >>> It is, in fact, a straw man. What you *can* ask, and what some >>> language designers and implementers strive to give you, is >>> decode(encode(decode(x))) == decode(x). >>> >>> But to repeat the point made earlier, the way that lists of plausible >>> character codes is printed is SEPARABLE from the way strings are >>> represented and handled and in an ancestral language is SEPARATE. >>> > >>> > 4) What is the right way to implement the function is_string(List) >>> > correctly? >>> > >>> > *ducks* >>> >>> That really is a "have you stopped beating your wife, answer yes or no" >>> sort of question. >>> >>> It depends on the semantics you *want* it to have. The Quintus >>> library didn't provide any such predicate, but it did provide >>> >>> plausible_chars(Term) >>> when Term is a sequence of integers satisfying >>> is_graphic(C) or is_space(C), >>> possibly ending with a tail that is a variable or >>> a variable bound by numbervars/3. >>> >>> Notice the careful choice of name: not IS (certainly) a string, >>> but is a PLAUSIBLE list of characters. >>> >>> It was good enough for paying customers to be happy with the >>> module it was part of (which was the one offering the >>> non-usual portray_chars(Term) command). >>> >>> One of the representations Quintus used for strings (again, a >>> library feature, not a core language feature) was in Erlang >>> notation {external_string,FileName,Offset,Length}, and idea >>> that struck the customer I developed it for as a great >>> innovation, when I'd simply stolen it from Smalltalk! >>> >>> The thing is that STRINGS ARE WRONG for most things, >>> however represented. For example, when Java changed >>> the representation of String so that slicing became a >>> costly operation, I laughed, because I had my own representation >>> of strings that provided O(1) concatenation as well as cheap >>> slicing. (Think Erlang iolists and you won't be far wrong.) >>> The Pop2 language developed and used at Edinburgh >>> represented file names as lists, e.g., [/dev/null] was in >>> Erlang notation ['/',dev.'/',null]. This made file name >>> manipulation easier than representing them as strings. >>> Any time there is internal structure, any time there is scope >>> for sharing substructure, any time you need to process >>> the parts of a string, strings are wrong. >>> >>> The PERL lesson is that regular expressions are a fantastic >>> tool for doing the wrong thing quite simply. >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Sat Mar 19 17:42:00 2016 From: michael.santos@REDACTED (Michael Santos) Date: Sat, 19 Mar 2016 12:42:00 -0400 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: Message-ID: <20160319164200.GA29068@brk> On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: > On 2016/3/19 19:45, Michael Santos wrote: > > > >On Mar 18, 2016 10:46 PM, "Khitai Pang" >> wrote: > >> > >> Hi, > >> > >> I have been using the esl-erlang pacakge from > >https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for > >Trusty. > >> > >> I have these in my bashrc: > >> > >> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in > >the cloud) > >> export ERL_EPMD_PORT="5678" > > > >Just tested and it worked for me. > > > >Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > > > > Yes it does, it's the IP of the LAN in the clound, and it all works > fine with 18.2. > > I believe it's possible that the issue is caused by the > configuration in the operating system (Ubuntu 14.04.4 x86_64) but > currently I have no idea how to further debug this issue. > > With esl-erlang 18.2 and the following configs in bashrc: > > export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" > export ERL_EPMD_PORT="5678" > > epmd works fine: > > tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd > tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd > > > Thanks > Khitai > Can you provide the he output of env and ifconfig -a? If you would like to keep it private, feel free to send it offlist. > >> > >> Everything worked fine with the 18.2 package. After upgrading > >to esl-erlang 18.3, epmd stopped to work: > >> > >> > epmd -debug > >> epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > >> epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot > >assign requested address > >> > >> There is a workaround: if I remove the ERL_EPMD_ADDRESS > >environment variable, epms works: > >> > >> tcp 0 0 0.0.0.0:5678 0.0.0.0:* > >LISTEN 3328/epmd > >> tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > >> > >> After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > >> > >> > >> Thanks > >> Khitai > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > From mjtruog@REDACTED Sat Mar 19 19:36:55 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 19 Mar 2016 11:36:55 -0700 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <20160319122425.GJ53561@ferdmbp.local> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> Message-ID: <56ED9C47.9000904@gmail.com> On 03/19/2016 05:24 AM, Fred Hebert wrote: > On 03/19, Roberto Ostinelli wrote: >> Tristan, Fred, Eric >> That being said, I'd like for you, Tristan, Fred, Eric, to try and welcome >> my little feedback here, and take it for what it is. I'd like it to be easy >> and possible to discuss these ideas serenely, because relying on a system >> that considers (or better yet, values) the community input such as mine >> would make me sleep better at night :) >> > > We do, Tristan even started a toy plugin here for you :) > > It would certainly have been easier for us if these concerns had been brought to us in the last year and a half we have spent in beta, alpha and pre-alpha, though, rather than week 1 after stable release. > > It's now too late to change the core, so we must think in plugins. Hopefully what we can come up with will prove satisfactory. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > . > I have made sure to ask about this in the past, so I don't think you can claim ignorance on this use-case. The vendor plugin has been the first attempt at a clear response, for which I am grateful. rebar2 had its core changed to become rebar3 and any source code that is not dead can change. So, I see no reason why rebar3 is unable to change. It isn't as if rebar3 has a technical design document to follow in a waterfall model or fits to an ISO spec, or even an RFC. From jesper.louis.andersen@REDACTED Sat Mar 19 20:43:47 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 19 Mar 2016 20:43:47 +0100 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> Message-ID: On Sat, Mar 19, 2016 at 5:34 PM, Emil Holmstrom wrote: > I fail to see the significance of the type system in this case, it doesn't > stop Erlang to have a char() type? Right. You would have to invent a new tag for it, and you would have to potentially take a performance hit due to the new tagging scheme, but adding distinct values at runtime is fairly easy to do. With different tags, $a and 97 are now different values, and conversion between them is now explicit. What makes this idea more powerful in statically typed languages has to do with erasure: once we have a compiled program with a picked representation, we can choose the same representation for multiple types. This is what makes it efficient: char and int are both implemented as integers internally, but the type system protects against misusing one as the other. Erlang terms must be self-describing. With a more complex type zoo comes the additional burden of managing differences among types however. This is not free for the programmer, and most programming languages try to strike a balance between precision and convenience. Of course, there are different opinions as to what level is needed. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 19 22:06:31 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 19 Mar 2016 16:06:31 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56ED9C47.9000904@gmail.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> Message-ID: <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> I'm pretty sure we always said it was possible with plugins or custom resources when it was brought up. Which remains the case. -- Tristan Sloughter t@REDACTED On Sat, Mar 19, 2016, at 01:36 PM, Michael Truog wrote: > On 03/19/2016 05:24 AM, Fred Hebert wrote: > > On 03/19, Roberto Ostinelli wrote: > >> Tristan, Fred, Eric > >> That being said, I'd like for you, Tristan, Fred, Eric, to try and welcome > >> my little feedback here, and take it for what it is. I'd like it to be easy > >> and possible to discuss these ideas serenely, because relying on a system > >> that considers (or better yet, values) the community input such as mine > >> would make me sleep better at night :) > >> > > > > We do, Tristan even started a toy plugin here for you :) > > > > It would certainly have been easier for us if these concerns had been brought to us in the last year and a half we have spent in beta, alpha and pre-alpha, though, rather than week 1 after stable release. > > > > It's now too late to change the core, so we must think in plugins. Hopefully what we can come up with will prove satisfactory. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > . > > > > I have made sure to ask about this in the past, so I don't think you can > claim > ignorance on this use-case. The vendor plugin has been the first attempt > at a clear response, for which I am grateful. rebar2 had its core > changed > to become rebar3 and any source code that is not dead can change. > So, I see no reason why rebar3 is unable to change. It isn't as if > rebar3 > has a technical design document to follow in a waterfall model or fits to > an ISO spec, or even an RFC. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.santos@REDACTED Sat Mar 19 23:53:25 2016 From: michael.santos@REDACTED (Michael Santos) Date: Sat, 19 Mar 2016 18:53:25 -0400 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: Message-ID: <20160319225325.GA29707@brk> On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: > On 2016/3/19 19:45, Michael Santos wrote: > > > >On Mar 18, 2016 10:46 PM, "Khitai Pang" >> wrote: > >> > >> Hi, > >> > >> I have been using the esl-erlang pacakge from > >https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for > >Trusty. > >> > >> I have these in my bashrc: > >> > >> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in > >the cloud) > >> export ERL_EPMD_PORT="5678" > > > >Just tested and it worked for me. > > > >Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > > > > Yes it does, it's the IP of the LAN in the clound, and it all works > fine with 18.2. > > I believe it's possible that the issue is caused by the > configuration in the operating system (Ubuntu 14.04.4 x86_64) but > currently I have no idea how to further debug this issue. > > With esl-erlang 18.2 and the following configs in bashrc: > > export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" > export ERL_EPMD_PORT="5678" > > epmd works fine: > > tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd > tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd > > > Thanks > Khitai > > >> > >> Everything worked fine with the 18.2 package. After upgrading > >to esl-erlang 18.3, epmd stopped to work: > >> > >> > epmd -debug > >> epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > >> epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot > >assign requested address > >> > >> There is a workaround: if I remove the ERL_EPMD_ADDRESS > >environment variable, epms works: > >> > >> tcp 0 0 0.0.0.0:5678 0.0.0.0:* > >LISTEN 3328/epmd > >> tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > >> > >> After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. Have you disabled IPv6? # disable IPv6 support on the loopback device $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1 $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address # enable IPv6 support on the loopback device $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd If we look at what is happening: $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd ... bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address) ... When you specify a specific address to bind, epmd will also bind to the IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has disabled it on the loopback device so the bind fails: $ ifconfig lo lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) So either enable IPv6 for the loopback or disable IPv6 completely by passing "ipv6.disable=1" at boot. From roberto@REDACTED Sun Mar 20 10:07:03 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2016 10:07:03 +0100 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: Thank you for the help. On Sat, Mar 19, 2016 at 1:14 PM, Fred Hebert wrote: > If the default mode isn't enough, the easiest solution is always: > > ct_run -dir test -pa `rebar3 path` > > Which makes your whole app available to the tests. You can bypass rebar3 > for the run, but still use it for the rest. > This fixes the node naming issue, but unfortunately this not enough to make it work. Long story short: in my tests I define some callback functions that need to be called for the tests to pass. However, I need to specify the path to the test directory too, because otherwise my slave node doesn't have these functions defined. This is how I do the trick now: https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L59 (this passes) https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L60 (this directory does not exist when I use rebar3). SO: is there a way to have rebar copy the test directory to: syn/_build/default/lib/syn/test ? You may want to try: > > rebar3 shell --sname=mynode > ... > r3:do(ct). > > To run tests from the interactive shell with a named node, but I have > never tried that and don't know how it would go. > This actually works, though I see an "experimental" warning on top and I cannot really automate it in a script, as there's no `--eval` option in `rebar shell`. So I've tried adding a test.escript file: %%%%%%%%%%%%%%%%%%% #!/usr/bin/env escript main(_) -> r3:do(ct), init:stop(). %%%%%%%%%%%%%%%%%%% But: $ rebar3 shell --sname=mynode --script test.escript ===> Verifying dependencies... ===> Compiling syn Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] [dtrace] Eshell V7.3 (abort with ^G) (mynode@REDACTED)1> ===> Couldn't run shell escript "/Users/roberto/workspace/syn/test.escript" - exit:{noproc, {gen_server, call, [rebar_agent, {cmd, ct}, infinity]}} Stack: [{gen_server,call,3,[{file,"gen_server.erl"},{line,212}]}, {test_escript__escript__1458__464329__444078__2,main,1, [{file,"/Users/roberto/workspace/syn/test.escript"},{line,5}]}, And I'm stuck. Any ideas? -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Mar 20 10:22:20 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2016 10:22:20 +0100 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: On Sun, Mar 20, 2016 at 10:07 AM, Roberto Ostinelli wrote: > > Long story short: in my tests I define some callback functions that need > to be called for the tests to pass. > However, I need to specify the path to the test directory too, because > otherwise my slave node doesn't have these functions defined. > > This is how I do the trick now: > > https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L59 > (this passes) > > https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L60 > (this directory does not exist when I use rebar3). > > SO: is there a way to have rebar copy the test directory to: > syn/_build/default/lib/syn/test > > ? > Found an alternative to this, leaving it here for anyone interested. I start the slave node with: start_slave(NodeShortName) -> CodePath = code:get_path(), {ok, Node} = ct_slave:start(NodeShortName, [{boot_timeout, 10}]), true = rpc:call(Node, code, set_path, [CodePath]), {ok, Node}. Hence I do not need to "rebuild" the directories. Fred, Tristan, thank you for your help I can consider this matter closed. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Mar 20 13:55:30 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2016 13:55:30 +0100 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: One question I need to ask though: why does `rebar ct` not set the node? This is actually set by CT itself, so why does it get removed? Thanks, r. On Sun, Mar 20, 2016 at 10:22 AM, Roberto Ostinelli wrote: > > > On Sun, Mar 20, 2016 at 10:07 AM, Roberto Ostinelli > wrote: > >> >> Long story short: in my tests I define some callback functions that need >> to be called for the tests to pass. >> However, I need to specify the path to the test directory too, because >> otherwise my slave node doesn't have these functions defined. >> >> This is how I do the trick now: >> >> https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L59 >> (this passes) >> >> https://github.com/ostinelli/syn/blob/1.2.0/test/syn_test_suite_helper.erl#L60 >> (this directory does not exist when I use rebar3). >> >> SO: is there a way to have rebar copy the test directory to: >> syn/_build/default/lib/syn/test >> >> ? >> > > Found an alternative to this, leaving it here for anyone interested. I > start the slave node with: > > start_slave(NodeShortName) -> > CodePath = code:get_path(), > {ok, Node} = ct_slave:start(NodeShortName, [{boot_timeout, 10}]), > true = rpc:call(Node, code, set_path, [CodePath]), > {ok, Node}. > > Hence I do not need to "rebuild" the directories. > > Fred, Tristan, thank you for your help I can consider this matter closed. > > Best, > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sun Mar 20 14:37:28 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 20 Mar 2016 09:37:28 -0400 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: References: Message-ID: <20160320133727.GL53561@ferdmbp.local> On 03/20, Roberto Ostinelli wrote: >One question I need to ask though: why does `rebar ct` not set the node? >This is actually set by CT itself, so why does it get removed? > I don't think we ever had a conscious decision to do it. When `rebar3 ct` was implemented, we switched away from rebar 2.x's shelling out to 'ct_run', since that had the nasty habit of leaving behind a named node you couldn't interrupt and had to call 'kill -9' on to be able to run further tests. When switching to the direct erlang API for 'ct', the node naming feature was probably just lost by accident. Incidentally it now lets people run more than one test run at a time on a given system, which may not be all that bad. One tricky aspect of distributed Erlang with escripts is that it still relies on epmd, but if epmd is not running yet, the escript/net_kernel will not boot the process for you; we'd have to shell out to it manually and hope it doesn't get taken down when the escript is done running (if any other system started relying on it at the tiem). I never felt super comfortable with that, and so far 'rebar3 shell' just displays a warning asking you to boot epmd and ignores distributed mode when that happens. That's probably the road I'd go for with `rebar3 ct` too: make in an explicit option, but don't lock up a node name (and possibly boot epmd) by default if the user hasn't asked for it? Currently it's just plain unsupported though. From lloyd@REDACTED Sun Mar 20 14:54:47 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sun, 20 Mar 2016 09:54:47 -0400 Subject: [erlang-questions] Erlang in education --- back from the dead In-Reply-To: References: Message-ID: Hi Michael, Actually, teaching elementary students how to program is an interesting problem, but wasn't what I had in mind when I started this thread. I was thinking about how to leverage low-cost, low-power ARM boards like the Raspberry Pi 3 to evangelize Erlang among three groups: o makers -- or at least those interested in advancing their skills beyond Arduino and wiring. A single board and Erlang stack would suffice. o software engineers interested in learning more about distributed computing; mnesia replication, failure, and recovery; and/or Riak. ARM clusters of two to five nodes would do the trick. o software engineers interested in learning all about network protocols. Two boards running Erlang would get these folks well under way. So, cost of entry would range from $50 to $450 per student --- not counting instructional materials. My sense is that there any number of hidden-gem Erlang libraries and open-source libraries that are neglected or, at best, under used for lack of good documentation. This could be a powerful way to expand interest and adoption. My assumption here is that a more vibrant and knowledgable Erlang community benefits us all. All the best, LRP Sent from my iPad > On Mar 19, 2016, at 9:17 PM, Michael Turner wrote: > > I'm not writing Erlang anymore. I keep meaning to unsubscribe from the list, but then something interesting pops up. > > I was involved briefly in the Computer Education Project at Lawrence Hall of Science in Berkeley, back in 1969-1971. Logo was just starting to happen there at the time. > > Regards, > Michael Turner > Executive Director > Project Persephone > K-1 bldg 3F > 7-2-6 Nishishinjuku > Shinjuku-ku Tokyo 160-0023 > Tel: +81 (3) 6890-1140 > Fax: +81 (3) 6890-1158 > Mobile: +81 (90) 5203-8682 > turner@REDACTED > http://www.projectpersephone.org/ > > "Love does not consist in gazing at each other, but in looking outward together in the same direction." -- Antoine de Saint-Exup?ry > >> On Sat, Mar 19, 2016 at 11:16 PM, Lloyd R. Prentice wrote: >> Thanks, Michael, >> >> Way back in the day I was the founding editor and publisher of Classroom Computer News, the first magazine devoted to computer applications in K-12 classrooms. I was aware of Alan Kay's work at the time, but haven't kept up. Seymour Papert's work with Logo at MIT was a major figure and influence at that time. The big controversy was Computer-Assisted Instruction vs. teaching programming skills. >> >> Are you currently teaching or developing in Erlang? >> >> All the best, >> >> Lloyd >> >> Sent from my iPad >> >>> On Mar 19, 2016, at 1:28 AM, Michael Turner wrote: >>> >>> Some years back, I mentioned that Erlang seemed a lot closer to what Alan Kay originally meant by "object oriented" (which at the time did not include inheritance) than most O-O languages since. He has long been occupied with the issues of education through writing software, even in the early grades. His foundation's software work seems a bit moribund (I guess Open Cobalt is what it has come down to, not a wildly active project) but what they've done over the years might provide some ideas. >>> >>> >>> >>> Regards, >>> Michael Turner >>> Executive Director >>> Project Persephone >>> K-1 bldg 3F >>> 7-2-6 Nishishinjuku >>> Shinjuku-ku Tokyo 160-0023 >>> Tel: +81 (3) 6890-1140 >>> Fax: +81 (3) 6890-1158 >>> Mobile: +81 (90) 5203-8682 >>> turner@REDACTED >>> http://www.projectpersephone.org/ >>> >>> "Love does not consist in gazing at each other, but in looking outward together in the same direction." -- Antoine de Saint-Exup?ry >>> >>>> On Thu, Mar 17, 2016 at 11:48 AM, Lloyd R. Prentice wrote: >>>> Hello, >>>> >>>> A year or so ago there was on this list a wonderful spirited discussion of how to get kids and educators interested in Erlang. >>>> >>>> We have now low-cost low-power 64-bit hardware platforms that open boundless opportunities: >>>> >>>> https://www.raspberrypi.org/blog/raspberry-pi-3-on-sale/ >>>> http://www.hardkernel.com/main/products/prdt_info.php?g_code=G145457216438 >>>> https://www.pine64.com/product >>>> >>>> http://www.techspot.com/news/64113-western-digital-creates-314gb-hard-drive-specifically-raspberry.html >>>> >>>> So, I'm wondering if the Erlang community, particularly our corporate members, would be interested in mounting a programming challenge to students of all ages to develop innovative applications on one or another of these platforms? All entries would be released as open-source. Incentives might range from hardware to cash to internships to jobs. I, in my Writers Glen incarnation, would be willing to seed the program with $500 and devote some time if we can secure enough interest and resources to mount a credible program. >>>> >>>> Or, short of such a grandiose scheme, I'm wondering if there are folks in the community interested in cooperating off-list in exploring and publishing on-line or in print under Creative Commons license how-tos and tutorials designed to inspire kids and push these platforms to their limits. >>>> >>>> Best wishes, >>>> >>>> LRP >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Sent from my iPad >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun Mar 20 15:08:45 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 20 Mar 2016 15:08:45 +0100 Subject: [erlang-questions] rebar3 and CT's main node In-Reply-To: <20160320133727.GL53561@ferdmbp.local> References: <20160320133727.GL53561@ferdmbp.local> Message-ID: Thank you Fred for clarifying. I guess that for tests that require distributed node, devs can just use the shell (assuming they are using a test profile): ct_run -dir test -pa `rebar3 as test path` Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Sun Mar 20 19:23:05 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 02:23:05 +0800 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: <20160319225325.GA29707@brk> References: <20160319225325.GA29707@brk> Message-ID: On 2016/3/20 6:53, Michael Santos wrote: > On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: >> On 2016/3/19 19:45, Michael Santos wrote: >>> On Mar 18, 2016 10:46 PM, "Khitai Pang" >> > wrote: >>>> Hi, >>>> >>>> I have been using the esl-erlang pacakge from >>> https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for >>> Trusty. >>>> I have these in my bashrc: >>>> >>>> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in >>> the cloud) >>>> export ERL_EPMD_PORT="5678" >>> Just tested and it worked for me. >>> >>> Does the IP address in ERL_EPMD_ADDRESS exist in the VM? >>> >> Yes it does, it's the IP of the LAN in the clound, and it all works >> fine with 18.2. >> >> I believe it's possible that the issue is caused by the >> configuration in the operating system (Ubuntu 14.04.4 x86_64) but >> currently I have no idea how to further debug this issue. >> >> With esl-erlang 18.2 and the following configs in bashrc: >> >> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" >> export ERL_EPMD_PORT="5678" >> >> epmd works fine: >> >> tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd >> tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd >> >> >> Thanks >> Khitai >> >>>> Everything worked fine with the 18.2 package. After upgrading >>> to esl-erlang 18.3, epmd stopped to work: >>>>> epmd -debug >>>> epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 >>>> epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot >>> assign requested address >>>> There is a workaround: if I remove the ERL_EPMD_ADDRESS >>> environment variable, epms works: >>>> tcp 0 0 0.0.0.0:5678 0.0.0.0:* >>> LISTEN 3328/epmd >>>> tcp6 0 0 :::5678 :::* LISTEN 3328/epmd >>>> >>>> After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > Have you disabled IPv6? > > # disable IPv6 support on the loopback device > $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1 > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address > > # enable IPv6 support on the loopback device > $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > > > If we look at what is happening: > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd > ... > bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address) > ... > > When you specify a specific address to bind, epmd will also bind to the > IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has > disabled it on the loopback device so the bind fails: > > $ ifconfig lo > lo Link encap:Local Loopback > inet addr:127.0.0.1 Mask:255.0.0.0 > UP LOOPBACK RUNNING MTU:65536 Metric:1 > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > So either enable IPv6 for the loopback or disable IPv6 completely by > passing "ipv6.disable=1" at boot. Michael, Thanks for the workaround. 2 questions: 1) I have these in my sysctl: net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.eth0.disable_ipv6 = 1 net.ipv6.conf.eth1.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 Aren't these enough to tell epmd that ipv6 is not available on loopback? 2) Why does 18.2 work in the same environment? Thanks Khitai From michael.santos@REDACTED Sun Mar 20 22:27:59 2016 From: michael.santos@REDACTED (Michael Santos) Date: Sun, 20 Mar 2016 17:27:59 -0400 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: <20160319225325.GA29707@brk> Message-ID: <20160320212759.GB29707@brk> On Mon, Mar 21, 2016 at 02:23:05AM +0800, Khitai Pang wrote: > On 2016/3/20 6:53, Michael Santos wrote: > >On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: > >>On 2016/3/19 19:45, Michael Santos wrote: > >>>On Mar 18, 2016 10:46 PM, "Khitai Pang" >>>> wrote: > >>>>Hi, > >>>> > >>>>I have been using the esl-erlang pacakge from > >>>https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for > >>>Trusty. > >>>>I have these in my bashrc: > >>>> > >>>>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in > >>>the cloud) > >>>>export ERL_EPMD_PORT="5678" > >>>Just tested and it worked for me. > >>> > >>>Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > >>> > >>Yes it does, it's the IP of the LAN in the clound, and it all works > >>fine with 18.2. > >> > >>I believe it's possible that the issue is caused by the > >>configuration in the operating system (Ubuntu 14.04.4 x86_64) but > >>currently I have no idea how to further debug this issue. > >> > >>With esl-erlang 18.2 and the following configs in bashrc: > >> > >>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" > >>export ERL_EPMD_PORT="5678" > >> > >>epmd works fine: > >> > >>tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd > >>tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd > >> > >> > >>Thanks > >>Khitai > >> > >>>>Everything worked fine with the 18.2 package. After upgrading > >>>to esl-erlang 18.3, epmd stopped to work: > >>>>>epmd -debug > >>>>epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > >>>>epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot > >>>assign requested address > >>>>There is a workaround: if I remove the ERL_EPMD_ADDRESS > >>>environment variable, epms works: > >>>>tcp 0 0 0.0.0.0:5678 0.0.0.0:* > >>>LISTEN 3328/epmd > >>>>tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > >>>> > >>>>After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > >Have you disabled IPv6? > > > > # disable IPv6 support on the loopback device > > $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1 > > > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > > epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address > > > > # enable IPv6 support on the loopback device > > $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 > > > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > > > > > >If we look at what is happening: > > > > $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd > > ... > > bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address) > > ... > > > >When you specify a specific address to bind, epmd will also bind to the > >IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has > >disabled it on the loopback device so the bind fails: > > > > $ ifconfig lo > > lo Link encap:Local Loopback > > inet addr:127.0.0.1 Mask:255.0.0.0 > > UP LOOPBACK RUNNING MTU:65536 Metric:1 > > RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > > > >So either enable IPv6 for the loopback or disable IPv6 completely by > >passing "ipv6.disable=1" at boot. > > Michael, > > Thanks for the workaround. 2 questions: > > 1) I have these in my sysctl: > > net.ipv6.conf.all.disable_ipv6 = 1 > net.ipv6.conf.default.disable_ipv6 = 1 > net.ipv6.conf.eth0.disable_ipv6 = 1 > net.ipv6.conf.eth1.disable_ipv6 = 1 > net.ipv6.conf.lo.disable_ipv6 = 1 > > Aren't these enough to tell epmd that ipv6 is not available on loopback? We can see from your first email that allocating an IPv6 socket and binding it to the IPv6 ANY address works: tcp 0 0 0.0.0.0:5678 0.0.0.0:* LISTEN 3328/epmd tcp6 0 0 :::5678 :::* LISTEN 3328/epmd A system with a working IPv6 implementation must have an IPv6 loopback. So I would say your system configuration is broken. If the intent was to disable IPv6, it didn't work: you have a half-working system. I do think epmd should require explicitly specifying all addresses to bind instead of implicitly binding the loopback. So in your case you would need to add the IPv4 loopback but it would otherwise work as before: $ ERL_EPMD_ADDRESS="10.131.70.80,127.0.0.1" ERL_EPMD_PORT=5678 epmd > 2) Why does 18.2 work in the same environment? 18.3 supports node registration for distribution over IPv6 (OTP-13364): http://erlang.org/download/otp_src_18.3.readme > Thanks > Khitai From ok@REDACTED Mon Mar 21 00:50:43 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 21 Mar 2016 12:50:43 +1300 Subject: [erlang-questions] Erlang Syntax and "Patterns" (Again) In-Reply-To: References: <01013f22-cfa8-dcb1-6414-f2173aab022a@cs.otago.ac.nz> Message-ID: On 19/03/16 5:30 am, Emil Holmstrom wrote: > The confusion between strings and [integer()] would have been greatly > reduced if char() existed, $a wouldn't have to be syntactic sugar for > 97 but would actually be "character a". You would have to explicitly > convert char() -> integer() and wise versa. This is how strings are > implemented in ML and Haskell. Actually no. The STRING signature specifies the basic operations on a string type, which is a vector of the underlying character type char as defined in the structure. signature STRING = ... structure String :> STRING where type string = string where type string = CharVector.vector where type char = Char.char structure WideString :> STRING where type string = WideCharacterVector.vector where type char = WideChar.char From the SML Basis Library book, page 360. There are indeed Char.char and WideChar.char types which are just wrappers for integer types such that chr and ord are implementation-level identities. But characters as such are so far from being basic to strings in ML that the very syntax, #"c", tells you that they were an afterthought. For what it's worth, when I'm working on strings in SML, I suppose I *should* make use of the StringCvt.reader type but it's usually easier to use explode to convert (*change* the representation of) strings to lists and implode to convert back. As for Haskell, String does indeed exist, but the preferred representation for strings in "efficient" programs is http://hackage.haskell.org/package/text-1.2.2.1/docs/Data-Text.html (It's interesting to note that a Text value in Haskell is NOT an arbitrary sequence of suitably bounded integral values; a sequence of bounded integral values yes, arbitrary no.) I completely agree that encodings should be handled at the edges of a system, *for data that the system will accept/generate*. From on-your-mark@REDACTED Mon Mar 21 07:00:35 2016 From: on-your-mark@REDACTED (YuanZhiqian) Date: Mon, 21 Mar 2016 14:00:35 +0800 Subject: [erlang-questions] gen_server:cast from escript In-Reply-To: References: , Message-ID: Hi Park, Thanks a lot. That works! I wonder, however, why it is needed to connect to target node manually in an escript, while in a shell I can directly execute gen_server:cast to another target node. RegardsZhiqian Date: Thu, 10 Mar 2016 12:35:21 +0900 Subject: Re: [erlang-questions] gen_server:cast from escript From: jinni.park@REDACTED To: on-your-mark@REDACTED CC: erlang-questions@REDACTED You didn't connect to target node.Have to call net_adm:ping(Node) first to connect. On Tue, Mar 8, 2016 at 3:47 PM, YuanZhiqian wrote: Hi guys, I was trying to call gen_server:cast from a escript file, everything that used to work well in erl shell won't work in escript, I have no idea what made the difference, here is the code, could anyone help me to find a clue? #!/usr/bin/env escript %%! -name bk@REDACTED -setcookie budget_keeper main(Argv) -> {Node, File} = case Argv of [] -> {'budget_keeper@REDACTED', "cache"}; [F] -> {'budget_keeper@REDACTED', F}; [N, F] -> {N, F} end, io:format("~p ~p ~p ~p~n", [node(), erlang:get_cookie(), Node, File]), gen_server:cast({bk_main, Node}, {dump_data, File}). As shown above, the target process is called "bk_main" which is on the node 'budget_keeper@REDACTED', I have run the same code in erl shell, and bk_main can get the notice, but nothing happened when I ran the same code in this script. Best regardsZhiqian _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Park, Sungjin-------------------------------------------------------------------------------------------------------------------Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Mon Mar 21 10:51:51 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 17:51:51 +0800 Subject: [erlang-questions] Problems stop/attach a release Message-ID: Hi, I build my release by erlang.mk and relx. I can start the release by running: $ _rel/myapp/bin/myapp start It starts well and listens properly, I tried to use my client app to connect to it, all functions well, and I can start other nodes, and distribution works just fine. Problem 1: I can't stop it by running $ _rel/myapp/bin/myapp stop after running the command, it's still running (the processes are still there). Problem 2: I can't start an attaching shell, when I run $ _rel/myapp/bin/myapp attach I get "Node is not running!" Any idea? Thanks Khitai From khitai.pang@REDACTED Mon Mar 21 10:57:45 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 17:57:45 +0800 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: <20160320212759.GB29707@brk> References: <20160319225325.GA29707@brk> <20160320212759.GB29707@brk> Message-ID: On 2016/3/21 5:27, Michael Santos wrote: > On Mon, Mar 21, 2016 at 02:23:05AM +0800, Khitai Pang wrote: >> On 2016/3/20 6:53, Michael Santos wrote: >>> On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: >>>> On 2016/3/19 19:45, Michael Santos wrote: >>>>> On Mar 18, 2016 10:46 PM, "Khitai Pang" >>>> > wrote: >>>>>> Hi, >>>>>> >>>>>> I have been using the esl-erlang pacakge from >>>>> https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for >>>>> Trusty. >>>>>> I have these in my bashrc: >>>>>> >>>>>> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in >>>>> the cloud) >>>>>> export ERL_EPMD_PORT="5678" >>>>> Just tested and it worked for me. >>>>> >>>>> Does the IP address in ERL_EPMD_ADDRESS exist in the VM? >>>>> >>>> Yes it does, it's the IP of the LAN in the clound, and it all works >>>> fine with 18.2. >>>> >>>> I believe it's possible that the issue is caused by the >>>> configuration in the operating system (Ubuntu 14.04.4 x86_64) but >>>> currently I have no idea how to further debug this issue. >>>> >>>> With esl-erlang 18.2 and the following configs in bashrc: >>>> >>>> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" >>>> export ERL_EPMD_PORT="5678" >>>> >>>> epmd works fine: >>>> >>>> tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd >>>> tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd >>>> >>>> >>>> Thanks >>>> Khitai >>>> >>>>>> Everything worked fine with the 18.2 package. After upgrading >>>>> to esl-erlang 18.3, epmd stopped to work: >>>>>>> epmd -debug >>>>>> epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 >>>>>> epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot >>>>> assign requested address >>>>>> There is a workaround: if I remove the ERL_EPMD_ADDRESS >>>>> environment variable, epms works: >>>>>> tcp 0 0 0.0.0.0:5678 0.0.0.0:* >>>>> LISTEN 3328/epmd >>>>>> tcp6 0 0 :::5678 :::* LISTEN 3328/epmd >>>>>> After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. >>> Have you disabled IPv6? >>> >>> # disable IPv6 support on the loopback device >>> $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1 >>> >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd >>> epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address >>> >>> # enable IPv6 support on the loopback device >>> $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 >>> >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd >>> >>> >>> If we look at what is happening: >>> >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd >>> ... >>> bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address) >>> ... >>> >>> When you specify a specific address to bind, epmd will also bind to the >>> IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has >>> disabled it on the loopback device so the bind fails: >>> >>> $ ifconfig lo >>> lo Link encap:Local Loopback >>> inet addr:127.0.0.1 Mask:255.0.0.0 >>> UP LOOPBACK RUNNING MTU:65536 Metric:1 >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> collisions:0 txqueuelen:0 >>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >>> >>> So either enable IPv6 for the loopback or disable IPv6 completely by >>> passing "ipv6.disable=1" at boot. >> Michael, >> >> Thanks for the workaround. 2 questions: >> >> 1) I have these in my sysctl: >> >> net.ipv6.conf.all.disable_ipv6 = 1 >> net.ipv6.conf.default.disable_ipv6 = 1 >> net.ipv6.conf.eth0.disable_ipv6 = 1 >> net.ipv6.conf.eth1.disable_ipv6 = 1 >> net.ipv6.conf.lo.disable_ipv6 = 1 >> >> Aren't these enough to tell epmd that ipv6 is not available on loopback? > We can see from your first email that allocating an IPv6 socket and > binding it to the IPv6 ANY address works: > > tcp 0 0 0.0.0.0:5678 0.0.0.0:* LISTEN 3328/epmd > tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > > A system with a working IPv6 implementation must have an IPv6 loopback. > So I would say your system configuration is broken. If the intent was to > disable IPv6, it didn't work: you have a half-working system. Agreed. > I do think epmd should require explicitly specifying all addresses to > bind instead of implicitly binding the loopback. So in your case you > would need to add the IPv4 loopback but it would otherwise work as > before: > > $ ERL_EPMD_ADDRESS="10.131.70.80,127.0.0.1" ERL_EPMD_PORT=5678 epmd So I should either disable IPv6 completely or enable IPv6 on loopback? >> 2) Why does 18.2 work in the same environment? > 18.3 supports node registration for distribution over IPv6 (OTP-13364): > > http://erlang.org/download/otp_src_18.3.readme I see, thanks for the information. Thanks Khitai From khitai.pang@REDACTED Mon Mar 21 11:08:21 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 18:08:21 +0800 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: I found these in my _rel/myapp/log/erlang.log.1: =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M ** Connection attempt from disallowed node 'myapp_maint_45928@REDACTED' ** ^M ^M =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M ** Connection attempt from disallowed node 'myapp_maint_45928@REDACTED' ** ^M ^M =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M ** Connection attempt from disallowed node 'myapp_maint_46033@REDACTED' ** ^M ^M =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M ** Connection attempt from disallowed node 'myapp_maint_46033@REDACTED' ** ^M Is it the reason why I can't stop or attach a console? If yes, how to fix it? I was guessing that maybe the issue is caused by erlang cookies in ~/.erlang.cookie and my release vm.config -setcookie being different, I made them the same the the issue still exists. Thanks Khitai On 2016/3/21 17:51, Khitai Pang wrote: > Hi, > > I build my release by erlang.mk and relx. > > I can start the release by running: > > $ _rel/myapp/bin/myapp start > > It starts well and listens properly, I tried to use my client app to > connect to it, all functions well, and I can start other nodes, and > distribution works just fine. > > > Problem 1: > > I can't stop it by running > > $ _rel/myapp/bin/myapp stop > > after running the command, it's still running (the processes are still > there). > > Problem 2: > > I can't start an attaching shell, when I run > > $ _rel/myapp/bin/myapp attach > > I get "Node is not running!" > > > Any idea? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From khitai.pang@REDACTED Mon Mar 21 11:38:22 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 18:38:22 +0800 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: I found the reason, in my vm.args, I have -setcookie 'abcde#12345' There is a # character, and when starting the release, 'abcde' is used for erlang cookie. Thanks Khitai On 2016/3/21 18:08, Khitai Pang wrote: > I found these in my _rel/myapp/log/erlang.log.1: > > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_45928@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_45928@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_46033@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_46033@REDACTED' ** ^M > > > Is it the reason why I can't stop or attach a console? If yes, how to > fix it? > > I was guessing that maybe the issue is caused by erlang cookies in > ~/.erlang.cookie and my release vm.config -setcookie being different, > I made them the same the the issue still exists. > > > Thanks > Khitai > > On 2016/3/21 17:51, Khitai Pang wrote: >> Hi, >> >> I build my release by erlang.mk and relx. >> >> I can start the release by running: >> >> $ _rel/myapp/bin/myapp start >> >> It starts well and listens properly, I tried to use my client app to >> connect to it, all functions well, and I can start other nodes, and >> distribution works just fine. >> >> >> Problem 1: >> >> I can't stop it by running >> >> $ _rel/myapp/bin/myapp stop >> >> after running the command, it's still running (the processes are >> still there). >> >> Problem 2: >> >> I can't start an attaching shell, when I run >> >> $ _rel/myapp/bin/myapp attach >> >> I get "Node is not running!" >> >> >> Any idea? >> >> >> Thanks >> Khitai >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From khitai.pang@REDACTED Mon Mar 21 12:08:49 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 19:08:49 +0800 Subject: [erlang-questions] enif_send: env==NULL on non-SMP VM[End] Message-ID: Hi, I have a cloud vm configured with only 1 cpu core. My app starts fine but when it starts processing client requests it dies with: enif_send: env==NULL on non-SMP VM[End] Is there a workaround for this? I'm using the esl-erlang package for Ubuntu trusty from erlang-solutions.com. Thanks Khitai From pierrefenoll@REDACTED Mon Mar 21 12:36:38 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 21 Mar 2016 12:36:38 +0100 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: We had something similar and it was related to the cookie. Are you sure there are no erlang:set_cookie/2 calls in your code? Cheers, -- Pierre Fenoll On 21 March 2016 at 11:08, Khitai Pang wrote: > I found these in my _rel/myapp/log/erlang.log.1: > > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node ' > myapp_maint_45928@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node ' > myapp_maint_45928@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node ' > myapp_maint_46033@REDACTED' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node ' > myapp_maint_46033@REDACTED' ** ^M > > > Is it the reason why I can't stop or attach a console? If yes, how to fix > it? > > I was guessing that maybe the issue is caused by erlang cookies in > ~/.erlang.cookie and my release vm.config -setcookie being different, I > made them the same the the issue still exists. > > > Thanks > Khitai > > > On 2016/3/21 17:51, Khitai Pang wrote: > >> Hi, >> >> I build my release by erlang.mk and relx. >> >> I can start the release by running: >> >> $ _rel/myapp/bin/myapp start >> >> It starts well and listens properly, I tried to use my client app to >> connect to it, all functions well, and I can start other nodes, and >> distribution works just fine. >> >> >> Problem 1: >> >> I can't stop it by running >> >> $ _rel/myapp/bin/myapp stop >> >> after running the command, it's still running (the processes are still >> there). >> >> Problem 2: >> >> I can't start an attaching shell, when I run >> >> $ _rel/myapp/bin/myapp attach >> >> I get "Node is not running!" >> >> >> Any idea? >> >> >> Thanks >> Khitai >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Mon Mar 21 12:40:35 2016 From: tony@REDACTED (Tony Rogvall) Date: Mon, 21 Mar 2016 12:40:35 +0100 Subject: [erlang-questions] enif_send: env==NULL on non-SMP VM[End] In-Reply-To: References: Message-ID: <5423339E-F9A8-4E4E-9711-A55B30940992@rogvall.se> Start erlang with erl -smp enable The alternatives are a bit harder. /Tony > On 21 mar 2016, at 12:08, Khitai Pang wrote: > > Hi, > > I have a cloud vm configured with only 1 cpu core. My app starts fine but when it starts processing client requests it dies with: > > > enif_send: env==NULL on non-SMP VM[End] > > > Is there a workaround for this? I'm using the esl-erlang package for Ubuntu trusty from erlang-solutions.com. > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From michael.santos@REDACTED Mon Mar 21 12:46:04 2016 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 21 Mar 2016 07:46:04 -0400 Subject: [erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3? In-Reply-To: References: <20160319225325.GA29707@brk> <20160320212759.GB29707@brk> Message-ID: <20160321114604.GA7630@brk> On Mon, Mar 21, 2016 at 05:57:45PM +0800, Khitai Pang wrote: > On 2016/3/21 5:27, Michael Santos wrote: > >On Mon, Mar 21, 2016 at 02:23:05AM +0800, Khitai Pang wrote: > >>On 2016/3/20 6:53, Michael Santos wrote: > >>>On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote: > >>>>On 2016/3/19 19:45, Michael Santos wrote: > >>>>>On Mar 18, 2016 10:46 PM, "Khitai Pang" >>>>>> wrote: > >>>>>>Hi, > >>>>>> > >>>>>>I have been using the esl-erlang pacakge from > >>>>>https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for > >>>>>Trusty. > >>>>>>I have these in my bashrc: > >>>>>> > >>>>>>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" (LAN IP of my VM in > >>>>>the cloud) > >>>>>>export ERL_EPMD_PORT="5678" > >>>>>Just tested and it worked for me. > >>>>> > >>>>>Does the IP address in ERL_EPMD_ADDRESS exist in the VM? > >>>>> > >>>>Yes it does, it's the IP of the LAN in the clound, and it all works > >>>>fine with 18.2. > >>>> > >>>>I believe it's possible that the issue is caused by the > >>>>configuration in the operating system (Ubuntu 14.04.4 x86_64) but > >>>>currently I have no idea how to further debug this issue. > >>>> > >>>>With esl-erlang 18.2 and the following configs in bashrc: > >>>> > >>>>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX" > >>>>export ERL_EPMD_PORT="5678" > >>>> > >>>>epmd works fine: > >>>> > >>>>tcp 0 0 127.0.0.1:5678 0.0.0.0:* LISTEN 6754/epmd > >>>>tcp 0 0 10.131.70.80:5678 0.0.0.0:* LISTEN 6754/epmd > >>>> > >>>> > >>>>Thanks > >>>>Khitai > >>>> > >>>>>>Everything worked fine with the 18.2 package. After upgrading > >>>>>to esl-erlang 18.3, epmd stopped to work: > >>>>>>>epmd -debug > >>>>>>epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0 > >>>>>>epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot > >>>>>assign requested address > >>>>>>There is a workaround: if I remove the ERL_EPMD_ADDRESS > >>>>>environment variable, epms works: > >>>>>>tcp 0 0 0.0.0.0:5678 0.0.0.0:* > >>>>>LISTEN 3328/epmd > >>>>>>tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > >>>>>>After downgrading esl-erlang from 18.3 to 18.2, the issue is gone. > >>>Have you disabled IPv6? > >>> > >>> # disable IPv6 support on the loopback device > >>> $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1 > >>> > >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > >>> epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address > >>> > >>> # enable IPv6 support on the loopback device > >>> $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 > >>> > >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd > >>> > >>> > >>>If we look at what is happening: > >>> > >>> $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd > >>> ... > >>> bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address) > >>> ... > >>> > >>>When you specify a specific address to bind, epmd will also bind to the > >>>IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has > >>>disabled it on the loopback device so the bind fails: > >>> > >>> $ ifconfig lo > >>> lo Link encap:Local Loopback > >>> inet addr:127.0.0.1 Mask:255.0.0.0 > >>> UP LOOPBACK RUNNING MTU:65536 Metric:1 > >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 > >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 > >>> collisions:0 txqueuelen:0 > >>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) > >>> > >>>So either enable IPv6 for the loopback or disable IPv6 completely by > >>>passing "ipv6.disable=1" at boot. > >>Michael, > >> > >>Thanks for the workaround. 2 questions: > >> > >>1) I have these in my sysctl: > >> > >>net.ipv6.conf.all.disable_ipv6 = 1 > >>net.ipv6.conf.default.disable_ipv6 = 1 > >>net.ipv6.conf.eth0.disable_ipv6 = 1 > >>net.ipv6.conf.eth1.disable_ipv6 = 1 > >>net.ipv6.conf.lo.disable_ipv6 = 1 > >> > >>Aren't these enough to tell epmd that ipv6 is not available on loopback? > >We can see from your first email that allocating an IPv6 socket and > >binding it to the IPv6 ANY address works: > > > >tcp 0 0 0.0.0.0:5678 0.0.0.0:* LISTEN 3328/epmd > >tcp6 0 0 :::5678 :::* LISTEN 3328/epmd > > > >A system with a working IPv6 implementation must have an IPv6 loopback. > >So I would say your system configuration is broken. If the intent was to > >disable IPv6, it didn't work: you have a half-working system. > > Agreed. > > >I do think epmd should require explicitly specifying all addresses to > >bind instead of implicitly binding the loopback. So in your case you > >would need to add the IPv4 loopback but it would otherwise work as > >before: > > > > $ ERL_EPMD_ADDRESS="10.131.70.80,127.0.0.1" ERL_EPMD_PORT=5678 epmd > > So I should either disable IPv6 completely or enable IPv6 on loopback? Yes, that's correct: $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0 Or: # /etc/default/grub # update-grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash ipv6.disable=1" > >>2) Why does 18.2 work in the same environment? > >18.3 supports node registration for distribution over IPv6 (OTP-13364): > > > >http://erlang.org/download/otp_src_18.3.readme > > I see, thanks for the information. > > > Thanks > Khitai From michael_489@REDACTED Mon Mar 21 12:36:59 2016 From: michael_489@REDACTED (Michael S) Date: Mon, 21 Mar 2016 12:36:59 +0100 Subject: [erlang-questions] Read out the operating system language Message-ID: Hello together, is there any possibility to read out the language of the operating system (linux, windows, mac etc.). I can?t find any way to get this information. Thanks Michael From jesper.louis.andersen@REDACTED Mon Mar 21 13:32:16 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 21 Mar 2016 13:32:16 +0100 Subject: [erlang-questions] Read out the operating system language In-Reply-To: References: Message-ID: On Mon, Mar 21, 2016 at 12:36 PM, Michael S wrote: > is there any possibility to read out the language of the operating system > (linux, windows, mac etc.). os:type() or erlang:system_info(os_type). -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael_489@REDACTED Mon Mar 21 14:23:41 2016 From: michael_489@REDACTED (Michael S) Date: Mon, 21 Mar 2016 14:23:41 +0100 Subject: [erlang-questions] Read out the operating system language In-Reply-To: References: Message-ID: Not the type ;) I mean the language english, german, spanish, french etc. Am 21.03.16 um 13:32 schrieb Jesper Louis Andersen: > > On Mon, Mar 21, 2016 at 12:36 PM, Michael S > wrote: > > is there any possibility to read out the language of the operating system (linux, windows, mac etc.). > > > os:type() or erlang:system_info(os_type). > > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Mon Mar 21 14:37:06 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 21:37:06 +0800 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: No, I have no erlang:set_cookie/2 in my code. Now the app gets cookie from vm.args (-setcookie). I do have another problem though, if I remove -setcookie from the vm.args and rebuild the release, the built release can't start. I don't understand why. Thanks Khitai On 2016/3/21 19:36, Pierre Fenoll wrote: > We had something similar and it was related to the cookie. > Are you sure there are no erlang:set_cookie/2 calls in your code? > > > > Cheers, > -- > Pierre Fenoll > > > On 21 March 2016 at 11:08, Khitai Pang > wrote: > > I found these in my _rel/myapp/log/erlang.log.1: > > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_45928@REDACTED > ' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:46:55 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_45928@REDACTED > ' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_46033@REDACTED > ' ** ^M > ^M > =ERROR REPORT==== 21-Mar-2016::17:50:03 ===^M > ** Connection attempt from disallowed node > 'myapp_maint_46033@REDACTED > ' ** ^M > > > Is it the reason why I can't stop or attach a console? If yes, > how to fix it? > > I was guessing that maybe the issue is caused by erlang cookies in > ~/.erlang.cookie and my release vm.config -setcookie being > different, I made them the same the the issue still exists. > > > Thanks > Khitai > > > On 2016/3/21 17:51, Khitai Pang wrote: > > Hi, > > I build my release by erlang.mk and relx. > > I can start the release by running: > > $ _rel/myapp/bin/myapp start > > It starts well and listens properly, I tried to use my client > app to connect to it, all functions well, and I can start > other nodes, and distribution works just fine. > > > Problem 1: > > I can't stop it by running > > $ _rel/myapp/bin/myapp stop > > after running the command, it's still running (the processes > are still there). > > Problem 2: > > I can't start an attaching shell, when I run > > $ _rel/myapp/bin/myapp attach > > I get "Node is not running!" > > > Any idea? > > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Mar 21 14:42:51 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 21 Mar 2016 09:42:51 -0400 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: <20160321134250.GQ31558@fhebert-ltm2.internal.salesforce.com> On 03/21, Khitai Pang wrote: >No, I have no erlang:set_cookie/2 in my code. Now the app gets cookie >from vm.args (-setcookie). > >I do have another problem though, if I remove -setcookie from the >vm.args and rebuild the release, the built release can't start. I >don't understand why. > Relx uses nodetools, and nodetools seems to want a cookie real bad to avoid different apps communicating each other accidentally. If the issue isn't related to cookies, it may be due to the app hierarchy itself. In some cases, applications do not get proper dependency lists (which app depends on which apps), or omit to include apps like 'stdlib' and 'kernel'. In such cases, what may happen is that when shutting down, the Erlang node tears down 'kernel' or 'stdlib' before some other running application. Then the node is stuck in that mode where it is technically still running, but no functions may be called and it just doesn't ever shut down. When you get to that point, forcefully killing the node is the only thing I know how to make work. The sad part is that any single dependency you include in your release may be to blame for this. No idea if this is specifically your problem, but it's one I've seen before. From jacob01@REDACTED Mon Mar 21 15:06:12 2016 From: jacob01@REDACTED (Jacob) Date: Mon, 21 Mar 2016 15:06:12 +0100 Subject: [erlang-questions] Read out the operating system language In-Reply-To: References: Message-ID: <56EFFFD4.8080807@gmx.net> Hi, On 21.03.2016 14:23, Michael S wrote: > Not the type ;) > I mean the language english, german, spanish, french etc. This is quite platform dependent. At least with Linux (more precisely on systems supporting GNU gettext) you can use something like os:getenv("LANGUAGE", os:getenv("LC_ALL", os:getenv("LANG", "C"))). which should do it in most cases (the format of the value may depend on which environment variable is used). "C" means 'disable localization', use the application's default language in that case. This is gettext specific (see info gettext, 'Locale Environment Variables' and 'Locale Names' for details). There might also be something helpful in the wxLocale module, but I didn't check that. HTH Jacob > > > Am 21.03.16 um 13:32 schrieb Jesper Louis Andersen: >> >> On Mon, Mar 21, 2016 at 12:36 PM, Michael S > > wrote: >> >> is there any possibility to read out the language of the operating >> system (linux, windows, mac etc.). >> >> >> os:type() or erlang:system_info(os_type). From khitai.pang@REDACTED Mon Mar 21 15:39:13 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 21 Mar 2016 22:39:13 +0800 Subject: [erlang-questions] enif_send: env==NULL on non-SMP VM[End] In-Reply-To: <5423339E-F9A8-4E4E-9711-A55B30940992@rogvall.se> References: <5423339E-F9A8-4E4E-9711-A55B30940992@rogvall.se> Message-ID: Hi Tony, I assume you mean "disable". I added the following to my vm.args: -smp disable and rebuilt, reran the release, but the issue is still there.. Thanks Khitai On 2016/3/21 19:40, Tony Rogvall wrote: > Start erlang with > > erl -smp enable > > The alternatives are a bit harder. > > /Tony > > >> On 21 mar 2016, at 12:08, Khitai Pang wrote: >> >> Hi, >> >> I have a cloud vm configured with only 1 cpu core. My app starts fine but when it starts processing client requests it dies with: >> >> >> enif_send: env==NULL on non-SMP VM[End] >> >> >> Is there a workaround for this? I'm using the esl-erlang package for Ubuntu trusty from erlang-solutions.com. >> >> >> Thanks >> Khitai >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From sverker.eriksson@REDACTED Mon Mar 21 16:02:16 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 21 Mar 2016 16:02:16 +0100 Subject: [erlang-questions] enif_send: env==NULL on non-SMP VM[End] In-Reply-To: References: <5423339E-F9A8-4E4E-9711-A55B30940992@rogvall.se> Message-ID: <56F00CF8.9030705@ericsson.com> No, he meant "enable". The message "enif_send: env==NULL on non-SMP VM" complains about a used feature that is not supported on a non-SMP VM. That is why you should *enable* smp support even though you are running on a single core (which disables smp by default). /Sverker, Erlang/OTP On 03/21/2016 03:39 PM, Khitai Pang wrote: > Hi Tony, > > I assume you mean "disable". I added the following to my vm.args: > > -smp disable > > and rebuilt, reran the release, but the issue is still there.. > > > Thanks > Khitai > > On 2016/3/21 19:40, Tony Rogvall wrote: >> Start erlang with >> >> erl -smp enable >> >> The alternatives are a bit harder. >> >> /Tony >> >> >>> On 21 mar 2016, at 12:08, Khitai Pang wrote: >>> >>> Hi, >>> >>> I have a cloud vm configured with only 1 cpu core. My app starts >>> fine but when it starts processing client requests it dies with: >>> >>> >>> enif_send: env==NULL on non-SMP VM[End] >>> >>> >>> Is there a workaround for this? I'm using the esl-erlang package >>> for Ubuntu trusty from erlang-solutions.com. >>> >>> >>> Thanks >>> Khitai >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From khitai.pang@REDACTED Mon Mar 21 17:05:21 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 22 Mar 2016 00:05:21 +0800 Subject: [erlang-questions] enif_send: env==NULL on non-SMP VM[End] In-Reply-To: <56F00CF8.9030705@ericsson.com> References: <5423339E-F9A8-4E4E-9711-A55B30940992@rogvall.se> <56F00CF8.9030705@ericsson.com> Message-ID: Hi Tony, Sverker, I tried "-smp enable" in vm.args. It works like a charming charm. Thanks Khitai On 2016/3/21 23:02, Sverker Eriksson wrote: > No, he meant "enable". > > The message "enif_send: env==NULL on non-SMP VM" > complains about a used feature that is not supported on a non-SMP VM. > > That is why you should *enable* smp support even though you are running > on a single core (which disables smp by default). > > > /Sverker, Erlang/OTP > > On 03/21/2016 03:39 PM, Khitai Pang wrote: >> Hi Tony, >> >> I assume you mean "disable". I added the following to my vm.args: >> >> -smp disable >> >> and rebuilt, reran the release, but the issue is still there.. >> >> >> Thanks >> Khitai >> >> On 2016/3/21 19:40, Tony Rogvall wrote: >>> Start erlang with >>> >>> erl -smp enable >>> >>> The alternatives are a bit harder. >>> >>> /Tony >>> >>> >>>> On 21 mar 2016, at 12:08, Khitai Pang wrote: >>>> >>>> Hi, >>>> >>>> I have a cloud vm configured with only 1 cpu core. My app starts >>>> fine but when it starts processing client requests it dies with: >>>> >>>> >>>> enif_send: env==NULL on non-SMP VM[End] >>>> >>>> >>>> Is there a workaround for this? I'm using the esl-erlang package >>>> for Ubuntu trusty from erlang-solutions.com. >>>> >>>> >>>> Thanks >>>> Khitai >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From khitai.pang@REDACTED Mon Mar 21 20:00:12 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 22 Mar 2016 03:00:12 +0800 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: <20160321134250.GQ31558@fhebert-ltm2.internal.salesforce.com> References: <20160321134250.GQ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On 2016/3/21 21:42, Fred Hebert wrote: > On 03/21, Khitai Pang wrote: >> No, I have no erlang:set_cookie/2 in my code. Now the app gets >> cookie from vm.args (-setcookie). >> >> I do have another problem though, if I remove -setcookie from the >> vm.args and rebuild the release, the built release can't start. I >> don't understand why. >> > Relx uses nodetools, and nodetools seems to want a cookie real bad to > avoid different apps communicating each other accidentally. > > If the issue isn't related to cookies, it may be due to the app > hierarchy itself. > > In some cases, applications do not get proper dependency lists (which > app depends on which apps), or omit to include apps like 'stdlib' and > 'kernel'. > > In such cases, what may happen is that when shutting down, the Erlang > node tears down 'kernel' or 'stdlib' before some other running > application. Then the node is stuck in that mode where it is > technically still running, but no functions may be called and it just > doesn't ever shut down. When you get to that point, forcefully killing > the node is the only thing I know how to make work. > > The sad part is that any single dependency you include in your release > may be to blame for this. > > No idea if this is specifically your problem, but it's one I've seen > before. I tried building (erlang.mk calling relx) the release with and without -setcookie in vm.args, and diff the result: diff -Nurp _rel.good/myapp/releases/1/myapp.script _rel.bad/myapp/releases/1/myapp.script --- _rel.good/myapp/releases/1/myapp.script 2016-03-22 02:53:10.486533796 +++ _rel.bad/myapp/releases/1/myapp.script 2016-03-22 02:54:04.983822959 @@ -1,4 +1,4 @@ -%% script generated at {2016,3,22} {2,53,10} +%% script generated at {2016,3,22} {2,54,4} {script, {"myapp","1"}, [{preLoaded, diff -Nurp _rel.good/myapp/releases/1/vm.args _rel.bad/myapp/releases/1/vm.args --- _rel.good/myapp/releases/1/vm.args 2016-03-22 02:52:57.000000000 +++ _rel.bad/myapp/releases/1/vm.args 2016-03-22 02:54:01.000000000 @@ -2,6 +2,6 @@ -name myapp ## Cookie for distributed erlang --setcookie f3IfFpeiI6n0JNhR6cGP +#-setcookie f3IfFpeiI6n0JNhR6cGP -smp enable The only difference is the vm.args file. It seems that the script _rel/myapp/bin/myapp just won't work if there's no -setcookie in vm.args. Thanks Khitai From roger@REDACTED Mon Mar 21 20:21:09 2016 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 21 Mar 2016 19:21:09 +0000 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: <20160321134250.GQ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On 21 March 2016 at 19:00, Khitai Pang wrote: > The only difference is the vm.args file. It seems that the script > _rel/myapp/bin/myapp just won't work if there's no -setcookie in vm.args. Building an {extended_start_script,true} script with relx version 1.0.4 (I've not tried anything newer) gives you a script that explicitly checks for -setcookie in vm.args: # Extract the target cookie COOKIE_ARG="$(grep '^-setcookie' "$VMARGS_PATH")" if [ -z "$COOKIE_ARG" ]; then echo "vm.args needs to have a -setcookie parameter." exit 1 fi # Extract cookie name from COOKIE_ARG COOKIE="$(echo "$COOKIE_ARG" | awk '{print $2}')" That reports the problem and quits. Maybe the non-extended start script has the same requirement, but doesn't check it explicitly? Or maybe there's a regression between 1.0.4 and whatever version of relx you're using? From sds6065@REDACTED Mon Mar 21 19:09:20 2016 From: sds6065@REDACTED (Sam Schneider) Date: Mon, 21 Mar 2016 14:09:20 -0400 Subject: [erlang-questions] Win32 API NIF(s) Message-ID: Hi everyone, Does anybody know of any nifs for working with the Win32 C API? I've tried searching around without any luck so far. In the meantime I've been playing with Roslyn open in a port, but a can't imagine that's very ideal. Thanks, Sam -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Mon Mar 21 21:22:51 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Mon, 21 Mar 2016 13:22:51 -0700 Subject: [erlang-questions] Win32 API NIF(s) In-Reply-To: References: Message-ID: the Win32 C API is pretty big; what in specific are you looking to try to do, and what sort of requirements envelope does it need to fit inside? generically, I'd think F# or C# open in a port would be close enough for a lot of use cases. F. On Mon, Mar 21, 2016 at 11:09 AM, Sam Schneider wrote: > Hi everyone, > > Does anybody know of any nifs for working with the Win32 C API? I've > tried searching around without any luck so far. In the meantime I've been > playing with Roslyn open in a port, but a can't imagine that's very ideal. > > Thanks, > Sam > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Mon Mar 21 21:44:35 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Mon, 21 Mar 2016 13:44:35 -0700 Subject: [erlang-questions] Win32 API NIF(s) In-Reply-To: References: Message-ID: Damn, that's an excellent candidate for a NIF. Don't know of one off hand, but that's small enough, self-contained enough, and benefits the most from not having another entire process hanging around that it makes a good amount of sense. The closest I could imagine would be to look at the Wings3D modeler, which is an erlang app that runs on Windows as a first class citizen. I don't know if it sets the shell icon, but it might, and if it does, that's probably the closest you're going to get. https://github.com/dgud/wings F. On Mon, Mar 21, 2016 at 1:34 PM, Sam Schneider wrote: > Specifically I'm looking to use the notification icon APIs (like > Shell_NotifyIcon) > On Mar 21, 2016 4:23 PM, "Felix Gallo" wrote: > >> the Win32 C API is pretty big; what in specific are you looking to try to >> do, and what sort of requirements envelope does it need to fit inside? >> >> generically, I'd think F# or C# open in a port would be close enough for >> a lot of use cases. >> >> F. >> >> On Mon, Mar 21, 2016 at 11:09 AM, Sam Schneider >> wrote: >> >>> Hi everyone, >>> >>> Does anybody know of any nifs for working with the Win32 C API? I've >>> tried searching around without any luck so far. In the meantime I've been >>> playing with Roslyn open in a port, but a can't imagine that's very ideal. >>> >>> Thanks, >>> Sam >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sds6065@REDACTED Mon Mar 21 21:34:19 2016 From: sds6065@REDACTED (Sam Schneider) Date: Mon, 21 Mar 2016 16:34:19 -0400 Subject: [erlang-questions] Win32 API NIF(s) In-Reply-To: References: Message-ID: Specifically I'm looking to use the notification icon APIs (like Shell_NotifyIcon) On Mar 21, 2016 4:23 PM, "Felix Gallo" wrote: > the Win32 C API is pretty big; what in specific are you looking to try to > do, and what sort of requirements envelope does it need to fit inside? > > generically, I'd think F# or C# open in a port would be close enough for a > lot of use cases. > > F. > > On Mon, Mar 21, 2016 at 11:09 AM, Sam Schneider wrote: > >> Hi everyone, >> >> Does anybody know of any nifs for working with the Win32 C API? I've >> tried searching around without any luck so far. In the meantime I've been >> playing with Roslyn open in a port, but a can't imagine that's very ideal. >> >> Thanks, >> Sam >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sds6065@REDACTED Mon Mar 21 21:51:49 2016 From: sds6065@REDACTED (Sam Schneider) Date: Mon, 21 Mar 2016 16:51:49 -0400 Subject: [erlang-questions] Win32 API NIF(s) In-Reply-To: References: Message-ID: Thanks for the link, I'll definitely check it out! If I can't end up finding anything I may just create one myself since yeah I could see it being useful for others. On Mar 21, 2016 4:44 PM, "Felix Gallo" wrote: > Damn, that's an excellent candidate for a NIF. Don't know of one off > hand, but that's small enough, self-contained enough, and benefits the most > from not having another entire process hanging around that it makes a good > amount of sense. > > The closest I could imagine would be to look at the Wings3D modeler, which > is an erlang app that runs on Windows as a first class citizen. I don't > know if it sets the shell icon, but it might, and if it does, that's > probably the closest you're going to get. > > https://github.com/dgud/wings > > F. > > On Mon, Mar 21, 2016 at 1:34 PM, Sam Schneider wrote: > >> Specifically I'm looking to use the notification icon APIs (like >> Shell_NotifyIcon) >> On Mar 21, 2016 4:23 PM, "Felix Gallo" wrote: >> >>> the Win32 C API is pretty big; what in specific are you looking to try >>> to do, and what sort of requirements envelope does it need to fit inside? >>> >>> generically, I'd think F# or C# open in a port would be close enough for >>> a lot of use cases. >>> >>> F. >>> >>> On Mon, Mar 21, 2016 at 11:09 AM, Sam Schneider >>> wrote: >>> >>>> Hi everyone, >>>> >>>> Does anybody know of any nifs for working with the Win32 C API? I've >>>> tried searching around without any luck so far. In the meantime I've been >>>> playing with Roslyn open in a port, but a can't imagine that's very ideal. >>>> >>>> Thanks, >>>> Sam >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dharalanov@REDACTED Mon Mar 21 22:35:04 2016 From: dharalanov@REDACTED (Dimitar Haralanov) Date: Mon, 21 Mar 2016 23:35:04 +0200 Subject: [erlang-questions] A Question About Processes Message-ID: Hello everyone! I am new to Erlang, but I?ve been really enjoying almost every bit of it. I stumbled upon something I am not exactly sure about. So far, my understanding is that processes are created by spawn()-ing them, however I think this is not the complete truth. I have the following contrived example: -module(foo). -export([bar/0]). bar() -> io:format("Hello from ~p~n", [self()]), spawn(fun loop/0). loop() -> receive Message -> io:format("PID ~p received ~p~n", [self(), Message]), loop() end. and after running the code we get: 1> Pid = foo:bar(). Hello from <0.33.0> <0.35.0> 2> Pid ! hello. PID <0.35.0> received hello hello 3> My question is - how is the process with PID <0.33.0> spawned? I?ve only read about using the erlang:spawn() function call, but I guess there are other implicit ways of starting new processes. Any further clarification would be really helpful. Thank you! Kind regards, Dimitar From felixgallo@REDACTED Mon Mar 21 23:16:06 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Mon, 21 Mar 2016 15:16:06 -0700 Subject: [erlang-questions] A Question About Processes In-Reply-To: References: Message-ID: <0.33.0> was spawned when you started the interactive shell. self() is the pid of the process which is performing the function call, which in this case is the interactive shell itself. You can test this: Eshell V7.2.1 (abort with ^G) 1> c(foo),l(foo),foo:bar(). Hello from <0.34.0> <0.40.0> 2> self(). <0.34.0> 3> On Mon, Mar 21, 2016 at 2:35 PM, Dimitar Haralanov wrote: > Hello everyone! > > I am new to Erlang, but I?ve been really enjoying almost every bit of it. > I stumbled upon > something I am not exactly sure about. So far, my understanding is that > processes are > created by spawn()-ing them, however I think this is not the complete > truth. I have the > following contrived example: > > -module(foo). > -export([bar/0]). > > bar() -> > io:format("Hello from ~p~n", [self()]), > spawn(fun loop/0). > > loop() -> > receive > Message -> > io:format("PID ~p received ~p~n", [self(), Message]), > loop() > end. > > and after running the code we get: > > 1> Pid = foo:bar(). > Hello from <0.33.0> > <0.35.0> > 2> Pid ! hello. > PID <0.35.0> received hello > hello > 3> > > My question is - how is the process with PID <0.33.0> spawned? I?ve only > read about > using the erlang:spawn() function call, but I guess there are other > implicit ways of starting > new processes. Any further clarification would be really helpful. Thank > you! > > Kind regards, > Dimitar > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Mon Mar 21 23:18:54 2016 From: hugo@REDACTED (Hugo Mills) Date: Mon, 21 Mar 2016 22:18:54 +0000 Subject: [erlang-questions] A Question About Processes In-Reply-To: References: Message-ID: <20160321221854.GA29474@carfax.org.uk> On Mon, Mar 21, 2016 at 11:35:04PM +0200, Dimitar Haralanov wrote: > Hello everyone! > > I am new to Erlang, but I?ve been really enjoying almost every bit of it. I stumbled upon > something I am not exactly sure about. So far, my understanding is that processes are > created by spawn()-ing them, however I think this is not the complete truth. I have the > following contrived example: > > -module(foo). > -export([bar/0]). > > bar() -> > io:format("Hello from ~p~n", [self()]), > spawn(fun loop/0). > > loop() -> > receive > Message -> > io:format("PID ~p received ~p~n", [self(), Message]), > loop() > end. > > and after running the code we get: > > 1> Pid = foo:bar(). > Hello from <0.33.0> > <0.35.0> > 2> Pid ! hello. > PID <0.35.0> received hello > hello > 3> > > My question is - how is the process with PID <0.33.0> spawned? I?ve only read about > using the erlang:spawn() function call, but I guess there are other implicit ways of starting > new processes. Any further clarification would be really helpful. Thank you! When you call foo:bar(), that's running in the context of the process that is running the shell. When you run it, it prints the pid of the shell (0.33.0), and then spawns a process running the loop function. That then prints its own pid (0.35.0) and the hello message. I think the thing you're missing is that the erlang shell is a process in its own right, and that's the first pid you see in your example. Hugo. -- Hugo Mills | If the first-ever performance is the premi?re, is hugo@REDACTED carfax.org.uk | the last-ever performance the derri?re? http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From kostis@REDACTED Mon Mar 21 23:19:46 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 21 Mar 2016 23:19:46 +0100 Subject: [erlang-questions] A Question About Processes In-Reply-To: References: Message-ID: <56F07382.7050702@cs.ntua.gr> On 03/21/2016 10:35 PM, Dimitar Haralanov wrote: > My question is - how is the process with PID <0.33.0> spawned? I?ve only read about > using the erlang:spawn() function call, but I guess there are other implicit ways of starting > new processes. Any further clarification would be really helpful. Thank you! This is the `shell' process: the process that runs the shell where you type your commands. In your example, rather than calling foo:bar() as the first command, issue a self() call first and you notice the printed PID and the PID that you get when you subsequently call foo:bar(). From antoine.koener@REDACTED Tue Mar 22 00:10:25 2016 From: antoine.koener@REDACTED (Antoine Koener) Date: Tue, 22 Mar 2016 00:10:25 +0100 Subject: [erlang-questions] Win32 API NIF(s) In-Reply-To: References: Message-ID: <4E30C0D2-A2B9-49B2-8576-4415B7CD15BC@gmail.com> > Le 21 mars 2016 ? 21:51, Sam Schneider a ?crit : > > Thanks for the link, I'll definitely check it out! If I can't end up finding anything I may just create one myself since yeah I could see it being useful for others. > > On Mar 21, 2016 4:44 PM, "Felix Gallo" > wrote: > Damn, that's an excellent candidate for a NIF. Don't know of one off hand, but that's small enough, self-contained enough, and benefits the most from not having another entire process hanging around that it makes a good amount of sense. > > The closest I could imagine would be to look at the Wings3D modeler, which is an erlang app that runs on Windows as a first class citizen. I don't know if it sets the shell icon, but it might, and if it does, that's probably the closest you're going to get. > > https://github.com/dgud/wings I've implemented in the past a small .exe that was just doing that (but also inject a dll in all processes but that's another story) that I was managing using a port. Because the exe was supervised it could crash, erlang restart it, and I was able to make that small .exe a lot more powerful... That was really simple; the only thing I remember was something related to UTF-16 instead of C strings. There might also be some way to use the rundll.exe to call what you need; but I don't know if that would work nowadays... > > F. > > On Mon, Mar 21, 2016 at 1:34 PM, Sam Schneider > wrote: > Specifically I'm looking to use the notification icon APIs (like Shell_NotifyIcon) > > On Mar 21, 2016 4:23 PM, "Felix Gallo" > wrote: > the Win32 C API is pretty big; what in specific are you looking to try to do, and what sort of requirements envelope does it need to fit inside? > > generically, I'd think F# or C# open in a port would be close enough for a lot of use cases. > > F. > > On Mon, Mar 21, 2016 at 11:09 AM, Sam Schneider > wrote: > Hi everyone, > > Does anybody know of any nifs for working with the Win32 C API? I've tried searching around without any luck so far. In the meantime I've been playing with Roslyn open in a port, but a can't imagine that's very ideal. > > Thanks, > Sam > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Mar 22 00:37:49 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 22 Mar 2016 12:37:49 +1300 Subject: [erlang-questions] Read out the operating system language In-Reply-To: References: Message-ID: <94a09fc9-8723-10bb-2ba2-654ee133721d@cs.otago.ac.nz> On 22/03/16 2:23 am, Michael S wrote: > Not the type ;) > I mean the language english, german, spanish, french etc. The operating system doesn't *have* a language. In the C / UNIX world, "language" is broken up into facets: $LANG - ultimate default $LC_ALL - interim default $LC_COLLATE - how to compare strings $LC_CTYPE - how to classify characters $LC_MESSAGE - how to display error/warning/info messages $LC_MONETARY - how to display amounts of money $LC_NUMERIC - how to display numbers $LC_TIME - how to display dates and times When you ask about a particular facet, if that facet is not set, LC_ALL will be tried, and if that is not set, LANG will be tried. Locales include C - the ASCII-ish default from the C standard POSIX - the ASCII-ish default from the POSIX standard [_][.][@] The locale names are less standardised than one would like. The format is *typically* as shown, where the is a 2-letter ISO language code if the language has one, a 3-letter ISO code if it doesn't, and dear knows what if it hasn't even one of those. That's usually in lower case. The is a 2-letter ISO country code if it has one, a 3-letter ISO code if it doesn't, and whatever otherwise. That's usually in upper case. The could be UTF-8, GBK, KOI8-U, ISO8859-n, &c. You might think it's upper case, but you might get BIG5 or Big5, EUC-JP or eucJP, and so on. The is used for things like @euro, @bokmal, @nynorsk As a rough suggestion, try $LC_MESSAGES, $LC_ALL, $LANG in that order until one of them starts with a bunch of letters and is not C or POSIX. language() -> case check_language("LC_MESSAGES") of {yes,Lang} -> Lang ; no -> case check_language("LC_ALL") of {yes,Lang} -> Lang ; no -> case check_language("LANG") of {yes,Lang} -> Lang end end end. check_language(Source) -> case os:getenv(Source) of false -> no ; Chars when is_list(Chars) -> check_language(Chars, []) end. check_language([Char|Chars], Acc) when Char >= $a, Char =< $z -> check_language(Chars, [Char|Acc]); check_language([Char|Chars], Acc) when Char >= $A, Char =< $Z -> check_language(Chars, [Char+32|Acc]); check_language([], "c") -> no; check_language([], "xisop") -> no; check_language([_|_], []) -> no; check_language(_, Acc) -> {yes, lists:reverse(Acc)}. Much depends on what you want this for. If you are going to use it for reporting messages, you really want to pay attention to the *whole* of $LC_MESSAGES, lest you display semi- comprehensible text with risible spelling and grammar errors, as will happen if you generate en_US text for an en_NZ user. One reason for @ variants is that countries sometimes go through spelling reforms, although from what a German student a couple of years ago told me, not always successfully. Point is, you might need a date or revision name as well as language and territory. From technion@REDACTED Tue Mar 22 01:54:12 2016 From: technion@REDACTED (Technion) Date: Tue, 22 Mar 2016 00:54:12 +0000 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: Message-ID: Hi, Are you by chance setting: env ERL_EPMD_ADDRESS 127.0.0.1 ? I thought this was a good idea for security if the application is running on a single node, but with rebar3, it just generates the exact issue you describe below. ________________________________________ From: erlang-questions-bounces@REDACTED on behalf of Khitai Pang Sent: Monday, 21 March 2016 8:51 PM To: Erlang Subject: [erlang-questions] Problems stop/attach a release Hi, I build my release by erlang.mk and relx. I can start the release by running: $ _rel/myapp/bin/myapp start It starts well and listens properly, I tried to use my client app to connect to it, all functions well, and I can start other nodes, and distribution works just fine. Problem 1: I can't stop it by running $ _rel/myapp/bin/myapp stop after running the command, it's still running (the processes are still there). Problem 2: I can't start an attaching shell, when I run $ _rel/myapp/bin/myapp attach I get "Node is not running!" Any idea? Thanks Khitai _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From steve.morin@REDACTED Tue Mar 22 03:26:23 2016 From: steve.morin@REDACTED (Steve Morin) Date: Mon, 21 Mar 2016 19:26:23 -0700 Subject: [erlang-questions] Looking to hire Erlang tutor - Help debugging in IntelliJ-Erlang of Basho_bench Message-ID: Erlang Community, I am looking for an Erlang Tutor to help get me up to speed, I am the CTO of a Big Data consultancy and need a jump start from someone much more experienced in Erlang than myself. Looking to do a screen share session once to a few times depending on how productive it is. If you don't have the time or want a few extra dollars for the help any recommendation to someone who might just want to help or make a few dollars would be greatly appreciated. Goal: I am trying to setup basho_bench ( https://github.com/basho/basho_bench ) in IntelliJ-Erlang which is an imported rebar project. I have gotten basic things working and all setup including getting "hello world" projects working but having trouble getting this working. I am trying to get basho_bench working so I can understand the workings better between the workers, sup, and stats/measurement modules of the application because I am looking to modify the code for my own use. Also been told using observer might be helpful, http://erlang.org/doc/man/observer.html Also looking for help setting things up and helping connect a few dots on a few issues. Background: I am an experienced developer but new overall pretty new to erlang though I have ready V1 and V2 of Armstrong's book. Still very much an Erlang noobie Thanks -Steve -- *Steve Morin | Managing Partner - CTO* *Nvent* O 800-407-1156 ext 803 <800-407-1156;803> | M 347-453-5579 smorin@REDACTED *Enabling the Data Driven Enterprise* *(Ask us how we can setup scalable open source realtime billion+ event/data collection/analytics infrastructure in weeks)* Service Areas: Management & Strategy Consulting | Data Engineering | Data Science & Visualization -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Tue Mar 22 04:06:16 2016 From: me@REDACTED (Matthew Shapiro) Date: Mon, 21 Mar 2016 23:06:16 -0400 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture Message-ID: I am trying to learn Erlang in order to create an application that seems to check off all the boxes of the Beam VM (low latency requirements, extremely concurrent, long lasting connections making hot code swapping desirable for upgrades, etc...). In order to learn I've gone through most of Learn You Some Erlang For Great Good, and so far have really been enjoying the process. I am now at the point where I want to test some of this knowledge out, and I thought a good idea was to create a (basic) IRC server (as I've written them in the past in other languages, and it seemed like a good use case for Erlang). Basic idea is to support allowing users to connect, set a nickname, connect to channels, send messages to channels, and send messages to users. However, I am a bit confused on the proper way to architect an OTP application and hoping that my current idea of how to architect this is correct or not. Also, I find rubber ducking helps :) The high level supervision tree I have floating in my head is to have: * application ** app_supervisor *** tcp_listener *** handshake_supervisor **** handshake servers (dynamic) *** user_supervisor **** user servers (dynamic) *** channel_supervisor **** channel servers (dynamic) My thinking is the tcp_listener, handshake, user, and channel modules would each be gen_servers, and the 4 supervisors would be standard otp supervisors. The app_supervisor would have a one_for_one spec while user_supervisor, chandshake_supervisor and channel supervisor would have simple_one_for_one spec since they can spin children up and down dynamically based on users connected and channels with at least one user in it. The flow I have in my head is: 1) tcp_listener is a server that waits for connections, accepts them and tells the handshake_supervisor to spawn a new handshake server for the accepted socket (and changes ownership of the socket to the handshake server. 2) The handshake server handles the IRC handshake, and when the user requests a nickname it asks the user_supervisor if any children exist with the id matching the requested nickname. 3) If the nickname is available, the handshake server tells the user_supervisor to start a new user server with the accepted socket and nickname, sets the user server as the socket owner, and then the handshake server terminates 4) When the user server is started, the user_supervisor would set its id of the child spec to be the user's nickname. 5) When the user joins a channel, the user module sends the accepted socket and the users nickname to the channel_supervisor and asks it to send it to the correct channel. The channel_supervisor will look for a child process with the id matching the name of the channel, and if none exists it will create a new child with the channel name as the id in the child spec. It will then send the child process with the matching id/name a message with the users nickname. 6) When the user leaves a channel it will call on the channel_supervisor to send a message to the child process with a matching id/name with the nickname of the user leaving the channel (or disconnecting). 7) The channel server will keep a state with a list of connected users, and when a join, leave, or channel message is received it will modify the client list as needed. 8) When required (user joins, user leaves, someone says something, etc....) The channel server will send a message to a specific user by calling the user_supervisor to send a string to a child process with an id matching the nickname of the user, and send that child process an erlang message with the irc message to send to the user. The user server will then take that, translate it into the IRC protocol and send it to the socket. 9) On client disconnection, the user server will announce to the channel supervisor that it has disconnected, and the channel supervisor will tell all children it no longer is connected to the channel. So the problem I have on this architecture is it requires a lot of logic in the supervisors (which I am not sure is a good or bad thing). It also seems to invite myself to race conditions due to all the supervisor:which_children() calls that would be required to identify the correct child process to send a message to, since theoretically a child process could die between a which_children() and actually sending the message. It also means going through several layers of supervisors in order to find the process another process needs to communicate with (for example, user server would have to ask the user_supervisor to ask the app_supervisor what process the channel_supervisor is, which would then have to ask the channel_supervisor what process the correct channel server is). This seems hacky but I am unsure of how to work around that. Am I on the correct path with this? Is there an easier way to facilitate servers talking to other servers managed by other supervisors? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Tue Mar 22 05:49:56 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 22 Mar 2016 12:49:56 +0800 Subject: [erlang-questions] Problems stop/attach a release In-Reply-To: References: <20160321134250.GQ31558@fhebert-ltm2.internal.salesforce.com> Message-ID: On 2016/3/22 3:21, Roger Lipscombe wrote: > On 21 March 2016 at 19:00, Khitai Pang wrote: >> The only difference is the vm.args file. It seems that the script >> _rel/myapp/bin/myapp just won't work if there's no -setcookie in vm.args. > Building an {extended_start_script,true} script with relx version > 1.0.4 (I've not tried anything newer) gives you a script that > explicitly checks for -setcookie in vm.args: > > # Extract the target cookie > COOKIE_ARG="$(grep '^-setcookie' "$VMARGS_PATH")" > if [ -z "$COOKIE_ARG" ]; then > echo "vm.args needs to have a -setcookie parameter." > exit 1 > fi > > # Extract cookie name from COOKIE_ARG > COOKIE="$(echo "$COOKIE_ARG" | awk '{print $2}')" > > That reports the problem and quits. Maybe the non-extended start > script has the same requirement, but doesn't check it explicitly? Or > maybe there's a regression between 1.0.4 and whatever version of relx > you're using? It seems my relx builds happily without -setcookie in vm.args. It;s automatically downloaded by erlang.mk: > ls -l relx -rwxrwxr-x 1 khitai khitai 166941 Aug 25 2015 relx > md5sum relx db0839594ab8edaa91567a033a98cc69 relx > file relx relx: data > ./relx --version 3.5.0 Thanks Khitai From Oliver.Korpilla@REDACTED Tue Mar 22 09:48:45 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Tue, 22 Mar 2016 09:48:45 +0100 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: From dharalanov@REDACTED Tue Mar 22 10:27:07 2016 From: dharalanov@REDACTED (Dimitar Haralanov) Date: Tue, 22 Mar 2016 11:27:07 +0200 Subject: [erlang-questions] A Question About Processes In-Reply-To: <56F07382.7050702@cs.ntua.gr> References: <56F07382.7050702@cs.ntua.gr> Message-ID: <61E99436-41DE-4BB9-A2E6-C83908734D83@me.com> Yes, it makes sense now - it?s the PID of the shell itself. Thank you all for your time and help! ? Dimitar > On Mar 22, 2016, at 12:19 AM, Kostis Sagonas wrote: > > On 03/21/2016 10:35 PM, Dimitar Haralanov wrote: >> My question is - how is the process with PID <0.33.0> spawned? I?ve only read about >> using the erlang:spawn() function call, but I guess there are other implicit ways of starting >> new processes. Any further clarification would be really helpful. Thank you! > > This is the `shell' process: the process that runs the shell where you type your commands. In your example, rather than calling foo:bar() as the first command, issue a self() call first and you notice the printed PID and the PID that you get when you subsequently call foo:bar(). > From me@REDACTED Tue Mar 22 14:11:13 2016 From: me@REDACTED (Matthew Shapiro) Date: Tue, 22 Mar 2016 09:11:13 -0400 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: Neat, that definitely makes things less complicated to talk to processes managed by another supervisor! On Tue, Mar 22, 2016 at 4:48 AM, Oliver Korpilla wrote: > Hello, Matthew. > > I'm a beginner, too. I just perked my ears when you wrote about the > "which_children()" call... > > The gen_xyz() modules all follow a similar startup logic that allow you to > register the child under an ID. The way you see used in "Learn you some > Erlang good" is the easiest one, the local registration ({local, }) > where must be an atom. You can, however, use other registries - like > "global". Google "man erlang global" for more details about that registry. > There an arbitary expression is good enough as a name. > > This is hugely advantageous for many applications, including your own, as > you can use something like {, } or {, } as > process name. With global:whereis_name you can ask for the needed PID to > send messages to. It returns undefined if it doesn't exist. So, instead of > asking the supervisor about the children, you just call whereis_name with > the computed ID you would be looking for, like {user, "DanWahlberg69" } and > if you get something else but undefined, you can directly talk to the > worker you are looking for. This should keep your supervisors simple. > > All you need is a scheme to generate unique names and you are set. > > Hope this helps, > Oliver > > *Gesendet:* Dienstag, 22. M?rz 2016 um 04:06 Uhr > *Von:* "Matthew Shapiro" > *An:* erlang-questions@REDACTED > *Betreff:* [erlang-questions] Beginner trying to figure out idiomatic OTP > architecture > I am trying to learn Erlang in order to create an application that seems > to check off all the boxes of the Beam VM (low latency requirements, > extremely concurrent, long lasting connections making hot code swapping > desirable for upgrades, etc...). In order to learn I've gone through most > of Learn You Some Erlang For Great Good, and so far have really been > enjoying the process. > > I am now at the point where I want to test some of this knowledge out, and > I thought a good idea was to create a (basic) IRC server (as I've written > them in the past in other languages, and it seemed like a good use case for > Erlang). Basic idea is to support allowing users to connect, set a > nickname, connect to channels, send messages to channels, and send messages > to users. > > However, I am a bit confused on the proper way to architect an OTP > application and hoping that my current idea of how to architect this is > correct or not. Also, I find rubber ducking helps :) > > The high level supervision tree I have floating in my head is to have: > > * application > ** app_supervisor > *** tcp_listener > *** handshake_supervisor > **** handshake servers (dynamic) > *** user_supervisor > **** user servers (dynamic) > *** channel_supervisor > **** channel servers (dynamic) > > My thinking is the tcp_listener, handshake, user, and channel modules > would each be gen_servers, and the 4 supervisors would be standard otp > supervisors. The app_supervisor would have a one_for_one spec while > user_supervisor, chandshake_supervisor and channel supervisor would have > simple_one_for_one spec since they can spin children up and down > dynamically based on users connected and channels with at least one user in > it. > > The flow I have in my head is: > > 1) tcp_listener is a server that waits for connections, accepts them and > tells the handshake_supervisor to spawn a new handshake server for the > accepted socket (and changes ownership of the socket to the handshake > server. > 2) The handshake server handles the IRC handshake, and when the user > requests a nickname it asks the user_supervisor if any children exist with > the id matching the requested nickname. > 3) If the nickname is available, the handshake server tells the > user_supervisor to start a new user server with the accepted socket and > nickname, sets the user server as the socket owner, and then the handshake > server terminates > 4) When the user server is started, the user_supervisor would set its id > of the child spec to be the user's nickname. > 5) When the user joins a channel, the user module sends the accepted > socket and the users nickname to the channel_supervisor and asks it to send > it to the correct channel. The channel_supervisor will look for a child > process with the id matching the name of the channel, and if none exists it > will create a new child with the channel name as the id in the child spec. > It will then send the child process with the matching id/name a message > with the users nickname. > 6) When the user leaves a channel it will call on the channel_supervisor > to send a message to the child process with a matching id/name with the > nickname of the user leaving the channel (or disconnecting). > 7) The channel server will keep a state with a list of connected users, > and when a join, leave, or channel message is received it will modify the > client list as needed. > 8) When required (user joins, user leaves, someone says something, > etc....) The channel server will send a message to a specific user by > calling the user_supervisor to send a string to a child process with an id > matching the nickname of the user, and send that child process an erlang > message with the irc message to send to the user. The user server will > then take that, translate it into the IRC protocol and send it to the > socket. > 9) On client disconnection, the user server will announce to the channel > supervisor that it has disconnected, and the channel supervisor will tell > all children it no longer is connected to the channel. > > > So the problem I have on this architecture is it requires a lot of logic > in the supervisors (which I am not sure is a good or bad thing). It also > seems to invite myself to race conditions due to all the > supervisor:which_children() calls that would be required to identify the > correct child process to send a message to, since theoretically a child > process could die between a which_children() and actually sending the > message. It also means going through several layers of supervisors in > order to find the process another process needs to communicate with (for > example, user server would have to ask the user_supervisor to ask the > app_supervisor what process the channel_supervisor is, which would then > have to ask the channel_supervisor what process the correct channel server > is). This seems hacky but I am unsure of how to work around that. > > Am I on the correct path with this? Is there an easier way to facilitate > servers talking to other servers managed by other supervisors? > > Thanks. > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liuz@REDACTED Wed Mar 23 04:51:48 2016 From: liuz@REDACTED (LiuZhen) Date: Wed, 23 Mar 2016 11:51:48 +0800 Subject: [erlang-questions] Way of using CRDT? Message-ID: Hi All, Recently I was trying to set up a prototype app with riak-core. It all went smooth until ran into CRDT. I have read: papers about CRDT from HAL `Conflict-free Replicated Data Types? at https://hal.inria.fr/file/index/docid/609399/filename/RR-7687.pdf `A Comprehensive Study of Convergent and Commutative Replicated Data Types? at https://hal.archives-ouvertes.fr/inria-00555588/ `Version Vectors are not Vector Clocks? at https://haslab.wordpress.com/2011/07/08/version-vectors-are-not-vector-clocks/ `Dotted Version Vector Sets' at https://github.com/ricardobcl/Dotted-Version-Vectors which by the way is used in riak-core for meta Project-FiFo?s fifo_dt at https://github.com/project-fifo/fifo_dt Then which following understandings are wrong? Version vector is for state-based crdt; vector clock is for op-based crdt >From #1, riak and similars use state-based CRDT, then it should use version vector rather than vector clocks Dotted version vector is more compact than plain version vector The reason fifo_dt and similars are using vclocks is because basicly vclock?s static structure is the same as version vector, but since the update rule is not the same, why it is used in fifo_dt? To explain #4, the object combining value & vclocks can be thought as a way of state-based crdt without using version vector What is the *most updated & correct* way of doing this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Mar 23 13:47:05 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 23 Mar 2016 13:47:05 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: On the subject on additional reasons to vendor dependencies: http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ BTW, not saying this can happen with hex.pm. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Mar 23 13:56:00 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 23 Mar 2016 13:56:00 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: <56F29260.9020300@ninenines.eu> Of course this can happen with hex.pm. :-) https://hex.pm/docs/codeofconduct Data published to Hex is hosted at the discretion of the Hex team, and may be removed. It can also happen to github, gitlab, bitbucket, and any other repository of code that allows removal. On 03/23/2016 01:47 PM, Roberto Ostinelli wrote: > On the subject on additional reasons to vendor dependencies: > http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ > > BTW, not saying this can happen with hex.pm . > > Best, > r. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roberto@REDACTED Wed Mar 23 14:00:38 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 23 Mar 2016 14:00:38 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56F29260.9020300@ninenines.eu> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> <56F29260.9020300@ninenines.eu> Message-ID: On Wed, Mar 23, 2016 at 1:56 PM, Lo?c Hoguin wrote: > Of course this can happen with hex.pm. :-) > > https://hex.pm/docs/codeofconduct > > Data published to Hex is hosted at the discretion > of the Hex team, and may be removed. > > It can also happen to github, gitlab, bitbucket, and any other repository > of code that allows removal. Indeed, but let me be more less cryptic on what I was referring to: what I find more dangerous in this npm story is that: "[...] the global names used by the removed packages are available for anyone to register and replace with any code they wish.The fact that this is possible with NPM seems really dangerous. The author unpublished (erm, "liberated") over 250 NPM modules, making those global names (e.g. "map", "alert", "iframe", "subscription", etc) available for anyone to register and replace with any code they wish. Since these libs are now baked into various package.json configuration files (some with 10s of thousands of installs per month, "left-pad" with 2.5M/month), meaning a malicious actor could publish a new patch version bump (for every major and minor version combination) of these libs and ship whatever they want to future npm builds." [1]. I just don't know if hex.pm does some checksum of code, which would impeded for this to happen. [1] https://news.ycombinator.com/item?id=11341006 -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Mar 23 14:17:10 2016 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 23 Mar 2016 14:17:10 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: On Wed, Mar 23, 2016 at 1:47 PM, Roberto Ostinelli wrote: > On the subject on additional reasons to vendor dependencies: > http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ This had to happen. :-) I just woke up from a long sleep and submitted a feature request to rebar3 for sha-locking the packages from hex.pm[1]. I know there are difficulties to make this happen, but, in the light of recent events, maybe enough people will appreciate checksums of their dependencies to make this a reality? :-) Fred? Tristan? Regards, Motiejus From essen@REDACTED Wed Mar 23 14:19:51 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 23 Mar 2016 14:19:51 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> <56F29260.9020300@ninenines.eu> Message-ID: <56F297F7.10203@ninenines.eu> On 03/23/2016 02:00 PM, Roberto Ostinelli wrote: > On Wed, Mar 23, 2016 at 1:56 PM, Lo?c Hoguin > wrote: > > Of course this can happen with hex.pm . :-) > > https://hex.pm/docs/codeofconduct > > Data published to Hex is hosted at the discretion > of the Hex team, and may be removed. > > It can also happen to github, gitlab, bitbucket, and any other > repository of code that allows removal. > > > Indeed, but let me be more less cryptic on what I was referring to: what > I find more dangerous in this npm story is that: > > "[...] the global names used by the removed packages are available for > anyone to register and replace with any code they wish.The fact that > this is possible with NPM seems really dangerous. The author unpublished > (erm, "liberated") over 250 NPM modules, making those global names (e.g. > "map", "alert", "iframe", "subscription", etc) available for anyone to > register and replace with any code they wish. Since these libs are now > baked into various package.json configuration files (some with 10s of > thousands of installs per month, "left-pad" with 2.5M/month), meaning a > malicious actor could publish a new patch version bump (for every major > and minor version combination) of these libs and ship whatever they want > to future npm builds." [1]. > > I just don't know if hex.pm does some checksum of code, > which would impeded for this to happen. Don't know about hex, but this particular problem doesn't exist when you refer to git commits directly. Just saying. :-) -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roberto@REDACTED Wed Mar 23 14:26:08 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 23 Mar 2016 14:26:08 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: <56F297F7.10203@ninenines.eu> References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> <56F29260.9020300@ninenines.eu> <56F297F7.10203@ninenines.eu> Message-ID: On Wed, Mar 23, 2016 at 2:19 PM, Lo?c Hoguin wrote: > > > Don't know about hex, but this particular problem doesn't exist when you > refer to git commits directly. > > Just saying. :-) Yes, but in hex dependencies you can just add a dependency by name :) Fortunately enough AFAIK rebar.lock sets the sha, and rebar with hex only allows for exact versions specifiers (i.e. *no* expressions like '~>1.1'). -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrallen1@REDACTED Wed Mar 23 15:04:56 2016 From: mrallen1@REDACTED (Mark Allen) Date: Wed, 23 Mar 2016 14:04:56 +0000 (UTC) Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: <1793871008.4086593.1458741896486.JavaMail.yahoo@mail.yahoo.com> One thing to be wary of is that in a distributed Erlang environment, using global process identifiers introduces significant latency compared to a local identity because the global module uses locks across all Erlang nodes to achieve consensus on the identities that have been assigned both at process start up time and at identity to PID lookup time. ? Most Erlang applications "in the wild" avoid global when at all possible. On Tuesday, March 22, 2016 8:42 AM, Matthew Shapiro wrote: Neat, that definitely makes things less complicated to talk to processes managed by another supervisor! On Tue, Mar 22, 2016 at 4:48 AM, Oliver Korpilla wrote: Hello, Matthew.?I'm a beginner, too. I just perked my ears when you wrote about the "which_children()" call...?The gen_xyz() modules all follow a similar startup logic that allow you to register the child under an ID. The way you see used in "Learn you some Erlang good" is the easiest one, the local registration ({local, }) where must be an atom. You can, however, use other registries - like "global". Google "man erlang global" for more details about that registry. There an arbitary expression is good enough as a name.?This is hugely advantageous for many applications, including your own, as you can use something like {, } or {, } as process name. With global:whereis_name you can ask for the needed PID to send messages to. It returns undefined if it doesn't exist. So, instead of asking the supervisor about the children, you just call whereis_name with the computed ID you would be looking for, like {user, "DanWahlberg69" } and if you get something else but undefined, you can directly talk to the worker you are looking for. This should keep your supervisors simple.?All you need is a scheme to generate unique names and you are set.??Hope this helps,Oliver?Gesendet:?Dienstag, 22. M?rz 2016 um 04:06 Uhr Von:?"Matthew Shapiro" An:?erlang-questions@REDACTED Betreff:?[erlang-questions] Beginner trying to figure out idiomatic OTP architectureI am trying to learn Erlang in order to create an application that seems to check off all the boxes of the Beam VM (low latency requirements, extremely concurrent, long lasting connections making hot code swapping desirable for upgrades, etc...).? In order to learn I've gone through most of Learn You Some Erlang For Great Good, and so far have really been enjoying the process.?I am now at the point where I want to test some of this knowledge out, and I thought a good idea was to create a (basic) IRC server (as I've written them in the past in other languages, and it seemed like a good use case for Erlang).? Basic idea is to support allowing users to connect, set a nickname, connect to channels, send messages to channels, and send messages to users.?However, I am a bit confused on the proper way to architect an OTP application and hoping that my current idea of how to architect this is correct or not.? Also, I find rubber ducking helps :)?The high level supervision tree I have floating in my head is to have:?* application** app_supervisor*** tcp_listener*** handshake_supervisor**** handshake servers (dynamic)*** user_supervisor**** user servers (dynamic)*** channel_supervisor**** channel servers (dynamic)?My thinking is the tcp_listener, handshake, user, and channel modules would each be gen_servers, and the 4 supervisors would be standard otp supervisors. The app_supervisor would have a one_for_one spec while user_supervisor, chandshake_supervisor and channel supervisor would have simple_one_for_one spec since they can spin children up and down dynamically based on users connected and channels with at least one user in it.?The flow I have in my head is:?1) tcp_listener is a server that waits for connections, accepts them and tells the handshake_supervisor to spawn a new handshake server for the accepted socket (and changes ownership of the socket to the handshake server.2) The handshake server handles the IRC handshake, and when the user requests a nickname it asks the user_supervisor if any children exist with the id matching the requested nickname.?3) If the nickname is available, the handshake server tells the user_supervisor to start a new user server with the accepted socket and nickname, sets the user server as the socket owner, and then the handshake server terminates4) When the user server is started, the user_supervisor would set its id of the child spec to be the user's nickname.5) When the user joins a channel, the user module sends the accepted socket and the users nickname to the channel_supervisor and asks it to send it to the correct channel.? The channel_supervisor will look for a child process with the id matching the name of the channel, and if none exists it will create a new child with the channel name as the id in the child spec.? It will then send the child process with the matching id/name a message with the users nickname.6) When the user leaves a channel it will call on the channel_supervisor to send a message to the child process with a matching id/name with the nickname of the user leaving the channel (or disconnecting).7) The channel server will keep a state with a list of connected users, and when a join, leave, or channel message is received it will modify the client list as needed. ?8) When required (user joins, user leaves, someone says something, etc....) The channel server will send a message to a specific user by calling the user_supervisor to send a string to a child process with an id matching the nickname of the user, and send that child process an erlang message with the irc message to send to the user.? The user server will then take that, translate it into the IRC protocol and send it to the socket.9) On client disconnection, the user server will announce to the channel supervisor that it has disconnected, and the channel supervisor will tell all children it no longer is connected to the channel.??So the problem I have on this architecture is it requires a lot of logic in the supervisors (which I am not sure is a good or bad thing).? It also seems to invite myself to race conditions due to all the supervisor:which_children() calls that would be required to identify the correct child process to send a message to, since theoretically a child process could die between a which_children() and actually sending the message.? It also means going through several layers of supervisors in order to find the process another process needs to communicate with (for example, user server would have to ask the user_supervisor to ask the app_supervisor what process the channel_supervisor is, which would then have to ask the channel_supervisor what process the correct channel server is).? This seems hacky but I am unsure of how to work around that.?Am I on the correct path with this?? Is there an easier way to facilitate servers talking to other servers managed by other supervisors??Thanks._______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Wed Mar 23 15:15:39 2016 From: t@REDACTED (Tristan Sloughter) Date: Wed, 23 Mar 2016 09:15:39 -0500 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: <1458742539.3573857.557544914.1F41DBFC@webmail.messagingengine.com> I was actually thinking of adding that feature for another reason, supporting multiple repositories. For this support we plan to simply iterate over the specified repositories to find the first match of the package. This worried some that - wasn't enough to guarantee it was really the same package, in which case we'd additionally store the hash in the lock. -- Tristan Sloughter t@REDACTED On Wed, Mar 23, 2016, at 08:17 AM, Motiejus Jak?tys wrote: > On Wed, Mar 23, 2016 at 1:47 PM, Roberto Ostinelli > wrote: > > On the subject on additional reasons to vendor dependencies: > > http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ > > This had to happen. :-) > > I just woke up from a long sleep and submitted a feature request to > rebar3 for sha-locking the packages from hex.pm[1]. > > I know there are difficulties to make this happen, but, in the light > of recent events, maybe enough people will appreciate checksums of > their dependencies to make this a reality? :-) > > Fred? Tristan? > > Regards, > Motiejus From jesper.louis.andersen@REDACTED Wed Mar 23 15:19:14 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 23 Mar 2016 15:19:14 +0100 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro wrote: > I am now at the point where I want to test some of this knowledge out, and > I thought a good idea was to create a (basic) IRC server (as I've written > them in the past in other languages, and it seemed like a good use case for > Erlang). Here is how I would do it: * There is no reason to run your own acceptor pool. Every client connecting runs behind the `ranch` acceptor pool. * The thing that happens concurrently in an IRC server are the connecting clients. There is relatively little need for a channel to act on behalf of itself, so one may look at a solution where a channel is just a list of members, handled by a general manager of channel lists in ETS. Posting a message to a channel is simply looking up interested parties, and sending the message to all of them. OTOH, having a process per channel could be helpful in order to proxy messages via the channel process: send to the chan process, and have it forward to the recipients. Which is best depends is not a priori clear to me, but when I did this years ago, I managed to do this without channel processes. * Consider managing the registry of Pids through either the `gproc` or the `syn` frameworks. This avoids you having to redo most of the nasty parts of registry and you can avoid the problems of using atoms only as in the local registry. * If you want your server to run as a cluster, you will have to think about the problem of a netsplit inside the cluster and what guarantees you want to have. This leaves your supervisor tree in one of two forms: Either the top-level supervisor runs a single channel manager process worker, or it runs a simple_one_for_one pool of channels together with a manager for creating/removing channels, if you deem it necessary to keep a process tracking each channel. In general, I would avoid solutions where you "hand off" state between processes as if they were sitting in a pipeline. It is often better to make the process itself run as an independent system. Joe said "An Erlang web server runs 2 million webservers, each serving one request." In this case, you could argue you want to run a couple thousand IRC servers, each serving one channel, and a couple thousand connection proxies, each serving one TCP connection, connecting to multiple such channels. A connection proxy then has the task of transforming the outside IRC protocol into nice symbolic Erlang terms band and forth. And the Channel servers are tasked with handling pub/sub between the processes, as well as ownership management of the channels in question. The trick is to get events/messages to flow between the connection processes such that the emergent behavior of a wild IRCd suddenly appears :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Mar 23 15:49:55 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 23 Mar 2016 10:49:55 -0400 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> <56F29260.9020300@ninenines.eu> <56F297F7.10203@ninenines.eu> Message-ID: <20160323144954.GA65360@fhebert-ltm2.internal.salesforce.com> On 03/23, Roberto Ostinelli wrote: > >Yes, but in hex dependencies you can just add a dependency by name :) >Fortunately enough AFAIK rebar.lock sets the sha, and rebar with hex only >allows for exact versions specifiers (i.e. *no* expressions like '~>1.1'). rebar.lock currently only holds the package name. We check SHAs, but against the index only. The feature for better hash checking is discussed at https://github.com/erlang/rebar3/issues/1136 right now. From me@REDACTED Thu Mar 24 03:02:46 2016 From: me@REDACTED (Matthew Shapiro) Date: Wed, 23 Mar 2016 22:02:46 -0400 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: Thank you very much, Ranch, gproc, and syn seem like perfect frameworks to handle those portions of it and simplifies a lot of things (though I need to do some research to figure out the difference between gproc and syn). So that solves those issues, well although it looks like I will have to jump through some hoops to get Ranch working on Windows. In regards to the channel per process vs not, I think my mind went to that idea due to knowing that in normal IRC servers channels have other particular aspects to them (such as titles and modes, etc...) and I guess those aspects made my mental model lean towards channels as individual processes (even though I admit those features aren't part of my requirements since this is just to get my feet wet). While I haven't gotten to clustering stuff in Erlang yet, my idea was to guarantee that if a netsplit occurs you can communicate with user in your channels that are connected to the same node as you are in. I don't know yet if that changes the architecture at all but in my mind I'm not sure if it does (channel manager/channel processes would just relay the messages to the other nodes). Anyways, some pretty interesting and enlightining things to think about On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro wrote: > >> I am now at the point where I want to test some of this knowledge out, >> and I thought a good idea was to create a (basic) IRC server (as I've >> written them in the past in other languages, and it seemed like a good use >> case for Erlang). > > > Here is how I would do it: > > * There is no reason to run your own acceptor pool. Every client > connecting runs behind the `ranch` acceptor pool. > > * The thing that happens concurrently in an IRC server are the connecting > clients. There is relatively little need for a channel to act on behalf of > itself, so one may look at a solution where a channel is just a list of > members, handled by a general manager of channel lists in ETS. Posting a > message to a channel is simply looking up interested parties, and sending > the message to all of them. OTOH, having a process per channel could be > helpful in order to proxy messages via the channel process: send to the > chan process, and have it forward to the recipients. Which is best depends > is not a priori clear to me, but when I did this years ago, I managed to do > this without channel processes. > > * Consider managing the registry of Pids through either the `gproc` or the > `syn` frameworks. This avoids you having to redo most of the nasty parts of > registry and you can avoid the problems of using atoms only as in the local > registry. > > * If you want your server to run as a cluster, you will have to think > about the problem of a netsplit inside the cluster and what guarantees you > want to have. > > This leaves your supervisor tree in one of two forms: Either the top-level > supervisor runs a single channel manager process worker, or it runs a > simple_one_for_one pool of channels together with a manager for > creating/removing channels, if you deem it necessary to keep a process > tracking each channel. > > In general, I would avoid solutions where you "hand off" state between > processes as if they were sitting in a pipeline. It is often better to make > the process itself run as an independent system. Joe said "An Erlang web > server runs 2 million webservers, each serving one request." In this case, > you could argue you want to run a couple thousand IRC servers, each serving > one channel, and a couple thousand connection proxies, each serving one TCP > connection, connecting to multiple such channels. A connection proxy then > has the task of transforming the outside IRC protocol into nice symbolic > Erlang terms band and forth. And the Channel servers are tasked with > handling pub/sub between the processes, as well as ownership management of > the channels in question. > > The trick is to get events/messages to flow between the connection > processes such that the emergent behavior of a wild IRCd suddenly appears :) > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Thu Mar 24 10:34:58 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 24 Mar 2016 12:34:58 +0300 Subject: [erlang-questions] rebar3 and ct Message-ID: Hi! I have the follow ct section in rebar.config: {ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, > all}]}. > {ct_compile_opts, [{i, "./include/"}]}. On the "rebar 3.0.0-beta.3+build.327.ref97ae8d6 on Erlang/OTP 18 Erts 7.0" it's work perfectly, but on the "rebar 3.0.0+build.273.refd2de55d on Erlang/OTP 18 Erts 7.0" it says: ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam > /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl > /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml > "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") > failed with return code 1 and the following output: > cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such > file or directory and after "rm -rf ./test/TEST*.xml" it says: ./rebar3 ct skip_deps=true verbose=3 > ===> Verifying dependencies... > ===> Member `{alias,myapp,"./test/"}' of option `ct_opts' must be a 2-tuple > make: *** [test] Error 1 How can I fix it? -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisdairsullivan@REDACTED Thu Mar 24 15:37:51 2016 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Thu, 24 Mar 2016 07:37:51 -0700 Subject: [erlang-questions] rebar3 and ct In-Reply-To: References: Message-ID: rebar3 doesn?t currently support the setting on test configuration via `ct_opts` like that. changing your `ct_opts` section to: `[{dir, ?test?}, {logdir, ?logs?}]` should work > On Mar 24, 2016, at 2:34 AM, Alexander Petrovsky wrote: > > Hi! > > I have the follow ct section in rebar.config: > > {ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, all}]}. > {ct_compile_opts, [{i, "./include/"}]}. > > On the "rebar 3.0.0-beta.3+build.327.ref97ae8d6 on Erlang/OTP 18 Erts 7.0" it's work perfectly, but on the "rebar 3.0.0+build.273.refd2de55d on Erlang/OTP 18 Erts 7.0" it says: > > ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") > failed with return code 1 and the following output: > cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such file or directory > > and after "rm -rf ./test/TEST*.xml" it says: > > ./rebar3 ct skip_deps=true verbose=3 > ===> Verifying dependencies... > ===> Member `{alias,myapp,"./test/"}' of option `ct_opts' must be a 2-tuple > make: *** [test] Error 1 > > How can I fix it? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Mar 24 15:51:24 2016 From: g@REDACTED (Garrett Smith) Date: Thu, 24 Mar 2016 14:51:24 +0000 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: Here's some detail behind syn: http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ It's also an excellent read on how to evaluate options and then, inevitably, build your own ;) Just an aside to your overall approach - I tend to blanch at architectural considerations like these as they tend to be academic, unless you *really* understand what you need to build ahead of time. And if you *really* understand what you're building (e.g. you've built something like it a few times before) then a so called architecture is implicit and you have only specific technical issues to solve. I have an admitted tendency to be glib about architecture and I realize nothing is ever as simple as I think it is :) But if you have a solid understanding of OTP application "architecture" (it's really more of configuration) you can start trail blazing without the slightest fear of building yourself into corners. The gist of OTP is to wrap, somewhat clumsily, Erlang processes in higher level interfaces. You get "process management" out of that - in a similar way you might get "memory management" out of some C/C++ framework. A very nice characteristic of the Erlang Tao is that you build small independent components (sure, call them micro services - they're closer to nano) that participate in a system. If you don't like something, you tend to add, modify or remove something small. So net splits and channels and fancy stuff like that are discrete problems in the context of your otherwise stable application. In my experience, you can defer solving these things until you're in the middle of them. And then solve them. Or try to. Then iterate because you probably not 100% happy. I.e. you don't need to get it all down before you start. If you have questions about canonical OTP - and this I think you should get correct - ask here. It's slightly under documented but e2 [1] is a solid wrapper around OTP that does get you a canonical OTP "architecture" - but with a higher signal-to-noise ratio in code. [1] http://e2project.org On Wed, Mar 23, 2016 at 11:53 PM Matthew Shapiro wrote: > Thank you very much, Ranch, gproc, and syn seem like perfect frameworks > to handle those portions of it and simplifies a lot of things (though I > need to do some research to figure out the difference between gproc and > syn). So that solves those issues, well although it looks like I will have > to jump through some hoops to get Ranch working on Windows. > > In regards to the channel per process vs not, I think my mind went to that > idea due to knowing that in normal IRC servers channels have other > particular aspects to them (such as titles and modes, etc...) and I guess > those aspects made my mental model lean towards channels as individual > processes (even though I admit those features aren't part of my > requirements since this is just to get my feet wet). > > While I haven't gotten to clustering stuff in Erlang yet, my idea was to > guarantee that if a netsplit occurs you can communicate with user in your > channels that are connected to the same node as you are in. I don't know > yet if that changes the architecture at all but in my mind I'm not sure if > it does (channel manager/channel processes would just relay the messages to > the other nodes). > > Anyways, some pretty interesting and enlightining things to think about > > > On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> >> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro wrote: >> >>> I am now at the point where I want to test some of this knowledge out, >>> and I thought a good idea was to create a (basic) IRC server (as I've >>> written them in the past in other languages, and it seemed like a good use >>> case for Erlang). >> >> >> Here is how I would do it: >> >> * There is no reason to run your own acceptor pool. Every client >> connecting runs behind the `ranch` acceptor pool. >> >> * The thing that happens concurrently in an IRC server are the connecting >> clients. There is relatively little need for a channel to act on behalf of >> itself, so one may look at a solution where a channel is just a list of >> members, handled by a general manager of channel lists in ETS. Posting a >> message to a channel is simply looking up interested parties, and sending >> the message to all of them. OTOH, having a process per channel could be >> helpful in order to proxy messages via the channel process: send to the >> chan process, and have it forward to the recipients. Which is best depends >> is not a priori clear to me, but when I did this years ago, I managed to do >> this without channel processes. >> >> * Consider managing the registry of Pids through either the `gproc` or >> the `syn` frameworks. This avoids you having to redo most of the nasty >> parts of registry and you can avoid the problems of using atoms only as in >> the local registry. >> >> * If you want your server to run as a cluster, you will have to think >> about the problem of a netsplit inside the cluster and what guarantees you >> want to have. >> >> This leaves your supervisor tree in one of two forms: Either the >> top-level supervisor runs a single channel manager process worker, or it >> runs a simple_one_for_one pool of channels together with a manager for >> creating/removing channels, if you deem it necessary to keep a process >> tracking each channel. >> >> In general, I would avoid solutions where you "hand off" state between >> processes as if they were sitting in a pipeline. It is often better to make >> the process itself run as an independent system. Joe said "An Erlang web >> server runs 2 million webservers, each serving one request." In this case, >> you could argue you want to run a couple thousand IRC servers, each serving >> one channel, and a couple thousand connection proxies, each serving one TCP >> connection, connecting to multiple such channels. A connection proxy then >> has the task of transforming the outside IRC protocol into nice symbolic >> Erlang terms band and forth. And the Channel servers are tasked with >> handling pub/sub between the processes, as well as ownership management of >> the channels in question. >> >> The trick is to get events/messages to flow between the connection >> processes such that the emergent behavior of a wild IRCd suddenly appears :) >> >> -- >> J. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Thu Mar 24 16:44:15 2016 From: me@REDACTED (Matthew Shapiro) Date: Thu, 24 Mar 2016 11:44:15 -0400 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: I can promise you putting a lot of effort at this level on the architecture definition isn't usual (not to say I don't put thought into architecture before coding, but usually not to this level of planning and discussion before coding). The main purpose of this thread is that I picked a project I've done before and using it as a launch pad on how to write Erlang applications in an idiomatic way and gather the solid understanding of OTP application architecture/configuration in the first place outside of just reading :). I understand the domain pieces of the application, I just wasn't sure if I was putting them together in a solid OTP way. I actually came across that Syn article in my googling last night, and it was a good read. That e2 framework has been bookmarked too, as it looks quite interesting and the documentation and tutorials on that site look extremely well written! I would have started the project already, just have been caught up with a lack of free time, but I can actually see how Erlang and OTP can actually make this project much easier than in the languages I've written an IRC server in before. It's actually gotten me pretty excited to code this up in a day or two (looking forward to the weekend) :). I actually think I"m going to forego ranch, Syn, and E2 for the first iteration and implement them by hand (since this is a learning exercise more than anything) and there are plenty of guides around to accomplish each of those individually. Then I can start pulling pieces out and implementing them via frameworks one by one. Thanks again! On Thu, Mar 24, 2016 at 10:51 AM, Garrett Smith wrote: > Here's some detail behind syn: > > > http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/ > > It's also an excellent read on how to evaluate options and then, > inevitably, build your own ;) > > Just an aside to your overall approach - I tend to blanch at architectural > considerations like these as they tend to be academic, unless you *really* > understand what you need to build ahead of time. And if you *really* > understand what you're building (e.g. you've built something like it a few > times before) then a so called architecture is implicit and you have only > specific technical issues to solve. > > I have an admitted tendency to be glib about architecture and I realize > nothing is ever as simple as I think it is :) But if you have a solid > understanding of OTP application "architecture" (it's really more of > configuration) you can start trail blazing without the slightest fear of > building yourself into corners. The gist of OTP is to wrap, somewhat > clumsily, Erlang processes in higher level interfaces. You get "process > management" out of that - in a similar way you might get "memory > management" out of some C/C++ framework. > > A very nice characteristic of the Erlang Tao is that you build small > independent components (sure, call them micro services - they're closer to > nano) that participate in a system. If you don't like something, you tend > to add, modify or remove something small. So net splits and channels and > fancy stuff like that are discrete problems in the context of your > otherwise stable application. In my experience, you can defer solving these > things until you're in the middle of them. And then solve them. Or try to. > Then iterate because you probably not 100% happy. > > I.e. you don't need to get it all down before you start. > > If you have questions about canonical OTP - and this I think you should > get correct - ask here. > > It's slightly under documented but e2 [1] is a solid wrapper around OTP > that does get you a canonical OTP "architecture" - but with a higher > signal-to-noise ratio in code. > > [1] http://e2project.org > > On Wed, Mar 23, 2016 at 11:53 PM Matthew Shapiro wrote: > >> Thank you very much, Ranch, gproc, and syn seem like perfect frameworks >> to handle those portions of it and simplifies a lot of things (though I >> need to do some research to figure out the difference between gproc and >> syn). So that solves those issues, well although it looks like I will have >> to jump through some hoops to get Ranch working on Windows. >> >> In regards to the channel per process vs not, I think my mind went to >> that idea due to knowing that in normal IRC servers channels have other >> particular aspects to them (such as titles and modes, etc...) and I guess >> those aspects made my mental model lean towards channels as individual >> processes (even though I admit those features aren't part of my >> requirements since this is just to get my feet wet). >> >> While I haven't gotten to clustering stuff in Erlang yet, my idea was to >> guarantee that if a netsplit occurs you can communicate with user in your >> channels that are connected to the same node as you are in. I don't know >> yet if that changes the architecture at all but in my mind I'm not sure if >> it does (channel manager/channel processes would just relay the messages to >> the other nodes). >> >> Anyways, some pretty interesting and enlightining things to think about >> >> >> On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> >>> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro >>> wrote: >>> >>>> I am now at the point where I want to test some of this knowledge out, >>>> and I thought a good idea was to create a (basic) IRC server (as I've >>>> written them in the past in other languages, and it seemed like a good use >>>> case for Erlang). >>> >>> >>> Here is how I would do it: >>> >>> * There is no reason to run your own acceptor pool. Every client >>> connecting runs behind the `ranch` acceptor pool. >>> >>> * The thing that happens concurrently in an IRC server are the >>> connecting clients. There is relatively little need for a channel to act on >>> behalf of itself, so one may look at a solution where a channel is just a >>> list of members, handled by a general manager of channel lists in ETS. >>> Posting a message to a channel is simply looking up interested parties, and >>> sending the message to all of them. OTOH, having a process per channel >>> could be helpful in order to proxy messages via the channel process: send >>> to the chan process, and have it forward to the recipients. Which is best >>> depends is not a priori clear to me, but when I did this years ago, I >>> managed to do this without channel processes. >>> >>> * Consider managing the registry of Pids through either the `gproc` or >>> the `syn` frameworks. This avoids you having to redo most of the nasty >>> parts of registry and you can avoid the problems of using atoms only as in >>> the local registry. >>> >>> * If you want your server to run as a cluster, you will have to think >>> about the problem of a netsplit inside the cluster and what guarantees you >>> want to have. >>> >>> This leaves your supervisor tree in one of two forms: Either the >>> top-level supervisor runs a single channel manager process worker, or it >>> runs a simple_one_for_one pool of channels together with a manager for >>> creating/removing channels, if you deem it necessary to keep a process >>> tracking each channel. >>> >>> In general, I would avoid solutions where you "hand off" state between >>> processes as if they were sitting in a pipeline. It is often better to make >>> the process itself run as an independent system. Joe said "An Erlang web >>> server runs 2 million webservers, each serving one request." In this case, >>> you could argue you want to run a couple thousand IRC servers, each serving >>> one channel, and a couple thousand connection proxies, each serving one TCP >>> connection, connecting to multiple such channels. A connection proxy then >>> has the task of transforming the outside IRC protocol into nice symbolic >>> Erlang terms band and forth. And the Channel servers are tasked with >>> handling pub/sub between the processes, as well as ownership management of >>> the channels in question. >>> >>> The trick is to get events/messages to flow between the connection >>> processes such that the emergent behavior of a wild IRCd suddenly appears :) >>> >>> -- >>> J. >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Mar 24 18:05:00 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 24 Mar 2016 18:05:00 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 Message-ID: Hi, I'm pleased to announce version 13.3.7 of the positive library. https://github.com/jlouis/positive The library solves the extremely hard problem of figuring out if an integer is positive or not. Drawn upon experience from the Node.js and Javascript communities, the necessity of such a library should be obvious. We already have several issues that needs solving, and we have had several pull requests in its short lifetime. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Thu Mar 24 18:17:31 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 24 Mar 2016 12:17:31 -0500 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: This kind of thing is unnecessary and toxic. You've just reinforced the perception that the Erlang community is made up of smug jerks. Please stop. Sean On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi, > > I'm pleased to announce version 13.3.7 of the positive library. > > https://github.com/jlouis/positive > > The library solves the extremely hard problem of figuring out if an > integer is positive or not. Drawn upon experience from the Node.js and > Javascript communities, the necessity of such a library should be obvious. > > We already have several issues that needs solving, and we have had several > pull requests in its short lifetime. > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordanrw@REDACTED Thu Mar 24 19:38:52 2016 From: jordanrw@REDACTED (Jordan West) Date: Thu, 24 Mar 2016 11:38:52 -0700 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: I'd like to second Sean's comments. I'd love to see the Erlang community constructively discuss what can be learned from the node.js event: like the affirmation that mutable artifacts are a poor design choice for a dependency manager, but we can do that w/o maligning another community. All languages, including Erlang, have "rough edges", that can be criticized, and this isn't the substantive one. Jordan On Thu, Mar 24, 2016 at 10:17 AM, Sean Cribbs wrote: > This kind of thing is unnecessary and toxic. You've just reinforced the > perception that the Erlang community is made up of smug jerks. Please stop. > > Sean > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> Hi, >> >> I'm pleased to announce version 13.3.7 of the positive library. >> >> https://github.com/jlouis/positive >> >> The library solves the extremely hard problem of figuring out if an >> integer is positive or not. Drawn upon experience from the Node.js and >> Javascript communities, the necessity of such a library should be obvious. >> >> We already have several issues that needs solving, and we have had >> several pull requests in its short lifetime. >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordanrw@REDACTED Thu Mar 24 20:25:32 2016 From: jordanrw@REDACTED (Jordan West) Date: Thu, 24 Mar 2016 12:25:32 -0700 Subject: [erlang-questions] Way of using CRDT? In-Reply-To: References: Message-ID: Originally, the Basho/Riak community referred to version vectors as vector clocks and some of this naming is still evident, like in the examples you mention. As you stated, how you use the data structure conveys whether or not it is a version vector or a vector clock. The dotted version vector is now considered the more correct data structure to use on the server because it removes the potential for concurrency anomalies (as discussed in the repo you linked) but some projects either a) do not have a use case that would trigger that anomaly so they choose to use version vectors (perhaps accidentally called vector clocks) or b) have not run into/proven the anomaly to themselves and have not invested time to fix it as a result (this was the case for Riak pre-2.0). Hope that helps, Jordan On Tue, Mar 22, 2016 at 8:51 PM, LiuZhen wrote: > Hi All, > > Recently I was trying to set up a prototype app with riak-core. It all > went smooth until ran into CRDT. > > I have read: > > > 1. papers about CRDT from HAL > 1. `Conflict-free Replicated Data Types? at > https://hal.inria.fr/file/index/docid/609399/filename/RR-7687.pdf > 2. `A Comprehensive Study of Convergent and Commutative Replicated > Data Types? at https://hal.archives-ouvertes.fr/inria-00555588/ > 2. `Version Vectors are not Vector Clocks? at > https://haslab.wordpress.com/2011/07/08/version-vectors-are-not-vector-clocks/ > 3. `Dotted Version Vector Sets' at > https://github.com/ricardobcl/Dotted-Version-Vectors which by the way > is used in riak-core for meta > 4. Project-FiFo?s fifo_dt at https://github.com/project-fifo/fifo_dt > > > Then which following understandings are wrong? > > 1. Version vector is for state-based crdt; vector clock is for > op-based crdt > 2. From #1, riak and similars use state-based CRDT, then it should use > version vector rather than vector clocks > 3. Dotted version vector is more compact than plain version vector > 4. The reason fifo_dt and similars are using vclocks is because > basicly vclock?s static structure is the same as version vector, but since > the update rule is not the same, why it is used in fifo_dt? > 5. To explain #4, the object combining value & vclocks can be thought > as a way of state-based crdt without using version vector > > What is the *most updated & correct* way of doing this? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcy.dengcaiyun@REDACTED Thu Mar 24 16:49:44 2016 From: dcy.dengcaiyun@REDACTED (Caiyun Deng) Date: Thu, 24 Mar 2016 23:49:44 +0800 Subject: [erlang-questions] What happens if two gen_server call each other at the exact same time? Message-ID: Hi! What happens if two gen_server call each other at the exact same time? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Mar 24 20:44:44 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 24 Mar 2016 20:44:44 +0100 Subject: [erlang-questions] What happens if two gen_server call each other at the exact same time? In-Reply-To: References: Message-ID: On Thu, Mar 24, 2016 at 4:49 PM, Caiyun Deng wrote: > What happens if two gen_server call each other at the exact same time? > > It depends on what the timeout values of the gen_server calls are set to. But with 5000ms, A will be waiting on B to process, and B will be waiting on A to process. Both servers will crash when the timeout is hit, unless one of A or B manages to process and return an answer before. In short: deadlock. One way to break these situations is to process messages asynchronously through casts, or to "order" your gen_servers such that the call direction does not allow for loops in callers. Your example is the simplest cycle possible, but you can easily hit the same problem for long call-chains ending in a loop. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Mar 24 21:22:33 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 24 Mar 2016 20:22:33 +0000 Subject: [erlang-questions] Example use of public_key:pkix_crls_validate? Message-ID: I have a requirement to implement client certificate revocation in my server. Because we use a custom CA, we don't have a convenient way to implement OCSP, so I'm looking at using locally-stored CRLs. I found http://erlang.org/doc/man/public_key.html#pkix_crls_validate-3 which sounds like it would be useful, but the documentation is ... sparse. Are there any good examples of using this function? More generally, are there any good examples of custom verify_fun implementations? My use case for the second is verifying a client certificate to ensure that it has the expected CN and uses a SHA-256 hash. Or am I going to have to write one myself...? :-) From jesper.louis.andersen@REDACTED Fri Mar 25 00:07:35 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 25 Mar 2016 00:07:35 +0100 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: On Thu, Mar 24, 2016 at 4:44 PM, Matthew Shapiro wrote: > I actually think I"m going to forego ranch, Syn, and E2 for the first > iteration and implement them by hand (since this is a learning exercise > more than anything) and there are plenty of guides around to accomplish > each of those individually. This is a really good idea from a learning point of view, since relying on baked solutions here reaps you of the knowledge at the lower levels, which is important to have. What I would steal though, is the notion that the libraries can be separate applications in the same Erlang VM. This allows you to evolve them independently of each other. And if you keep a module in each application which is the interface, you can make it way easier for yourself in the long run. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Mar 25 02:13:32 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 25 Mar 2016 10:13:32 +0900 Subject: [erlang-questions] What happens if two gen_server call each other at the exact same time? In-Reply-To: References: Message-ID: <2355893.y0HBscFPUU@changa> On 2016?3?24? ??? 23:49:44 Caiyun Deng wrote: > Hi! > What happens if two gen_server call each other at the exact same time? > Thanks. A deadlock until the timeout, (A waits on B, B waits on A) with the default gen_* timeout being 5 seconds (defined as 5000ms). Any time you write a protocol between two *identical* processes you always wind up creating the potential for deadlock if you use synchronous calls. So the first rule of No Deadlock Club is you don't synchronously talk about No Deadlock Club. The second rule of No Deadlock Club is to make everything async unless you have a definite reason that you can't. Its surprising, actually, to find just how much stuff in a system you can do async, especially once you separate the "I'm a data processor/transformer" sort of tasks from the "I hold state and serve answers to questions" type tasks within a system. This is an oversimplification, of course, but often you'll have things that are pretty clearly state managers and other things that are pretty clearly workers that are tasked with something to perform in relative isolation of the rest of the system. This requires identifying both kinds of tasks, though, and at the outset of a project that is not always obvious at first -- rubber ducking, crayons and construction paper, and mocking up a system (which in can quickly turn into a full-blown system) can all be effective tools to help identify the pieces and what their responsibilities are. This brings me back to identical peers communicating, though. Occasionally you really will require peers to contact one another directly, and you won't be able to do everything asynchronously (or if you do, you'll wind up re-implementing call/2 without realizing it, and then phased transactions along with it). In this case I cheat and write another process definition that exists only to represent the transaction between the two peers. When Worker.A needs to contact Worker.B it can now spawn_link Hand.A to stand between them. Worker.A -> Hand.A is async, Hand.A <=> Worker.B is sync. If Worker.B spawn_links a Hand.B at the exact same time the communication between Hand.B <=> Worker.A is sync, but Worker.B is free again because Worker.B -> Hand.B was async. Along with creating "hands" to handle the peer synch communication, Worker.A and Worker.B can mark themselves in the state of being in a transaction with the other, and decide to kill one of the hands (I resolve this by sorting on the Worker.A and Worker.B Pids and always killing the higher-value one -- arbitrary, yes, but never hangs). Making sure that it is not possible for two processes to synchronously contact each other at exactly the same time is a big task. When you are writing a totally new project you can catalog your synch calls, but in a pre-existing project things can be far more mysterious and simply grepping for things that look like sync calls may not reveal them all. So always have timeouts. Timeout of 'infinity' is cute or some ideal or whatever to some people, but it can leave deadlocks between peers literally for months without you noticing. Unless you are loading billions of objects from a text file or waiting on an external network resources, 5 seconds is a *very* long time within an Erlang system. Think of it this way: we don't usually wait 5 seconds for a network file system to mount or to acquire an auth key from a remote resource before deciding that we should timeout and try again. Why would writing infinite timeouts into a massively concurrent system make sense? I have simplified some things above, but hopefully expressed the main ideas surrounding this. This is actually a pretty big deal in certain kinds of systems -- not so much in others. -Craig From zxq9@REDACTED Fri Mar 25 02:20:30 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 25 Mar 2016 10:20:30 +0900 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: <2531248.X1g9vx8mhL@changa> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! I thought it was kind of funny. I have a long list of satirical code stored in my personal files, and a few classic Usenet posts of this nature as well. This is the first time anything like this has ever been called "toxic" that I can remember. Let's go back and beat people over the head with our new morality about that rather in-depth April Fool's joke about C having been "a practical joke". Being a real jerk would be to start actually calling Node.js people stupid. Satirizing a situation that should have never happened is a light-hearted way of leaving an artifact in the net somewhere, a cautionary notice of a sort, that is easy to remember (because it was funny) and reference mentally the next time *we* are creating a new system and run the risk of painting ourselves into the same sort of corner. -Craig On 2016?3?24? ??? 11:38:52 Jordan West wrote: > I'd like to second Sean's comments. I'd love to see the Erlang community > constructively discuss what can be learned from the node.js event: like the > affirmation that mutable artifacts are a poor design choice for a > dependency manager, but we can do that w/o maligning another community. All > languages, including Erlang, have "rough edges", that can be criticized, > and this isn't the substantive one. > > Jordan > > On Thu, Mar 24, 2016 at 10:17 AM, Sean Cribbs wrote: > > > This kind of thing is unnecessary and toxic. You've just reinforced the > > perception that the Erlang community is made up of smug jerks. Please stop. > > > > Sean > > > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen < > > jesper.louis.andersen@REDACTED> wrote: > > > >> Hi, > >> > >> I'm pleased to announce version 13.3.7 of the positive library. > >> > >> https://github.com/jlouis/positive > >> > >> The library solves the extremely hard problem of figuring out if an > >> integer is positive or not. Drawn upon experience from the Node.js and > >> Javascript communities, the necessity of such a library should be obvious. > >> > >> We already have several issues that needs solving, and we have had > >> several pull requests in its short lifetime. > >> > >> -- > >> J. > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > From zxq9@REDACTED Fri Mar 25 02:24:15 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 25 Mar 2016 10:24:15 +0900 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: <2846679.xN8DXsK6O3@changa> On 2016?3?25? ??? 00:07:35 Jesper Louis Andersen wrote: > On Thu, Mar 24, 2016 at 4:44 PM, Matthew Shapiro wrote: > > > I actually think I"m going to forego ranch, Syn, and E2 for the first > > iteration and implement them by hand (since this is a learning exercise > > more than anything) and there are plenty of guides around to accomplish > > each of those individually. > > > This is a really good idea from a learning point of view, since relying on > baked solutions here reaps you of the knowledge at the lower levels, which > is important to have. What I would steal though, is the notion that the > libraries can be separate applications in the same Erlang VM. This allows > you to evolve them independently of each other. And if you keep a module in > each application which is the interface, you can make it way easier for > yourself in the long run. +1 This reminds me... One really nice thing about having a flat namespace for modules is that if you start writing a module directly into your current project, and then later realize it (and some others) should be split out into a separate, more generic, library or application, none of the calling sites elsewhere have to change and you don't have a minefield of "from foo.bar import biz.baz" locations to update (or pray that a tool or script fixes correctly). -Craig From lee.sylvester@REDACTED Fri Mar 25 02:26:19 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Fri, 25 Mar 2016 14:26:19 +1300 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <2531248.X1g9vx8mhL@changa> References: <2531248.X1g9vx8mhL@changa> Message-ID: How about we start talking about the brand new and amazing Puff.JS, the scalable JS platform that can reuse all Node.JS libraries but magically runs on the Erlang VM? On Fri, Mar 25, 2016 at 2:20 PM, zxq9 wrote: > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > I thought it was kind of funny. I have a long list of satirical code stored > in my personal files, and a few classic Usenet posts of this nature as > well. > This is the first time anything like this has ever been called "toxic" that > I can remember. > > Let's go back and beat people over the head with our new morality about > that > rather in-depth April Fool's joke about C having been "a practical joke". > > Being a real jerk would be to start actually calling Node.js people stupid. > Satirizing a situation that should have never happened is a light-hearted > way of leaving an artifact in the net somewhere, a cautionary notice of a > sort, that is easy to remember (because it was funny) and reference > mentally > the next time *we* are creating a new system and run the risk of painting > ourselves into the same sort of corner. > > -Craig > > On 2016?3?24? ??? 11:38:52 Jordan West wrote: > > I'd like to second Sean's comments. I'd love to see the Erlang community > > constructively discuss what can be learned from the node.js event: like > the > > affirmation that mutable artifacts are a poor design choice for a > > dependency manager, but we can do that w/o maligning another community. > All > > languages, including Erlang, have "rough edges", that can be criticized, > > and this isn't the substantive one. > > > > Jordan > > > > On Thu, Mar 24, 2016 at 10:17 AM, Sean Cribbs > wrote: > > > > > This kind of thing is unnecessary and toxic. You've just reinforced the > > > perception that the Erlang community is made up of smug jerks. Please > stop. > > > > > > Sean > > > > > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen < > > > jesper.louis.andersen@REDACTED> wrote: > > > > > >> Hi, > > >> > > >> I'm pleased to announce version 13.3.7 of the positive library. > > >> > > >> https://github.com/jlouis/positive > > >> > > >> The library solves the extremely hard problem of figuring out if an > > >> integer is positive or not. Drawn upon experience from the Node.js and > > >> Javascript communities, the necessity of such a library should be > obvious. > > >> > > >> We already have several issues that needs solving, and we have had > > >> several pull requests in its short lifetime. > > >> > > >> -- > > >> J. > > >> > > >> _______________________________________________ > > >> erlang-questions mailing list > > >> erlang-questions@REDACTED > > >> http://erlang.org/mailman/listinfo/erlang-questions > > >> > > >> > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Fri Mar 25 04:21:11 2016 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Thu, 24 Mar 2016 23:21:11 -0400 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> Message-ID: I, for one, welcome our Node.js overlords... On Thu, Mar 24, 2016 at 9:26 PM, Lee Sylvester wrote: > How about we start talking about the brand new and amazing Puff.JS, the > scalable JS platform that can reuse all Node.JS libraries but magically > runs on the Erlang VM? > > On Fri, Mar 25, 2016 at 2:20 PM, zxq9 wrote: > >> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >> >> I thought it was kind of funny. I have a long list of satirical code >> stored >> in my personal files, and a few classic Usenet posts of this nature as >> well. >> This is the first time anything like this has ever been called "toxic" >> that >> I can remember. >> >> Let's go back and beat people over the head with our new morality about >> that >> rather in-depth April Fool's joke about C having been "a practical joke". >> >> Being a real jerk would be to start actually calling Node.js people >> stupid. >> Satirizing a situation that should have never happened is a light-hearted >> way of leaving an artifact in the net somewhere, a cautionary notice of a >> sort, that is easy to remember (because it was funny) and reference >> mentally >> the next time *we* are creating a new system and run the risk of painting >> ourselves into the same sort of corner. >> >> -Craig >> >> On 2016?3?24? ??? 11:38:52 Jordan West wrote: >> > I'd like to second Sean's comments. I'd love to see the Erlang community >> > constructively discuss what can be learned from the node.js event: like >> the >> > affirmation that mutable artifacts are a poor design choice for a >> > dependency manager, but we can do that w/o maligning another community. >> All >> > languages, including Erlang, have "rough edges", that can be criticized, >> > and this isn't the substantive one. >> > >> > Jordan >> > >> > On Thu, Mar 24, 2016 at 10:17 AM, Sean Cribbs >> wrote: >> > >> > > This kind of thing is unnecessary and toxic. You've just reinforced >> the >> > > perception that the Erlang community is made up of smug jerks. Please >> stop. >> > > >> > > Sean >> > > >> > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen < >> > > jesper.louis.andersen@REDACTED> wrote: >> > > >> > >> Hi, >> > >> >> > >> I'm pleased to announce version 13.3.7 of the positive library. >> > >> >> > >> https://github.com/jlouis/positive >> > >> >> > >> The library solves the extremely hard problem of figuring out if an >> > >> integer is positive or not. Drawn upon experience from the Node.js >> and >> > >> Javascript communities, the necessity of such a library should be >> obvious. >> > >> >> > >> We already have several issues that needs solving, and we have had >> > >> several pull requests in its short lifetime. >> > >> >> > >> -- >> > >> J. >> > >> >> > >> _______________________________________________ >> > >> erlang-questions mailing list >> > >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> >> > >> >> > > >> > > _______________________________________________ >> > > erlang-questions mailing list >> > > erlang-questions@REDACTED >> > > http://erlang.org/mailman/listinfo/erlang-questions >> > > >> > > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Twitter | Blog | G+ | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Fri Mar 25 04:36:54 2016 From: jinni.park@REDACTED (Park, Sungjin) Date: Fri, 25 Mar 2016 12:36:54 +0900 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: On Thu, Mar 24, 2016 at 11:02 AM, Matthew Shapiro wrote: > Thank you very much, Ranch, gproc, and syn seem like perfect frameworks > to handle those portions of it and simplifies a lot of things (though I > need to do some research to figure out the difference between gproc and > syn). So that solves those issues, well although it looks like I will have > to jump through some hoops to get Ranch working on Windows. > > In regards to the channel per process vs not, I think my mind went to that > idea due to knowing that in normal IRC servers channels have other > particular aspects to them (such as titles and modes, etc...) and I guess > those aspects made my mental model lean towards channels as individual > processes (even though I admit those features aren't part of my > requirements since this is just to get my feet wet). > This may be too early to point but I think you'd be better avoid dedicating a process to handle a channel. When it comes to scaling things out, there would eventually be clients connected to different servers than the channel process. Then imagine what, all the messages should travel to the process and back, which is too inefficient obviously. Better design things in distributed fashion. > While I haven't gotten to clustering stuff in Erlang yet, my idea was to > guarantee that if a netsplit occurs you can communicate with user in your > channels that are connected to the same node as you are in. I don't know > yet if that changes the architecture at all but in my mind I'm not sure if > it does (channel manager/channel processes would just relay the messages to > the other nodes). > > Anyways, some pretty interesting and enlightining things to think about > > > On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> >> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro wrote: >> >>> I am now at the point where I want to test some of this knowledge out, >>> and I thought a good idea was to create a (basic) IRC server (as I've >>> written them in the past in other languages, and it seemed like a good use >>> case for Erlang). >> >> >> Here is how I would do it: >> >> * There is no reason to run your own acceptor pool. Every client >> connecting runs behind the `ranch` acceptor pool. >> >> * The thing that happens concurrently in an IRC server are the connecting >> clients. There is relatively little need for a channel to act on behalf of >> itself, so one may look at a solution where a channel is just a list of >> members, handled by a general manager of channel lists in ETS. Posting a >> message to a channel is simply looking up interested parties, and sending >> the message to all of them. OTOH, having a process per channel could be >> helpful in order to proxy messages via the channel process: send to the >> chan process, and have it forward to the recipients. Which is best depends >> is not a priori clear to me, but when I did this years ago, I managed to do >> this without channel processes. >> >> * Consider managing the registry of Pids through either the `gproc` or >> the `syn` frameworks. This avoids you having to redo most of the nasty >> parts of registry and you can avoid the problems of using atoms only as in >> the local registry. >> >> * If you want your server to run as a cluster, you will have to think >> about the problem of a netsplit inside the cluster and what guarantees you >> want to have. >> >> This leaves your supervisor tree in one of two forms: Either the >> top-level supervisor runs a single channel manager process worker, or it >> runs a simple_one_for_one pool of channels together with a manager for >> creating/removing channels, if you deem it necessary to keep a process >> tracking each channel. >> >> In general, I would avoid solutions where you "hand off" state between >> processes as if they were sitting in a pipeline. It is often better to make >> the process itself run as an independent system. Joe said "An Erlang web >> server runs 2 million webservers, each serving one request." In this case, >> you could argue you want to run a couple thousand IRC servers, each serving >> one channel, and a couple thousand connection proxies, each serving one TCP >> connection, connecting to multiple such channels. A connection proxy then >> has the task of transforming the outside IRC protocol into nice symbolic >> Erlang terms band and forth. And the Channel servers are tasked with >> handling pub/sub between the processes, as well as ownership management of >> the channels in question. >> >> The trick is to get events/messages to flow between the connection >> processes such that the emergent behavior of a wild IRCd suddenly appears :) >> >> -- >> J. >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Mar 25 08:38:49 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 25 Mar 2016 10:38:49 +0300 Subject: [erlang-questions] What happens if two gen_server call each other at the exact same time? In-Reply-To: <2355893.y0HBscFPUU@changa> References: <2355893.y0HBscFPUU@changa> Message-ID: When it happens, it is usually a design bug. but if you cannot refactor your code to remove such mutual call, I can advise you to replace one of your calls with asking process dictionary. It is not a good thing, but it works if you need just to query some variable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 12:17:28 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 12:17:28 +0100 Subject: [erlang-questions] Erlang in a WAN Message-ID: Dear list, In order to substantially reduce latencies I need to deploy some servers of an Erlang cluster in distant AWS data centres. I remember reading Erlang having potential issues in WAN settings. Also, if I'm not mistaken the communication between nodes is not encrypted, which is fine if you're in the same VPC but it is not if your nodes have public IPs. Does anyone have war stories / pointers on how to distribute an Erlang cluster across different AWS zones (US, EU, Asia, etc)? Thank you in advance. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 12:20:25 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 12:20:25 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: Message-ID: Forgot to mention that this is the guide I found: http://erlang.org/doc/apps/ssl/ssl_distribution.html Still, war stories and opinions are warmly welcome. Best, r. On Fri, Mar 25, 2016 at 12:17 PM, Roberto Ostinelli wrote: > Dear list, > In order to substantially reduce latencies I need to deploy some servers > of an Erlang cluster in distant AWS data centres. > > I remember reading Erlang having potential issues in WAN settings. Also, > if I'm not mistaken the communication between nodes is not encrypted, which > is fine if you're in the same VPC but it is not if your nodes have public > IPs. > > Does anyone have war stories / pointers on how to distribute an Erlang > cluster across different AWS zones (US, EU, Asia, etc)? > > Thank you in advance. > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Mar 25 12:25:23 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 25 Mar 2016 12:25:23 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: Message-ID: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> I would not use distributed erlang for this. It is way too insecure and unsuitable for large latencies. HTTP/2 is I think the best option. Chatterbox looks quite good: https://github.com/joedevivo/chatterbox Sergej > On 25 Mar 2016, at 12:20, Roberto Ostinelli wrote: > > Forgot to mention that this is the guide I found: > http://erlang.org/doc/apps/ssl/ssl_distribution.html > > Still, war stories and opinions are warmly welcome. > > Best, > r. > > > On Fri, Mar 25, 2016 at 12:17 PM, Roberto Ostinelli > wrote: > Dear list, > In order to substantially reduce latencies I need to deploy some servers of an Erlang cluster in distant AWS data centres. > > I remember reading Erlang having potential issues in WAN settings. Also, if I'm not mistaken the communication between nodes is not encrypted, which is fine if you're in the same VPC but it is not if your nodes have public IPs. > > Does anyone have war stories / pointers on how to distribute an Erlang cluster across different AWS zones (US, EU, Asia, etc)? > > Thank you in advance. > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Mar 25 12:38:16 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 25 Mar 2016 20:38:16 +0900 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> Message-ID: <5487488.63WYlhtjQT@changa> Erlang <-> Erlang I wouldn't do HTTP at all. It is just a big ball of added serialization weight and awful type tradeoffs to boot. term_binary/1 <-> binary_term/1 over TLS. And that's about the whole story. You can't do monitors and links and whatnot, so you will have to split your architecture in a way that accommodates that. -Craig On 2016?3?25? ??? 12:25:23 Sergej Jure?ko wrote: > I would not use distributed erlang for this. It is way too insecure and unsuitable for large latencies. HTTP/2 is I think the best option. Chatterbox looks quite good: https://github.com/joedevivo/chatterbox > > Sergej > > > On 25 Mar 2016, at 12:20, Roberto Ostinelli wrote: > > > > Forgot to mention that this is the guide I found: > > http://erlang.org/doc/apps/ssl/ssl_distribution.html > > > > Still, war stories and opinions are warmly welcome. > > > > Best, > > r. > > > > > > On Fri, Mar 25, 2016 at 12:17 PM, Roberto Ostinelli > wrote: > > Dear list, > > In order to substantially reduce latencies I need to deploy some servers of an Erlang cluster in distant AWS data centres. > > > > I remember reading Erlang having potential issues in WAN settings. Also, if I'm not mistaken the communication between nodes is not encrypted, which is fine if you're in the same VPC but it is not if your nodes have public IPs. > > > > Does anyone have war stories / pointers on how to distribute an Erlang cluster across different AWS zones (US, EU, Asia, etc)? > > > > Thank you in advance. > > r. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Fri Mar 25 12:40:43 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 12:40:43 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> Message-ID: HTTP/2? That is rather surprising. Why would that be a best option? For instance, this would mean that a message going from a node in the EU to a node in the US would need to be serialized into an HTTP/2 request? -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Fri Mar 25 12:42:52 2016 From: radek@REDACTED (Rad Gruchalski) Date: Fri, 25 Mar 2016 12:42:52 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> Message-ID: <8663AE5BEB2A43B98CAECF3E5F411150@gruchalski.com> Roberto, For the VPCs, you can use http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-peering.html. For the communication, I?d go with something gossip based. riak_core or gossiperl. Kind regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Friday, 25 March 2016 at 12:40, Roberto Ostinelli wrote: > HTTP/2? That is rather surprising. Why would that be a best option? > > For instance, this would mean that a message going from a node in the EU to a node in the US would need to be serialized into an HTTP/2 request? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Mar 25 12:56:53 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 25 Mar 2016 12:56:53 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> Message-ID: <26156402-CB01-47B8-977F-32F356D730B0@gmail.com> Any kind of way you slice it you're doing serialization. Either Erlang's RPC does term_to_binary or you do it. Makes very little difference. At least with HTTP/2 or something similar, you can carefully manage what gets called. Personally I would be very worried about exposing the capability of calling any erlang function in my application from across the world. HTTP/2 or Thrift would honestly be my only choices. HTTP/2 is easier. > On 25 Mar 2016, at 12:40, Roberto Ostinelli wrote: > > HTTP/2? That is rather surprising. Why would that be a best option? > > For instance, this would mean that a message going from a node in the EU to a node in the US would need to be serialized into an HTTP/2 request? From eric.meadows.jonsson@REDACTED Fri Mar 25 13:16:21 2016 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Fri, 25 Mar 2016 13:16:21 +0100 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: Hex.pm does not allow users to remove or overwrite published packages so the issue that happened with npm cannot happen. Packages will only be removed in very special circumstances, such as us being forced to do so for legal reasons and even then we will of course not allow a new package to be published with the removed's package name. Rebar and Mix will also add package checksums to the lock so if you don't trust the Hex repository you are using you can at least trust the checksum check. Additionally, over the next days I will work on improving and documenting hex.pm's policies so that it will hopefully be clear how we will act in circumstances such as these. On Wed, Mar 23, 2016 at 1:47 PM, Roberto Ostinelli wrote: > On the subject on additional reasons to vendor dependencies: > http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ > > BTW, not saying this can happen with hex.pm. > > Best, > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 13:17:19 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 13:17:19 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <8663AE5BEB2A43B98CAECF3E5F411150@gruchalski.com> References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <8663AE5BEB2A43B98CAECF3E5F411150@gruchalski.com> Message-ID: > > For the VPCs, you can use > http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-peering.html. > For the communication, I?d go with something gossip based. riak_core or > gossiperl. > Thank you Rad, Can't just standard Erlang (epmd etc) used then? -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 13:19:37 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 13:19:37 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <5487488.63WYlhtjQT@changa> References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> Message-ID: > > term_binary/1 <-> binary_term/1 over TLS. > > And that's about the whole story. Well that's one part of it. :) We are using Syn in a distributed environment, so that's how we know which messages need to be routed to which processes. If we were to not use standard Erlang, then it would make things way more complicated: how would we know how to route a request then? -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Mar 25 13:32:28 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 25 Mar 2016 21:32:28 +0900 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <5487488.63WYlhtjQT@changa> Message-ID: <3605124.n99sEQ0FB2@changa> On 2016?3?25? ??? 13:19:37 you wrote: > > > > term_binary/1 <-> binary_term/1 over TLS. > > > > And that's about the whole story. > > > Well that's one part of it. :) > > We are using Syn in a distributed environment, so that's how we know which > messages need to be routed to which processes. > If we were to not use standard Erlang, then it would make things way more > complicated: how would we know how to route a request then? This becomes a questions about how you have the system itself broken up into partitions. Typically the moment you say "WAN" I say "the socket is the border of the system's visibility". That is your new design constraint. If you are currently relying on the idea that everything has a Pid and you can just flip it a message then with a partitioned system you now need to determine *why* process A is foreign relative to process B. If you can determine why that is so you can discover design rules for partitioning the system into two clusters, and at that point the matter of how two processes within a cluster, two processes in two different clusters, or the clusters themselves should interact becomes (in my experience so far) pretty obvious. Its that first step, going from a single mesh of "everything is a Pid or registered name" to "this is now two independent systems" that is the hard step mentally. Most of the time what you find is there are a few places fairly high in the message graph where things split and don't need to interact very closely, or if they do it is either rare, or indirect (usually via some common data store which itself can be conceptually split out as a separate service). So far I haven't had a case in business, chat or game software where this constraint didn't actually lead to cleaner architecture, but on two occasions it *has* led to a bit of extra work cleaning things up that had previously been considered very low-priority refactoring. Hopefully I expressed that without too many typos or skipping much. I'm in a bit of a rush. -Craig From bchesneau@REDACTED Fri Mar 25 13:34:25 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 12:34:25 +0000 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> Message-ID: On Fri, Mar 25, 2016 at 1:19 PM Roberto Ostinelli wrote: > term_binary/1 <-> binary_term/1 over TLS. >> >> And that's about the whole story. > > > Well that's one part of it. :) > > We are using Syn in a distributed environment, so that's how we know which > messages need to be routed to which processes. > If we were to not use standard Erlang, then it would make things way more > complicated: how would we know how to route a request then? > > It all depends what you mean by distribution. Will it be a global registry or a replicated registry to different center? In the first case I would do something like gen_rpc and use some nodes as proxies. In second case it all depends on the latency you can expect. You will have to handle conflicts that can appear between zones. - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Mar 25 13:47:13 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 25 Mar 2016 13:47:13 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> Message-ID: <360DFC14-5C0B-4E98-A9BB-E8A29D88D7EC@gmail.com> > We are using Syn in a distributed environment, so that's how we know which messages need to be routed to which processes. > If we were to not use standard Erlang, then it would make things way more complicated: how would we know how to route a request then? There are no free lunches. You're trying to create a globally distributed app. You can't just ignore the issues that come from that. We use syn, we support global deployments. Syn only runs in local clusters and users are handled wherever their closest cluster is. Sergej From roberto@REDACTED Fri Mar 25 14:02:05 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 14:02:05 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <3605124.n99sEQ0FB2@changa> References: <5487488.63WYlhtjQT@changa> <3605124.n99sEQ0FB2@changa> Message-ID: On Fri, Mar 25, 2016 at 1:32 PM, zxq9 wrote: > > This becomes a questions about how you have the system itself broken up > into partitions. > Craig, I think you hit the spot. Basically this is the question, and it looks like I will need to go this way. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 14:03:14 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 14:03:14 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> Message-ID: On Fri, Mar 25, 2016 at 1:34 PM, Benoit Chesneau wrote: > > It all depends what you mean by distribution. Will it be a global registry > or a replicated registry to different center? In the first case I would do > something like gen_rpc and use some nodes as proxies. > The registry being in mnesia, it boils down to the same thing - Erlang in a WAN, or the alternative: partitioning. Thanks for the gen_rpc input, I will take a look at it. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 14:04:02 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 14:04:02 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: <360DFC14-5C0B-4E98-A9BB-E8A29D88D7EC@gmail.com> References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> <360DFC14-5C0B-4E98-A9BB-E8A29D88D7EC@gmail.com> Message-ID: On Fri, Mar 25, 2016 at 1:47 PM, Sergej Jure?ko wrote: > > There are no free lunches. You're trying to create a globally distributed > app. You can't just ignore the issues that come from that. We use syn, we > support global deployments. Syn only runs in local clusters and users are > handled wherever their closest cluster is. So my understanding is that you partition your users before they hit a specific cluster? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Mar 25 14:13:20 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 25 Mar 2016 14:13:20 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> <360DFC14-5C0B-4E98-A9BB-E8A29D88D7EC@gmail.com> Message-ID: On 25 Mar 2016, at 14:04, Roberto Ostinelli wrote: > > So my understanding is that you partition your users before they hit a specific cluster? Every user has an ID of the form: userid@REDACTED This way it is always clear what needs to be done. Are we using syn to communicate or are we using a long range channel. Sergej From me@REDACTED Fri Mar 25 14:23:57 2016 From: me@REDACTED (Matthew Shapiro) Date: Fri, 25 Mar 2016 09:23:57 -0400 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: I guess I need to read more on how clustering actually works in an OTP way then as I'm not totally sure how I would handle that without at least one dedicated process (even if it's one for all channels that just manages the members list). I figured the clustering would come into play by having either a channel manager process or dedicated channel process for each channel connected to on a specific node, then have the channel manager/channel process relay that out to the other nodes. Otherwise I"m not clear on how a user process would send a message to all users in a channel because it would have to have some way to find out what user processes it would have to send the message to without having to broadcast it out to every user process and let them decide if they should process it or not. Unless the solution to that is Mnesia to manage channel details (haven't gotten that far in my Erlang learning yet). On Thu, Mar 24, 2016 at 11:36 PM, Park, Sungjin wrote: > > > On Thu, Mar 24, 2016 at 11:02 AM, Matthew Shapiro wrote: > >> Thank you very much, Ranch, gproc, and syn seem like perfect frameworks >> to handle those portions of it and simplifies a lot of things (though I >> need to do some research to figure out the difference between gproc and >> syn). So that solves those issues, well although it looks like I will have >> to jump through some hoops to get Ranch working on Windows. >> >> In regards to the channel per process vs not, I think my mind went to >> that idea due to knowing that in normal IRC servers channels have other >> particular aspects to them (such as titles and modes, etc...) and I guess >> those aspects made my mental model lean towards channels as individual >> processes (even though I admit those features aren't part of my >> requirements since this is just to get my feet wet). >> > > This may be too early to point but I think you'd be better avoid > dedicating a process to handle a channel. When it comes to scaling things > out, there would eventually be clients connected to different servers than > the channel process. Then imagine what, all the messages should travel to > the process and back, which is too inefficient obviously. Better design > things in distributed fashion. > > >> While I haven't gotten to clustering stuff in Erlang yet, my idea was to >> guarantee that if a netsplit occurs you can communicate with user in your >> channels that are connected to the same node as you are in. I don't know >> yet if that changes the architecture at all but in my mind I'm not sure if >> it does (channel manager/channel processes would just relay the messages to >> the other nodes). >> >> Anyways, some pretty interesting and enlightining things to think about >> >> >> On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> >>> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro >>> wrote: >>> >>>> I am now at the point where I want to test some of this knowledge out, >>>> and I thought a good idea was to create a (basic) IRC server (as I've >>>> written them in the past in other languages, and it seemed like a good use >>>> case for Erlang). >>> >>> >>> Here is how I would do it: >>> >>> * There is no reason to run your own acceptor pool. Every client >>> connecting runs behind the `ranch` acceptor pool. >>> >>> * The thing that happens concurrently in an IRC server are the >>> connecting clients. There is relatively little need for a channel to act on >>> behalf of itself, so one may look at a solution where a channel is just a >>> list of members, handled by a general manager of channel lists in ETS. >>> Posting a message to a channel is simply looking up interested parties, and >>> sending the message to all of them. OTOH, having a process per channel >>> could be helpful in order to proxy messages via the channel process: send >>> to the chan process, and have it forward to the recipients. Which is best >>> depends is not a priori clear to me, but when I did this years ago, I >>> managed to do this without channel processes. >>> >>> * Consider managing the registry of Pids through either the `gproc` or >>> the `syn` frameworks. This avoids you having to redo most of the nasty >>> parts of registry and you can avoid the problems of using atoms only as in >>> the local registry. >>> >>> * If you want your server to run as a cluster, you will have to think >>> about the problem of a netsplit inside the cluster and what guarantees you >>> want to have. >>> >>> This leaves your supervisor tree in one of two forms: Either the >>> top-level supervisor runs a single channel manager process worker, or it >>> runs a simple_one_for_one pool of channels together with a manager for >>> creating/removing channels, if you deem it necessary to keep a process >>> tracking each channel. >>> >>> In general, I would avoid solutions where you "hand off" state between >>> processes as if they were sitting in a pipeline. It is often better to make >>> the process itself run as an independent system. Joe said "An Erlang web >>> server runs 2 million webservers, each serving one request." In this case, >>> you could argue you want to run a couple thousand IRC servers, each serving >>> one channel, and a couple thousand connection proxies, each serving one TCP >>> connection, connecting to multiple such channels. A connection proxy then >>> has the task of transforming the outside IRC protocol into nice symbolic >>> Erlang terms band and forth. And the Channel servers are tasked with >>> handling pub/sub between the processes, as well as ownership management of >>> the channels in question. >>> >>> The trick is to get events/messages to flow between the connection >>> processes such that the emergent behavior of a wild IRCd suddenly appears :) >>> >>> -- >>> J. >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Mar 25 14:34:14 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 25 Mar 2016 14:34:14 +0100 Subject: [erlang-questions] Erlang in a WAN In-Reply-To: References: <8FC732DA-92D9-4687-BDCA-5A85809F6DA1@gmail.com> <5487488.63WYlhtjQT@changa> <360DFC14-5C0B-4E98-A9BB-E8A29D88D7EC@gmail.com> Message-ID: > > > So my understanding is that you partition your users before they hit a > specific cluster? > > Every user has an ID of the form: userid@REDACTED > This way it is always clear what needs to be done. Are we using syn to > communicate or are we using a long range channel. > Understood. -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Fri Mar 25 16:04:38 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Fri, 25 Mar 2016 10:04:38 -0500 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <2531248.X1g9vx8mhL@changa> References: <2531248.X1g9vx8mhL@changa> Message-ID: On Thu, Mar 24, 2016 at 8:20 PM, zxq9 wrote: > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > I'm reminded of this Molly Ivins' quote ( http://www.goodreads.com/quotes/8534-there-are-two-kinds-of-humor-one-kind-that-makes ): "There are two kinds of humor. One kind that makes us chuckle about our foibles and our shared humanity -- like what Garrison Keillor does. The other kind holds people up to public contempt and ridicule -- that's what I do. Satire is traditionally the weapon of the powerless against the powerful. I only aim at the powerful. When satire is aimed at the powerless, it is not only cruel -- it's vulgar." I'm sure Jesper intended it to be funny, but it just rubs salt in the wound of something that was very embarrassing for the JS community, and makes Erlangers look bad for doing so. All I'm suggesting is that we have some compassion for a change. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Mar 25 18:08:49 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 17:08:49 +0000 Subject: [erlang-questions] pre-load large data files when the application start Message-ID: Hi all, I have a large data file provided as comma separated values (unicode data) I need to load and parse it ASAP since it will be used by all the functions. The current implementation consists in parsing the file and generate either a source file or an include file that will be then compiled. My issue with it for now is that the compilation will use more than 1GB and then crash on small machines or containers. Other solutions I tried: - use merl + `-onload` to build a module on first call of the module (too long the first time) - store an ets file and load it later, which can be an issue if you need to create an escript will all modules later - load an parse in a gen_server (same result as using merl) Thinks I have in mind: - generate a DETS file or small binary tree on disk and cache the content on demand - generate a beam and ship it Is there anything else I can do? I am curious how others are doing in that case. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Mar 25 18:39:54 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 26 Mar 2016 02:39:54 +0900 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> Message-ID: <1940366.BNDHfvLPn8@changa> On 2016?3?25? ??? 10:04:38 Sean Cribbs wrote: > On Thu, Mar 24, 2016 at 8:20 PM, zxq9 wrote: > > > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > > > > I'm reminded of this Molly Ivins' quote ( > http://www.goodreads.com/quotes/8534-there-are-two-kinds-of-humor-one-kind-that-makes > ): > > "There are two kinds of humor. One kind that makes us chuckle about our > foibles and our shared humanity -- like what Garrison Keillor does. The > other kind holds people up to public contempt and ridicule -- that's what I > do. Satire is traditionally the weapon of the powerless against the > powerful. I only aim at the powerful. When satire is aimed at the > powerless, it is not only cruel -- it's vulgar." > > I'm sure Jesper intended it to be funny, but it just rubs salt in the wound > of something that was very embarrassing for the JS community, and makes > Erlangers look bad for doing so. All I'm suggesting is that we have some > compassion for a change. Your quotation references the "powerless" VS the "powerful". WTF Since when was Erlang the big dog and js the outsider? Get some perspective. And yes, massively, unfixably huge screwups that affect users who don't even know what is going on but depend on the magic of actual engineers doing, on occasion, actual engineering is absolutely worthy of satire. Especially if said screwups were satirized loud and clear very early in the life of the environment in which said screwup later occurred. You *do* realize it is just about impossible to get funding for an Erlang based project merely because of the word "Erlang" being associated with it and a crapshoot to get funding for a js project regardless of merit, right? Or have you not recently walked the streets of LA? (Hint: I have.) Get off your horse. It already died. "We shouldn't judge people." "But we must somehow make hiring decisions." "Stop being so judgemental about tech, its all just tradeoffs." "That's nice, but our infrastructure has to run on *something* we decide today, and the balance of all tradeoffs are not equal." -Craig PS: Will anyone second this view? No. Of course not. That is a radical thing today. That only means that we are nearer the end of this cycle than the beginning. Much less controversial to chain one's self to a convention center door with a "code of conduct" taped to one's chest that states attendees are forbidden to take actions that are already illegal. From zxq9@REDACTED Fri Mar 25 18:50:10 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 26 Mar 2016 02:50:10 +0900 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: <1882942.T6njgpjh4E@changa> On 2016?3?25? ??? 17:08:49 Benoit Chesneau wrote: > Hi all, > > I have a large data file provided as comma separated values (unicode data) > I need to load and parse it ASAP since it will be used by all the > functions. ...snip... > Is there anything else I can do? I am curious how others are doing in that > case. Does it all need to be in memory all the time? Based on whether or not this is true and the context of use, I opt for - generate a smaller, more Erlangish version of the dataset (what you're doing with DETS, for example) - load it into a database that is a common resource (not always an option) - write a routine that makes smarter use of file reads than loading everything at once -- this can be surprisingly fast, even in Erlang, and be made to utilize a fixed amount of memory (but is not always a good fit for the problem) But use-case drives everything. Honestly, you're one of the guys I tend to grep posts from when looking for answers to my own questions, so I reckon my ideas above are things you have already considered. Also, with regard to datasets in general, if there is any way to rule out any of the data on load, a combination of a filter + a constant memory read-in can be a big win if you do need it all in memory, but have some criteria by which the data you need all at once can be reduced (again, though, not always the case). -Craig From essen@REDACTED Fri Mar 25 18:55:00 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 25 Mar 2016 18:55:00 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <2531248.X1g9vx8mhL@changa> References: <2531248.X1g9vx8mhL@changa> Message-ID: <56F57B74.3060706@ninenines.eu> On 03/25/2016 02:20 AM, zxq9 wrote: > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! Shame is temporary. A good story is for life. What happened is a story for the ages. And a pretty good one. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From bchesneau@REDACTED Fri Mar 25 19:03:25 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 18:03:25 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: <1882942.T6njgpjh4E@changa> References: <1882942.T6njgpjh4E@changa> Message-ID: On Fri, Mar 25, 2016 at 6:50 PM zxq9 wrote: > On 2016?3?25? ??? 17:08:49 Benoit Chesneau wrote: > > Hi all, > > > > I have a large data file provided as comma separated values (unicode > data) > > I need to load and parse it ASAP since it will be used by all the > > functions. > > ...snip... > > > Is there anything else I can do? I am curious how others are doing in > that > > case. > > Does it all need to be in memory all the time? > > Based on whether or not this is true and the context of use, I opt for > > - generate a smaller, more Erlangish version of the dataset > (what you're doing with DETS, for example) > - load it into a database that is a common resource > (not always an option) > - write a routine that makes smarter use of file reads than loading > everything at once -- this can be surprisingly fast, even in Erlang, > and be made to utilize a fixed amount of memory > (but is not always a good fit for the problem) > > But use-case drives everything. > > Honestly, you're one of the guys I tend to grep posts from when looking for > answers to my own questions, so I reckon my ideas above are things you have > already considered. > > Also, with regard to datasets in general, if there is any way to rule out > any of the data on load, a combination of a filter + a constant memory > read-in can be a big win if you do need it all in memory, but have some > criteria by which the data you need all at once can be reduced (again, > though, not always the case). > > -Craig > Heh, thanks :) DETS sounds a good idea. I didn't try it yet. Doing it now. In the mean time I tried another idea i still need to test on different platforms: - I build the beam first and place it in priv dir - when the application is loaded, I load the beam in memory using `code:load_binary/3` The code is here: https://github.com/benoitc/erlang-idna/tree/precompile The beam is generated using a simple shell file for now: https://github.com/benoitc/erlang-idna/blob/precompile/mkdata.sh and loaded here: https://github.com/benoitc/erlang-idna/blob/precompile/src/idna_unicode.erl#L3 https://github.com/benoitc/erlang-idna/blob/precompile/src/idna_unicode.erl#L216-L226 Tests pass. Right now the only annoying thing with it is the need to also include that separated beam file when you escriptize but I guess I can live with it. I also need to check what is the impact of using such trick when you need to upgrade the application (ie how is handle the -on_load attribute). Maybe I don't need that. Just copying the beam to the correct ebin dir should be enough. Doing it for all build tools around (erlang.mk, rebar3, rebar, ..) makes it had though. Thoughts? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Mar 25 19:05:55 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 25 Mar 2016 18:05:55 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: On Fri, Mar 25, 2016 at 12:09 PM Benoit Chesneau wrote: > Hi all, > > I have a large data file provided as comma separated values (unicode data) > I need to load and parse it ASAP since it will be used by all the > functions. > What's the interface? > The current implementation consists in parsing the file and generate > either a source file or an include file that will be then compiled. My > issue with it for now is that the compilation will use more than 1GB and > then crash on small machines or containers. > > Other solutions I tried: > > - use merl + `-onload` to build a module on first call of the module (too > long the first time) > - store an ets file and load it later, which can be an issue if you need > to create an escript will all modules later > - load an parse in a gen_server (same result as using merl) > > Thinks I have in mind: > > - generate a DETS file or small binary tree on disk and cache the content > on demand > - generate a beam and ship it > > Is there anything else I can do? I am curious how others are doing in > that case. > I think this depends entirely on your interface :) Do you have to scan the entire table? If so why? If not, why not treat this as a indexing problem and start from disk, assuming you can defer loading of any data until it's read? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Mar 25 19:11:20 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 18:11:20 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: On Fri, Mar 25, 2016 at 7:06 PM Garrett Smith wrote: > On Fri, Mar 25, 2016 at 12:09 PM Benoit Chesneau > wrote: > >> Hi all, >> >> I have a large data file provided as comma separated values (unicode >> data) I need to load and parse it ASAP since it will be used by all the >> functions. >> > > What's the interface? > > >> The current implementation consists in parsing the file and generate >> either a source file or an include file that will be then compiled. My >> issue with it for now is that the compilation will use more than 1GB and >> then crash on small machines or containers. >> >> Other solutions I tried: >> >> - use merl + `-onload` to build a module on first call of the module (too >> long the first time) >> - store an ets file and load it later, which can be an issue if you need >> to create an escript will all modules later >> - load an parse in a gen_server (same result as using merl) >> >> Thinks I have in mind: >> >> - generate a DETS file or small binary tree on disk and cache the content >> on demand >> - generate a beam and ship it >> >> Is there anything else I can do? I am curious how others are doing in >> that case. >> > > I think this depends entirely on your interface :) > > Do you have to scan the entire table? If so why? If not, why not treat > this as a indexing problem and start from disk, assuming you can defer > loading of any data until it's read? > Sorry I should have just posted the code I was working on (the advantage of working on opensource stuffs). The code I'm referring is here : https://github.com/benoitc/erlang-idna and the recent change I describe: https://github.com/benoitc/erlang-idna/tree/precompile The table really need to be in memory somehow or need to be accessed very fast while reading it, since it will be used to encode any domain names used in a requests (can be xmpp, http..) . It basically check the code for each chars in a string and try to compose/decompose it. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Mar 25 19:21:16 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Mar 2016 11:21:16 -0700 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F57B74.3060706@ninenines.eu> References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> Message-ID: <56F5819C.50906@gmail.com> On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: > On 03/25/2016 02:20 AM, zxq9 wrote: >> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > Shame is temporary. A good story is for life. > > What happened is a story for the ages. And a pretty good one. > I agree. This is a positive contribution and no one can deny that. From g@REDACTED Fri Mar 25 19:25:55 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 25 Mar 2016 18:25:55 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: On Fri, Mar 25, 2016 at 1:11 PM Benoit Chesneau wrote: > On Fri, Mar 25, 2016 at 7:06 PM Garrett Smith wrote: > >> On Fri, Mar 25, 2016 at 12:09 PM Benoit Chesneau >> wrote: >> >>> Hi all, >>> >>> I have a large data file provided as comma separated values (unicode >>> data) I need to load and parse it ASAP since it will be used by all the >>> functions. >>> >> >> What's the interface? >> >> >>> The current implementation consists in parsing the file and generate >>> either a source file or an include file that will be then compiled. My >>> issue with it for now is that the compilation will use more than 1GB and >>> then crash on small machines or containers. >>> >>> Other solutions I tried: >>> >>> - use merl + `-onload` to build a module on first call of the module >>> (too long the first time) >>> - store an ets file and load it later, which can be an issue if you need >>> to create an escript will all modules later >>> - load an parse in a gen_server (same result as using merl) >>> >>> Thinks I have in mind: >>> >>> - generate a DETS file or small binary tree on disk and cache the >>> content on demand >>> - generate a beam and ship it >>> >>> Is there anything else I can do? I am curious how others are doing in >>> that case. >>> >> >> I think this depends entirely on your interface :) >> >> Do you have to scan the entire table? If so why? If not, why not treat >> this as a indexing problem and start from disk, assuming you can defer >> loading of any data until it's read? >> > > > Sorry I should have just posted the code I was working on (the advantage > of working on opensource stuffs). > > The code I'm referring is here : https://github.com/benoitc/erlang-idna > and the recent change I describe: > https://github.com/benoitc/erlang-idna/tree/precompile > > The table really need to be in memory somehow or need to be accessed very > fast while reading it, since it will be used to encode any domain names > used in a requests (can be xmpp, http..) . > > It basically check the code for each chars in a string and try to > compose/decompose it. > Messing around with includes, beam generation, etc. seems like a possibly plan B or C. I'd start with dets or leveldb and measure performance. But seems like you're well down all roads anyway. So then it's a matter of publishing your results :) Btw, this reminds me a larger scale version of this: https://github.com/gar1t/erlang-bench/blob/master/name-lookup.escript I routinely create an erlang-bench script to satisfy my curiosity and explore different methods. It might not be a good fit as a harness for your test, given the scale - but might useful to sniff test smaller variants. -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Fri Mar 25 19:46:58 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Fri, 25 Mar 2016 13:46:58 -0500 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <1940366.BNDHfvLPn8@changa> References: <2531248.X1g9vx8mhL@changa> <1940366.BNDHfvLPn8@changa> Message-ID: You have just proved my point. I ask for compassion and you tell me how wrong I am. /quit On Fri, Mar 25, 2016 at 12:39 PM, zxq9 wrote: > On 2016?3?25? ??? 10:04:38 Sean Cribbs wrote: > > On Thu, Mar 24, 2016 at 8:20 PM, zxq9 wrote: > > > > > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > > > > > > > I'm reminded of this Molly Ivins' quote ( > > > http://www.goodreads.com/quotes/8534-there-are-two-kinds-of-humor-one-kind-that-makes > > ): > > > > "There are two kinds of humor. One kind that makes us chuckle about our > > foibles and our shared humanity -- like what Garrison Keillor does. The > > other kind holds people up to public contempt and ridicule -- that's > what I > > do. Satire is traditionally the weapon of the powerless against the > > powerful. I only aim at the powerful. When satire is aimed at the > > powerless, it is not only cruel -- it's vulgar." > > > > I'm sure Jesper intended it to be funny, but it just rubs salt in the > wound > > of something that was very embarrassing for the JS community, and makes > > Erlangers look bad for doing so. All I'm suggesting is that we have some > > compassion for a change. > > Your quotation references the "powerless" VS the "powerful". > > WTF > > Since when was Erlang the big dog and js the outsider? > > Get some perspective. > > And yes, massively, unfixably huge screwups that affect users who don't > even know what is going on but depend on the magic of actual engineers > doing, on occasion, actual engineering is absolutely worthy of satire. > > Especially if said screwups were satirized loud and clear very early in > the life of the environment in which said screwup later occurred. > > You *do* realize it is just about impossible to get funding for an Erlang > based project merely because of the word "Erlang" being associated with it > and a crapshoot to get funding for a js project regardless of merit, right? > Or have you not recently walked the streets of LA? > (Hint: I have.) > > Get off your horse. It already died. > > "We shouldn't judge people." > "But we must somehow make hiring decisions." > > "Stop being so judgemental about tech, its all just tradeoffs." > "That's nice, but our infrastructure has to run on *something* we decide > today, and the balance of all tradeoffs are not equal." > > -Craig > > PS: Will anyone second this view? No. Of course not. That is a radical > thing today. That only means that we are nearer the end of this cycle than > the beginning. Much less controversial to chain one's self to a convention > center door with a "code of conduct" taped to one's chest that states > attendees are forbidden to take actions that are already illegal. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Fri Mar 25 19:47:16 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Sat, 26 Mar 2016 07:47:16 +1300 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F5819C.50906@gmail.com> References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> Message-ID: JS is a major player due to laziness. I'm sorry, but a JS runtime on the server is never a good idea. I don't use Elixir / Erlang for every project, I use the right tool for the job, whether I've used it before or not. It just so happens that Elixir / Erlang is often the right tool. I'm sure it's the same for you guys? The fact that Erlangs language is poetry and Elixir's eco-system is bliss means nothing :-P On Mar 26, 2016 7:21 AM, "Michael Truog" wrote: > On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: > >> On 03/25/2016 02:20 AM, zxq9 wrote: >> >>> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >>> >> >> Shame is temporary. A good story is for life. >> >> What happened is a story for the ages. And a pretty good one. >> >> > I agree. This is a positive contribution and no one can deny that. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Mar 25 19:49:12 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Mar 2016 11:49:12 -0700 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: <56F58828.1030906@gmail.com> On 03/25/2016 11:11 AM, Benoit Chesneau wrote: > > > On Fri, Mar 25, 2016 at 7:06 PM Garrett Smith > wrote: > > On Fri, Mar 25, 2016 at 12:09 PM Benoit Chesneau > wrote: > > Hi all, > > I have a large data file provided as comma separated values (unicode data) I need to load and parse it ASAP since it will be used by all the functions. > > > What's the interface? > > The current implementation consists in parsing the file and generate either a source file or an include file that will be then compiled. My issue with it for now is that the compilation will use more than 1GB and then crash on small machines or containers. > > Other solutions I tried: > > - use merl + `-onload` to build a module on first call of the module (too long the first time) > - store an ets file and load it later, which can be an issue if you need to create an escript will all modules later > - load an parse in a gen_server (same result as using merl) > > Thinks I have in mind: > > - generate a DETS file or small binary tree on disk and cache the content on demand > - generate a beam and ship it > > Is there anything else I can do? I am curious how others are doing in that case. > > > I think this depends entirely on your interface :) > > Do you have to scan the entire table? If so why? If not, why not treat this as a indexing problem and start from disk, assuming you can defer loading of any data until it's read? > > > > Sorry I should have just posted the code I was working on (the advantage of working on opensource stuffs). > > The code I'm referring is here : https://github.com/benoitc/erlang-idna > and the recent change I describe: https://github.com/benoitc/erlang-idna/tree/precompile > > The table really need to be in memory somehow or need to be accessed very fast while reading it, since it will be used to encode any domain names used in a requests (can be xmpp, http..) . > > It basically check the code for each chars in a string and try to compose/decompose it. > > - beno?t > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Having the build process generate the module file and the beam file seems decent. There isn't a need to build the module dynamically (during runtime, upon startup) or store the unicode data in global storage due to the unicode changes being infrequent. Then, if you do need to update due to unicode changes, you can always hot-load a new version of the module, during runtime and the usage of the module shouldn't have problems with that, if it is kept as a simple utility/library module. This problem reminds me of the code at https://github.com/rambocoder/unistring and there might be overlap in the goals of these two repositories. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 25 20:10:18 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 25 Mar 2016 20:10:18 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> Message-ID: <56F58D1A.8030903@ninenines.eu> There is no such thing as the right tool for the job. There's the tools that work *for you*, and those that don't. JS is working for a lot of people. Erlang for a lot less. In the real world, all that matters is that the tool is *good enough* and that you are *familiar* with it. Any combination where one of these is false leads to disaster. People who never used Erlang before will not magically come up with a good implementation (they can, but it takes a lot more time). Similarly, people who are trying to use Erlang for what it's not good at will also fail, or struggle to make it work. When choosing a tool for a project, the question should really be "Which tool do I know or can quickly get comfortable with, and can help me produce a working solution?" The answer to that question is different for everyone. On 03/25/2016 07:47 PM, Lee Sylvester wrote: > JS is a major player due to laziness. I'm sorry, but a JS runtime on > the server is never a good idea. I don't use Elixir / Erlang for every > project, I use the right tool for the job, whether I've used it before > or not. It just so happens that Elixir / Erlang is often the right tool. > > I'm sure it's the same for you guys? The fact that Erlangs language is > poetry and Elixir's eco-system is bliss means nothing :-P > > On Mar 26, 2016 7:21 AM, "Michael Truog" > wrote: > > On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: > > On 03/25/2016 02:20 AM, zxq9 wrote: > > EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! > > > Shame is temporary. A good story is for life. > > What happened is a story for the ages. And a pretty good one. > > > I agree. This is a positive contribution and no one can deny that. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From bchesneau@REDACTED Fri Mar 25 22:33:23 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 21:33:23 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: <56F58828.1030906@gmail.com> References: <56F58828.1030906@gmail.com> Message-ID: On Friday, March 25, 2016, Michael Truog wrote: > > Having the build process generate the module file and the beam file seems > decent. There isn't a need to build the module dynamically (during > runtime, upon startup) or store the unicode data in global storage due to > the unicode changes being infrequent. Then, if you do need to update due > to unicode changes, you can always hot-load a new version of the module, > during runtime and the usage of the module shouldn't have problems with > that, if it is kept as a simple utility/library module. This problem > reminds me of the code at https://github.com/rambocoder/unistring and > there might be overlap in the goals of these two repositories. > this is what the current release (1.2) does. But it doesn't compile in containers or machines =< 1GB. The build crash. This is why i'm looking at shipping a pre-compiled beam. or maybe include the data in a db. but for now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms using maps and a pre-compiled beam. Also maps use less storage compared to simply using function pattern matching in the beam. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Fri Mar 25 22:42:23 2016 From: me@REDACTED (Matthew Shapiro) Date: Fri, 25 Mar 2016 17:42:23 -0400 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F58D1A.8030903@ninenines.eu> References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> <56F58D1A.8030903@ninenines.eu> Message-ID: Unfortunately, that's way too true :(. My real project I"m learning Erlang for is for an video ingestion server, because I believe I can create something that works better than what we are currently using at our company, and Erlang and the Beam VM hits every checkbox so much better than every other language and runtime out there. Unfortunately, I am also keenly aware I will never bring this into production at my company since we are a small startup (5 people total, 2 engineers) in Orlando, FL which has zero Erlang developers positions around (and probably thus a small pool of potential developers). It's so fundamentally different (both on a framework and language level) from most other languages in this area that onboarding a new developer onto Erlang in sufficient amount of time is not going to be trivial or cheap, and anything I put into production needs to be able to be maintained by others that are not me. So while Erlang checks all the boxes I still can't say it's the right tool due to that :-/. Of course, Javascript 100% is not the right tool for this job in every way, shape, or form (I actually had a Javascript developer in my coworking space ask why I wasn't doing it in Node.js, sigh). On Fri, Mar 25, 2016 at 3:10 PM, Lo?c Hoguin wrote: > There is no such thing as the right tool for the job. > > There's the tools that work *for you*, and those that don't. > > JS is working for a lot of people. Erlang for a lot less. > > In the real world, all that matters is that the tool is *good enough* and > that you are *familiar* with it. > > Any combination where one of these is false leads to disaster. People who > never used Erlang before will not magically come up with a good > implementation (they can, but it takes a lot more time). Similarly, people > who are trying to use Erlang for what it's not good at will also fail, or > struggle to make it work. > > When choosing a tool for a project, the question should really be "Which > tool do I know or can quickly get comfortable with, and can help me produce > a working solution?" > > The answer to that question is different for everyone. > > On 03/25/2016 07:47 PM, Lee Sylvester wrote: > >> JS is a major player due to laziness. I'm sorry, but a JS runtime on >> the server is never a good idea. I don't use Elixir / Erlang for every >> project, I use the right tool for the job, whether I've used it before >> or not. It just so happens that Elixir / Erlang is often the right tool. >> >> I'm sure it's the same for you guys? The fact that Erlangs language is >> poetry and Elixir's eco-system is bliss means nothing :-P >> >> On Mar 26, 2016 7:21 AM, "Michael Truog" > > wrote: >> >> On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: >> >> On 03/25/2016 02:20 AM, zxq9 wrote: >> >> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >> >> >> Shame is temporary. A good story is for life. >> >> What happened is a story for the ages. And a pretty good one. >> >> >> I agree. This is a positive contribution and no one can deny that. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Fri Mar 25 22:46:32 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Sat, 26 Mar 2016 10:46:32 +1300 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> <56F58D1A.8030903@ninenines.eu> Message-ID: @Matthew, I'm giving a talk in Wellington in August at ScaleConf on moving from Ruby to Elixir. Yes, the underlying fundamentals are different, and OTP is different, but moving from Ruby to Elixir is a much gentler curve than from JS to Erlang. Maybe you could consider hiring Ruby devs? I recently read an article (I forget where) about a company who code with Erlang who don't hire Erlang devs. They hire creative devs with a passion for learning, and simply ask them to learn Erlang. It's a thought. On Sat, Mar 26, 2016 at 10:42 AM, Matthew Shapiro wrote: > Unfortunately, that's way too true :(. > > My real project I"m learning Erlang for is for an video ingestion server, > because I believe I can create something that works better than what we are > currently using at our company, and Erlang and the Beam VM hits every > checkbox so much better than every other language and runtime out there. > > Unfortunately, I am also keenly aware I will never bring this into > production at my company since we are a small startup (5 people total, 2 > engineers) in Orlando, FL which has zero Erlang developers positions around > (and probably thus a small pool of potential developers). It's so > fundamentally different (both on a framework and language level) from most > other languages in this area that onboarding a new developer onto Erlang in > sufficient amount of time is not going to be trivial or cheap, and anything > I put into production needs to be able to be maintained by others that are > not me. So while Erlang checks all the boxes I still can't say it's the > right tool due to that :-/. > > Of course, Javascript 100% is not the right tool for this job in every > way, shape, or form (I actually had a Javascript developer in my coworking > space ask why I wasn't doing it in Node.js, sigh). > > On Fri, Mar 25, 2016 at 3:10 PM, Lo?c Hoguin wrote: > >> There is no such thing as the right tool for the job. >> >> There's the tools that work *for you*, and those that don't. >> >> JS is working for a lot of people. Erlang for a lot less. >> >> In the real world, all that matters is that the tool is *good enough* and >> that you are *familiar* with it. >> >> Any combination where one of these is false leads to disaster. People who >> never used Erlang before will not magically come up with a good >> implementation (they can, but it takes a lot more time). Similarly, people >> who are trying to use Erlang for what it's not good at will also fail, or >> struggle to make it work. >> >> When choosing a tool for a project, the question should really be "Which >> tool do I know or can quickly get comfortable with, and can help me produce >> a working solution?" >> >> The answer to that question is different for everyone. >> >> On 03/25/2016 07:47 PM, Lee Sylvester wrote: >> >>> JS is a major player due to laziness. I'm sorry, but a JS runtime on >>> the server is never a good idea. I don't use Elixir / Erlang for every >>> project, I use the right tool for the job, whether I've used it before >>> or not. It just so happens that Elixir / Erlang is often the right tool. >>> >>> I'm sure it's the same for you guys? The fact that Erlangs language is >>> poetry and Elixir's eco-system is bliss means nothing :-P >>> >>> On Mar 26, 2016 7:21 AM, "Michael Truog" >> > wrote: >>> >>> On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: >>> >>> On 03/25/2016 02:20 AM, zxq9 wrote: >>> >>> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >>> >>> >>> Shame is temporary. A good story is for life. >>> >>> What happened is a story for the ages. And a pretty good one. >>> >>> >>> I agree. This is a positive contribution and no one can deny that. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Mar 25 23:00:07 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 25 Mar 2016 23:00:07 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: Since when is humor toxic? I suspect that node folks don't actually read posts relating to Erlang stuff, and even if they did might be amused or more likely puzzled. When I first saw Jespers post I thought WFT - I didn't think that figuring out if an integer was positive was a problem and I thought maybe Jesper has seen deeper into integers than I had imagined, perhaps figuring out if lazy large integers are positive is undecidable. Then I read the code and realised it was a joke - so Jesper had fooled me for the time it took to check his code. Does satire have to be prefaced by a warning - "might be offensive to XXXX" the version 13.3.7 might have warned me - can you imagine a world without satire? I even used to watch Jon Stewart from Sweden of all places just because he was good at satire - he poked fun at stuff that I had no knowledge of, but he did have a point of view and that I liked. I buy the Guardian just to read John Crace whose political commentary is devastatingly funny and nice to no man. A good critic has a pen dipped in venom ... (Google and read John Crace, and you'll get the idea) - in his hands the pen is a powerful weapon. I actually like reading stuff from "smug jerks" since this gives me the opportunity to either agree or disagree - bland text that nobody could possibly disagree with is not my cup of tea. Can you imagine world with no smug jerks? Nothing to get cross about, nothing to like? Is being called a smug jurk and insult or a compliment? Many of the funniest writers I know are very definitely smug jerks if you disagree with them - or brilliant humorists if you agree with them. The jerkiness is in the eye of the beholder, not the the beholden. Viewing software as practised today is potentially harmful - I could react with horror or amusement to what I see - a lot of what I see stinks so how should I react? - be angry?, be cross? be pios, be smug. I choose to be amused. It's a symptom of our humanity - we muddle through. Cheers /Joe On Thu, Mar 24, 2016 at 6:17 PM, Sean Cribbs wrote: > This kind of thing is unnecessary and toxic. You've just reinforced the > perception that the Erlang community is made up of smug jerks. Please stop. > > Sean > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen > wrote: >> >> Hi, >> >> I'm pleased to announce version 13.3.7 of the positive library. >> >> https://github.com/jlouis/positive >> >> The library solves the extremely hard problem of figuring out if an >> integer is positive or not. Drawn upon experience from the Node.js and >> Javascript communities, the necessity of such a library should be obvious. >> >> We already have several issues that needs solving, and we have had several >> pull requests in its short lifetime. >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From me@REDACTED Fri Mar 25 23:05:10 2016 From: me@REDACTED (Matthew Shapiro) Date: Fri, 25 Mar 2016 18:05:10 -0400 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> <56F58D1A.8030903@ninenines.eu> Message-ID: I'm definitely considering Elixir for the final project, partially due to the language being a bit less foreign to developers in more mainstream languages. I decided to start with Erlang first due to the better OTP resources and once I feel comfortable I'll do a more thorough look at Elixir to determine which I want to do the project in. I'm still not totally confident that even Elixir would make it easier to justify the business risk to the company, but for now I'm treating it as a personal project and I'll deal with that once I'm able to at least successfully show a product that can solve our business problems better than our current systems. And I 1000% agree with the idea of hiring developers based on skills, creativeness and passion instead of basing it on language (though that's mostly because of my personal experience of being quickly dismissed just because the majority of my professional experience is in .Net). However, it's a lot easier to hire based on those when you have a good financial backing to risk developers that end up not being able to be onboarded to foreign systems as easily. Hopefully we get that way sooner rather than later, but until then :) On Fri, Mar 25, 2016 at 5:46 PM, Lee Sylvester wrote: > @Matthew, I'm giving a talk in Wellington in August at ScaleConf on moving > from Ruby to Elixir. Yes, the underlying fundamentals are different, and > OTP is different, but moving from Ruby to Elixir is a much gentler curve > than from JS to Erlang. Maybe you could consider hiring Ruby devs? > > I recently read an article (I forget where) about a company who code with > Erlang who don't hire Erlang devs. They hire creative devs with a passion > for learning, and simply ask them to learn Erlang. > > It's a thought. > > On Sat, Mar 26, 2016 at 10:42 AM, Matthew Shapiro wrote: > >> Unfortunately, that's way too true :(. >> >> My real project I"m learning Erlang for is for an video ingestion server, >> because I believe I can create something that works better than what we are >> currently using at our company, and Erlang and the Beam VM hits every >> checkbox so much better than every other language and runtime out there. >> >> Unfortunately, I am also keenly aware I will never bring this into >> production at my company since we are a small startup (5 people total, 2 >> engineers) in Orlando, FL which has zero Erlang developers positions around >> (and probably thus a small pool of potential developers). It's so >> fundamentally different (both on a framework and language level) from most >> other languages in this area that onboarding a new developer onto Erlang in >> sufficient amount of time is not going to be trivial or cheap, and anything >> I put into production needs to be able to be maintained by others that are >> not me. So while Erlang checks all the boxes I still can't say it's the >> right tool due to that :-/. >> >> Of course, Javascript 100% is not the right tool for this job in every >> way, shape, or form (I actually had a Javascript developer in my coworking >> space ask why I wasn't doing it in Node.js, sigh). >> >> On Fri, Mar 25, 2016 at 3:10 PM, Lo?c Hoguin wrote: >> >>> There is no such thing as the right tool for the job. >>> >>> There's the tools that work *for you*, and those that don't. >>> >>> JS is working for a lot of people. Erlang for a lot less. >>> >>> In the real world, all that matters is that the tool is *good enough* >>> and that you are *familiar* with it. >>> >>> Any combination where one of these is false leads to disaster. People >>> who never used Erlang before will not magically come up with a good >>> implementation (they can, but it takes a lot more time). Similarly, people >>> who are trying to use Erlang for what it's not good at will also fail, or >>> struggle to make it work. >>> >>> When choosing a tool for a project, the question should really be "Which >>> tool do I know or can quickly get comfortable with, and can help me produce >>> a working solution?" >>> >>> The answer to that question is different for everyone. >>> >>> On 03/25/2016 07:47 PM, Lee Sylvester wrote: >>> >>>> JS is a major player due to laziness. I'm sorry, but a JS runtime on >>>> the server is never a good idea. I don't use Elixir / Erlang for every >>>> project, I use the right tool for the job, whether I've used it before >>>> or not. It just so happens that Elixir / Erlang is often the right >>>> tool. >>>> >>>> I'm sure it's the same for you guys? The fact that Erlangs language is >>>> poetry and Elixir's eco-system is bliss means nothing :-P >>>> >>>> On Mar 26, 2016 7:21 AM, "Michael Truog" >>> > wrote: >>>> >>>> On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: >>>> >>>> On 03/25/2016 02:20 AM, zxq9 wrote: >>>> >>>> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >>>> >>>> >>>> Shame is temporary. A good story is for life. >>>> >>>> What happened is a story for the ages. And a pretty good one. >>>> >>>> >>>> I agree. This is a positive contribution and no one can deny that. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> -- >>> Lo?c Hoguin >>> http://ninenines.eu >>> Author of The Erlanger Playbook, >>> A book about software development using Erlang >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Fri Mar 25 23:08:04 2016 From: ali.sabil@REDACTED (Ali Sabil) Date: Fri, 25 Mar 2016 22:08:04 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> Message-ID: Did you consider adding a swap file to your low memory machine? On Fri, Mar 25, 2016 at 9:33 PM Benoit Chesneau wrote: > > > On Friday, March 25, 2016, Michael Truog wrote: > >> >> Having the build process generate the module file and the beam file seems >> decent. There isn't a need to build the module dynamically (during >> runtime, upon startup) or store the unicode data in global storage due to >> the unicode changes being infrequent. Then, if you do need to update due >> to unicode changes, you can always hot-load a new version of the module, >> during runtime and the usage of the module shouldn't have problems with >> that, if it is kept as a simple utility/library module. This problem >> reminds me of the code at https://github.com/rambocoder/unistring and >> there might be overlap in the goals of these two repositories. >> > > > this is what the current release (1.2) does. But it doesn't compile in > containers or machines =< 1GB. The build crash. This is why i'm looking at > shipping a pre-compiled beam. or maybe include the data in a db. but for > now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms > using maps and a pre-compiled beam. Also maps use less storage compared > to simply using function pattern matching in the beam. > > - beno?t > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Fri Mar 25 23:10:34 2016 From: t@REDACTED (Tristan Sloughter) Date: Fri, 25 Mar 2016 17:10:34 -0500 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: <1458943834.1038894.559912290.36B05791@webmail.messagingengine.com> All I want to say, since the argument, 'it is just a joke/satire', is so often used in this discussion and many, many others, is that something being a joke in itself does not mean it is in good taste. Nor does it mean satirical take doesn't come from a place of smug jerkiness. -- Tristan Sloughter t@REDACTED On Fri, Mar 25, 2016, at 05:00 PM, Joe Armstrong wrote: > Since when is humor toxic? > > I suspect that node folks don't actually read posts relating to Erlang > stuff, and even if they did might be amused or more likely puzzled. > > When I first saw Jespers post I thought WFT - I didn't think that > figuring out if an integer was positive was a problem and I thought > maybe Jesper has seen deeper into integers than I had imagined, > perhaps figuring out if lazy large integers are positive is undecidable. > > Then I read the code and realised it was a joke - so Jesper had fooled > me for the time it took to check his code. > > Does satire have to be prefaced by a warning - "might be offensive to > XXXX" the version 13.3.7 might have warned me - can you imagine a world > without satire? I even used to watch Jon Stewart from Sweden of all > places > just because he was good at satire - he poked fun at stuff that I had > no knowledge of, but he did have a point of view and that I liked. > > I buy the Guardian just to read John Crace whose political commentary > is devastatingly funny and nice to no man. A good critic has a pen > dipped > in venom ... (Google and read John Crace, and you'll get the idea) - > in his hands the pen is a powerful weapon. > > I actually like reading stuff from "smug jerks" since this gives me > the opportunity to either agree or disagree - bland text that nobody > could possibly > disagree with is not my cup of tea. > > Can you imagine world with no smug jerks? Nothing to get cross about, > nothing to like? Is being called a smug jurk and insult or a compliment? > > Many of the funniest writers I know are very definitely smug jerks if > you disagree with them - or brilliant humorists if you agree with > them. The jerkiness is in the eye of the beholder, not the the > beholden. > > Viewing software as practised today is potentially harmful - I could > react with > horror or amusement to what I see - a lot of what I see stinks so how > should I react? - be angry?, be cross? be pios, be smug. I choose to > be amused. It's a > symptom of our humanity - we muddle through. > > Cheers > > /Joe > > > > > > > On Thu, Mar 24, 2016 at 6:17 PM, Sean Cribbs > wrote: > > This kind of thing is unnecessary and toxic. You've just reinforced the > > perception that the Erlang community is made up of smug jerks. Please stop. > > > > Sean > > > > On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen > > wrote: > >> > >> Hi, > >> > >> I'm pleased to announce version 13.3.7 of the positive library. > >> > >> https://github.com/jlouis/positive > >> > >> The library solves the extremely hard problem of figuring out if an > >> integer is positive or not. Drawn upon experience from the Node.js and > >> Javascript communities, the necessity of such a library should be obvious. > >> > >> We already have several issues that needs solving, and we have had several > >> pull requests in its short lifetime. > >> > >> -- > >> J. > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Fri Mar 25 23:12:03 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 22:12:03 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> Message-ID: On Fri, Mar 25, 2016 at 11:08 PM Ali Sabil wrote: > Did you consider adding a swap file to your low memory machine? > Thanks for the suggestion. People are also able to increase the RAM sometimes. But since I'm shipping a library I have no control on what knows or could the user. Instead I would prefer to offer a good default experience rather than forcing them to play with the OS configuration or the hardware. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Mar 25 23:19:12 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Mar 2016 15:19:12 -0700 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> Message-ID: <56F5B960.40307@gmail.com> On 03/25/2016 02:33 PM, Benoit Chesneau wrote: > > > On Friday, March 25, 2016, Michael Truog > wrote: > > > Having the build process generate the module file and the beam file seems decent. There isn't a need to build the module dynamically (during runtime, upon startup) or store the unicode data in global storage due to the unicode changes being infrequent. Then, if you do need to update due to unicode changes, you can always hot-load a new version of the module, during runtime and the usage of the module shouldn't have problems with that, if it is kept as a simple utility/library module. This problem reminds me of the code at https://github.com/rambocoder/unistring and there might be overlap in the goals of these two repositories. > > > > this is what the current release (1.2) does. But it doesn't compile in containers or machines =< 1GB. The build crash. This is why i'm looking at shipping a pre-compiled beam. or maybe include the data in a db. but for now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms using maps and a pre-compiled beam. Also maps use less storage compared to simply using function pattern matching in the beam. > > - beno?t > I think you need to switch to using function pattern matching, when keeping it in a module to keep memory usage down. Storing everything in a map has to deal with a big chunk of map data, but storing everything in the module as function pattern matching cases is just part of the module data (should be better for GC due to less heap usage and should be more efficient). You probably want to try and keep all the function pattern matching cases in-order, though it isn't mentioned as helpful at http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might affect the compiler execution, if not the efficiency of the pattern matching). If you used more formal processing of the unicode CSV data it will be easier, perhaps with a python script (instead of awk/shell-utilities, also portability is better as a single script), to create the Erlang module. If necessary, you could use more than a single Erlang module to deal with separate functions, but a single function should require a single module to keep its update atomic (not trying to split a function into multiple modules based on the input). -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Mar 25 23:32:59 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 25 Mar 2016 22:32:59 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: <56F5B960.40307@gmail.com> References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: On Fri, Mar 25, 2016 at 11:19 PM Michael Truog wrote: > On 03/25/2016 02:33 PM, Benoit Chesneau wrote: > > > > On Friday, March 25, 2016, Michael Truog wrote: > >> >> Having the build process generate the module file and the beam file seems >> decent. There isn't a need to build the module dynamically (during >> runtime, upon startup) or store the unicode data in global storage due to >> the unicode changes being infrequent. Then, if you do need to update due >> to unicode changes, you can always hot-load a new version of the module, >> during runtime and the usage of the module shouldn't have problems with >> that, if it is kept as a simple utility/library module. This problem >> reminds me of the code at https://github.com/rambocoder/unistring and >> there might be overlap in the goals of these two repositories. >> > > > this is what the current release (1.2) does. But it doesn't compile in > containers or machines =< 1GB. The build crash. This is why i'm looking at > shipping a pre-compiled beam. or maybe include the data in a db. but for > now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms > using maps and a pre-compiled beam. Also maps use less storage compared > to simply using function pattern matching in the beam. > > - beno?t > > I think you need to switch to using function pattern matching, when > keeping it in a module to keep memory usage down. Storing everything in a > map has to deal with a big chunk of map data, but storing everything in the > module as function pattern matching cases is just part of the module data > (should be better for GC due to less heap usage and should be more > efficient). You probably want to try and keep all the function pattern > matching cases in-order, though it isn't mentioned as helpful at > http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might > affect the compiler execution, if not the efficiency of the pattern > matching). If you used more formal processing of the unicode CSV data it > will be easier, perhaps with a python script (instead of > awk/shell-utilities, also portability is better as a single script), to > create the Erlang module. If necessary, you could use more than a single > Erlang module to deal with separate functions, but a single function should > require a single module to keep its update atomic (not trying to split a > function into multiple modules based on the input). > I agree pattern matching should be probably better than the maps for GC (they are only 1ms faster on lookup). But the problem is really not generating the module: https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl The current issue with above is the amount of RAM needed to compile the beam. If the application is built on a machine with RAM => 1GB it will fail. I guess I could just generate the beam with pattern matching and ship it like I do in the "precompiled" branch . Unless some come with a better idea, i think i will go with it. WWhat do you think? The annoying thing is having to do the `-on_load` hack (just cause i'm lazy). Using rebar or erlang.mk i wouldjust generate and ship it in ebin dir. But rebar3 doesn't copy any content from it to its _build directory :| - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Mar 25 23:41:10 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 25 Mar 2016 22:41:10 +0000 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <2531248.X1g9vx8mhL@changa> <56F57B74.3060706@ninenines.eu> <56F5819C.50906@gmail.com> <56F58D1A.8030903@ninenines.eu> Message-ID: I used to run a Java PaaS. A client had major growth every Jan 1 - the New Year would literally trigger a 2x factor in subscriptions. So every Jan 1, while the rest of the western world enjoyed time off with family, I fought fires in trying to keep this client's services afloat during the onslaught. When we wanted to understand a performance problem in the code, we'd inject some debug statements, or instrumentation - cut a release, redeploy. With some data we'd try to understand the problem, fix, redeploy, rinse, repeat. If you've ever deployed a debug version of your application in the middle of an outage, you have some appreciation for the pain I felt. On a holiday. With people demanding when I'd come down for dinner. As a human, you remember these moments. You remember the details - every little technical detail. They draw the line between your happiness and your hell. The moral of the story is, if you don't have these problems, don't worry about Erlang. The Erlang VM lets you poke inside a running app to see what's broken - and fix it without stopping. The Erlang language has features that exploit this. OTP adds further features. These have been exercised in production for more than a decade. That's enough time to get things working. If finding the right language for your corporate culture is the actual problem, then move Erlang off the board. It's just too fringe. Too hard to find developers. Too controversial. If you have other actual problems, Erlang might prove useful. For my part, I'll never put a back-end system into production that's not written in Erlang, unless there's a good technical reason. And there certainly may be. But Erlang is my starting point, on its merits. And I'm not saying every part of a back end system is running in an Erlang VM. That's crazy and no one does that. But if you're writing software that runs on a server - and you care about it running and working - what are your options? Seriously. PHP? Node? Go? Ruby? JBoss? Forth? K? Culture and history matter in technology. And Erlang has a long history of operational engineering discipline. If that doesn't matter, it might not be a fit. Okay, you run stuff in production, so how to get started? Find a subsystem that's trivial to implement, but important to get right. Build that in Erlang. It should take a few weeks. If your experience is awful, what have you lost? If it's good, you've opened a path to a way of building and running software. But the problems have to be real. Oh, and apologies if this comes across as elite. I'm actually just an average guy who wants to have a New Year's meal with his family. True story. On Fri, Mar 25, 2016 at 4:42 PM Matthew Shapiro wrote: > Unfortunately, that's way too true :(. > > My real project I"m learning Erlang for is for an video ingestion server, > because I believe I can create something that works better than what we are > currently using at our company, and Erlang and the Beam VM hits every > checkbox so much better than every other language and runtime out there. > > Unfortunately, I am also keenly aware I will never bring this into > production at my company since we are a small startup (5 people total, 2 > engineers) in Orlando, FL which has zero Erlang developers positions around > (and probably thus a small pool of potential developers). It's so > fundamentally different (both on a framework and language level) from most > other languages in this area that onboarding a new developer onto Erlang in > sufficient amount of time is not going to be trivial or cheap, and anything > I put into production needs to be able to be maintained by others that are > not me. So while Erlang checks all the boxes I still can't say it's the > right tool due to that :-/. > > Of course, Javascript 100% is not the right tool for this job in every > way, shape, or form (I actually had a Javascript developer in my coworking > space ask why I wasn't doing it in Node.js, sigh). > > On Fri, Mar 25, 2016 at 3:10 PM, Lo?c Hoguin wrote: > >> There is no such thing as the right tool for the job. >> >> There's the tools that work *for you*, and those that don't. >> >> JS is working for a lot of people. Erlang for a lot less. >> >> In the real world, all that matters is that the tool is *good enough* and >> that you are *familiar* with it. >> >> Any combination where one of these is false leads to disaster. People who >> never used Erlang before will not magically come up with a good >> implementation (they can, but it takes a lot more time). Similarly, people >> who are trying to use Erlang for what it's not good at will also fail, or >> struggle to make it work. >> >> When choosing a tool for a project, the question should really be "Which >> tool do I know or can quickly get comfortable with, and can help me produce >> a working solution?" >> >> The answer to that question is different for everyone. >> >> On 03/25/2016 07:47 PM, Lee Sylvester wrote: >> >>> JS is a major player due to laziness. I'm sorry, but a JS runtime on >>> the server is never a good idea. I don't use Elixir / Erlang for every >>> project, I use the right tool for the job, whether I've used it before >>> or not. It just so happens that Elixir / Erlang is often the right tool. >>> >>> I'm sure it's the same for you guys? The fact that Erlangs language is >>> poetry and Elixir's eco-system is bliss means nothing :-P >>> >>> On Mar 26, 2016 7:21 AM, "Michael Truog" >> > wrote: >>> >>> On 03/25/2016 10:55 AM, Lo?c Hoguin wrote: >>> >>> On 03/25/2016 02:20 AM, zxq9 wrote: >>> >>> EVERYONE! STOP EVERYTHING! SATIRE IS NOW "TOXIC"! >>> >>> >>> Shame is temporary. A good story is for life. >>> >>> What happened is a story for the ages. And a pretty good one. >>> >>> >>> I agree. This is a positive contribution and no one can deny that. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -- >> Lo?c Hoguin >> http://ninenines.eu >> Author of The Erlanger Playbook, >> A book about software development using Erlang >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Mar 25 23:46:56 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 25 Mar 2016 23:46:56 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <1458943834.1038894.559912290.36B05791@webmail.messagingengine.com> References: <1458943834.1038894.559912290.36B05791@webmail.messagingengine.com> Message-ID: <56F5BFE0.9090201@ninenines.eu> On 03/25/2016 11:10 PM, Tristan Sloughter wrote: > All I want to say, since the argument, 'it is just a joke/satire', is so > often used in this discussion and many, many others, is that something > being a joke in itself does not mean it is in good taste. Nor does it > mean satirical take doesn't come from a place of smug jerkiness. You guys make it sound like it's not possible to both laugh at and feel bad for someone. If you trip and fall in a funny way, I will laugh and be worried that you may have gotten hurt. At the same time. After I know you're OK, I will laugh some more and joke about it. That's what happened to NPM. They trip and fall in a funny way. They're OK now. And people are laughing some more. Nothing wrong with that. Going on a tangent here, but that's a pretty good thing for NPM that people laugh about it. There's plenty of essays on the subject, but for the sake of keeping it short I'll take a very well known quote: First they ignore you, then they laugh at you, then they fight you, then you win. People who have nothing to do with NPM (or JS) are laughing about it. Server-side JS is probably at the "they fight you" step. Now let's imagine what would happen if the same thing happened to Erlang. Can you guess what would happen? Nobody would care. Most JS developers wouldn't even notice. I'll even go as far as say this whole fiasco benefits NPM and Node in the long term, but that's a story for another day. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From g@REDACTED Fri Mar 25 23:55:30 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 25 Mar 2016 22:55:30 +0000 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F5BFE0.9090201@ninenines.eu> References: <1458943834.1038894.559912290.36B05791@webmail.messagingengine.com> <56F5BFE0.9090201@ninenines.eu> Message-ID: Something Mom would always tell me in situations like this, "Make fun of the language, not the packager manger." On Fri, Mar 25, 2016 at 5:47 PM Lo?c Hoguin wrote: > On 03/25/2016 11:10 PM, Tristan Sloughter wrote: > > All I want to say, since the argument, 'it is just a joke/satire', is so > > often used in this discussion and many, many others, is that something > > being a joke in itself does not mean it is in good taste. Nor does it > > mean satirical take doesn't come from a place of smug jerkiness. > > You guys make it sound like it's not possible to both laugh at and feel > bad for someone. > > If you trip and fall in a funny way, I will laugh and be worried that > you may have gotten hurt. At the same time. After I know you're OK, I > will laugh some more and joke about it. > > That's what happened to NPM. They trip and fall in a funny way. They're > OK now. And people are laughing some more. Nothing wrong with that. > > > Going on a tangent here, but that's a pretty good thing for NPM that > people laugh about it. There's plenty of essays on the subject, but for > the sake of keeping it short I'll take a very well known quote: > > First they ignore you, > then they laugh at you, > then they fight you, > then you win. > > People who have nothing to do with NPM (or JS) are laughing about it. > Server-side JS is probably at the "they fight you" step. > > Now let's imagine what would happen if the same thing happened to > Erlang. Can you guess what would happen? Nobody would care. Most JS > developers wouldn't even notice. > > > I'll even go as far as say this whole fiasco benefits NPM and Node in > the long term, but that's a story for another day. > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Mar 26 00:11:56 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Mar 2016 16:11:56 -0700 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: <56F5C5BC.2010207@gmail.com> On 03/25/2016 03:32 PM, Benoit Chesneau wrote: > > > On Fri, Mar 25, 2016 at 11:19 PM Michael Truog > wrote: > > On 03/25/2016 02:33 PM, Benoit Chesneau wrote: >> >> >> On Friday, March 25, 2016, Michael Truog > wrote: >> >> >> Having the build process generate the module file and the beam file seems decent. There isn't a need to build the module dynamically (during runtime, upon startup) or store the unicode data in global storage due to the unicode changes being infrequent. Then, if you do need to update due to unicode changes, you can always hot-load a new version of the module, during runtime and the usage of the module shouldn't have problems with that, if it is kept as a simple utility/library module. This problem reminds me of the code at https://github.com/rambocoder/unistring and there might be overlap in the goals of these two repositories. >> >> >> >> this is what the current release (1.2) does. But it doesn't compile in containers or machines =< 1GB. The build crash. This is why i'm looking at shipping a pre-compiled beam. or maybe include the data in a db. but for now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms using maps and a pre-compiled beam. Also maps use less storage compared to simply using function pattern matching in the beam. >> >> - beno?t >> > I think you need to switch to using function pattern matching, when keeping it in a module to keep memory usage down. Storing everything in a map has to deal with a big chunk of map data, but storing everything in the module as function pattern matching cases is just part of the module data (should be better for GC due to less heap usage and should be more efficient). You probably want to try and keep all the function pattern matching cases in-order, though it isn't mentioned as helpful at http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might affect the compiler execution, if not the efficiency of the pattern matching). If you used more formal processing of the unicode CSV data it will be easier, perhaps with a python script (instead of awk/shell-utilities, also portability is better as a single script), to create the Erlang module. If necessary, you could use more than a single Erlang module to deal with separate functions, but a single function > should require a single module to keep its update atomic (not trying to split a function into multiple modules based on the input). > > > I agree pattern matching should be probably better than the maps for GC (they are only 1ms faster on lookup). But the problem is really not generating the module: > https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl > > The current issue with above is the amount of RAM needed to compile the beam. If the application is built on a machine with RAM => 1GB it will fail. I guess I could just generate the beam with pattern matching and ship it like I do in the "precompiled" branch . Unless some come with a better idea, i think i will go with it. WWhat do you think? The annoying thing is having to do the `-on_load` hack (just cause i'm lazy). Using rebar or erlang.mk i wouldjust generate and ship it in ebin dir. But rebar3 doesn't copy any content from it to its _build directory :| > > > - beno?t You can split the returned tuple into 3 separate modules with separate functions, each for a tuple element. That should reduce the amount of memory necessary for compilation. I know that is a bit odd, and would make the module update process less atomic (which is a bad thing), but as long as a separate module is used to call the 3 separate modules (their respective functions for each element) it can handle an error when the data is inconsistent between them. I don't think there is a problem with treating this as a normal module, and I don't think you will be forced to use a beam file for deployment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sat Mar 26 00:30:59 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Fri, 25 Mar 2016 19:30:59 -0400 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: I do hate to enter this already ridiculous thread, but for what it's worth: Read an article recently by a psychiatrist who contends that anti-authoritarians in the US tend to be diagnosed with psychiatric disorders like ADHD and ODD and drugged up at an early age. http://www.alternet.org/story/154225/would_we_have_drugged_up_einstein_how_anti-authoritarianism_is_deemed_a_mental_health_problem Personally, I think we need more irascible bastards and satirists to challenge the status quo. After all, isn't skepticism the beating heart of science? On the other hand, ad hominem attacks are pretty tacky. All the best, LRP Sent from my iPad > On Mar 25, 2016, at 6:00 PM, Joe Armstrong wrote: > > Since when is humor toxic? > > I suspect that node folks don't actually read posts relating to Erlang > stuff, and even if they did might be amused or more likely puzzled. > > When I first saw Jespers post I thought WFT - I didn't think that > figuring out if an integer was positive was a problem and I thought > maybe Jesper has seen deeper into integers than I had imagined, > perhaps figuring out if lazy large integers are positive is undecidable. > > Then I read the code and realised it was a joke - so Jesper had fooled > me for the time it took to check his code. > > Does satire have to be prefaced by a warning - "might be offensive to > XXXX" the version 13.3.7 might have warned me - can you imagine a world > without satire? I even used to watch Jon Stewart from Sweden of all places > just because he was good at satire - he poked fun at stuff that I had > no knowledge of, but he did have a point of view and that I liked. > > I buy the Guardian just to read John Crace whose political commentary > is devastatingly funny and nice to no man. A good critic has a pen > dipped > in venom ... (Google and read John Crace, and you'll get the idea) - > in his hands the pen is a powerful weapon. > > I actually like reading stuff from "smug jerks" since this gives me > the opportunity to either agree or disagree - bland text that nobody > could possibly > disagree with is not my cup of tea. > > Can you imagine world with no smug jerks? Nothing to get cross about, > nothing to like? Is being called a smug jurk and insult or a compliment? > > Many of the funniest writers I know are very definitely smug jerks if > you disagree with them - or brilliant humorists if you agree with > them. The jerkiness is in the eye of the beholder, not the the > beholden. > > Viewing software as practised today is potentially harmful - I could react with > horror or amusement to what I see - a lot of what I see stinks so how > should I react? - be angry?, be cross? be pios, be smug. I choose to > be amused. It's a > symptom of our humanity - we muddle through. > > Cheers > > /Joe > > > > > > >> On Thu, Mar 24, 2016 at 6:17 PM, Sean Cribbs wrote: >> This kind of thing is unnecessary and toxic. You've just reinforced the >> perception that the Erlang community is made up of smug jerks. Please stop. >> >> Sean >> >> On Thu, Mar 24, 2016 at 12:05 PM, Jesper Louis Andersen >> wrote: >>> >>> Hi, >>> >>> I'm pleased to announce version 13.3.7 of the positive library. >>> >>> https://github.com/jlouis/positive >>> >>> The library solves the extremely hard problem of figuring out if an >>> integer is positive or not. Drawn upon experience from the Node.js and >>> Javascript communities, the necessity of such a library should be obvious. >>> >>> We already have several issues that needs solving, and we have had several >>> pull requests in its short lifetime. >>> >>> -- >>> J. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Sat Mar 26 01:25:53 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 25 Mar 2016 17:25:53 -0700 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: <56F5C5BC.2010207@gmail.com> References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> <56F5C5BC.2010207@gmail.com> Message-ID: <56F5D711.3090407@gmail.com> On 03/25/2016 04:11 PM, Michael Truog wrote: > On 03/25/2016 03:32 PM, Benoit Chesneau wrote: >> >> >> On Fri, Mar 25, 2016 at 11:19 PM Michael Truog > wrote: >> >> On 03/25/2016 02:33 PM, Benoit Chesneau wrote: >>> >>> >>> On Friday, March 25, 2016, Michael Truog > wrote: >>> >>> >>> Having the build process generate the module file and the beam file seems decent. There isn't a need to build the module dynamically (during runtime, upon startup) or store the unicode data in global storage due to the unicode changes being infrequent. Then, if you do need to update due to unicode changes, you can always hot-load a new version of the module, during runtime and the usage of the module shouldn't have problems with that, if it is kept as a simple utility/library module. This problem reminds me of the code at https://github.com/rambocoder/unistring and there might be overlap in the goals of these two repositories. >>> >>> >>> >>> this is what the current release (1.2) does. But it doesn't compile in containers or machines =< 1GB. The build crash. This is why i'm looking at shipping a pre-compiled beam. or maybe include the data in a db. but for now my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms using maps and a pre-compiled beam. Also maps use less storage compared to simply using function pattern matching in the beam. >>> >>> - beno?t >>> >> I think you need to switch to using function pattern matching, when keeping it in a module to keep memory usage down. Storing everything in a map has to deal with a big chunk of map data, but storing everything in the module as function pattern matching cases is just part of the module data (should be better for GC due to less heap usage and should be more efficient). You probably want to try and keep all the function pattern matching cases in-order, though it isn't mentioned as helpful at http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might affect the compiler execution, if not the efficiency of the pattern matching). If you used more formal processing of the unicode CSV data it will be easier, perhaps with a python script (instead of awk/shell-utilities, also portability is better as a single script), to create the Erlang module. If necessary, you could use more than a single Erlang module to deal with separate functions, but a single function >> should require a single module to keep its update atomic (not trying to split a function into multiple modules based on the input). >> >> >> I agree pattern matching should be probably better than the maps for GC (they are only 1ms faster on lookup). But the problem is really not generating the module: >> https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl >> >> The current issue with above is the amount of RAM needed to compile the beam. If the application is built on a machine with RAM => 1GB it will fail. I guess I could just generate the beam with pattern matching and ship it like I do in the "precompiled" branch . Unless some come with a better idea, i think i will go with it. WWhat do you think? The annoying thing is having to do the `-on_load` hack (just cause i'm lazy). Using rebar or erlang.mk i wouldjust generate and ship it in ebin dir. But rebar3 doesn't copy any content from it to its _build directory :| >> >> >> - beno?t > You can split the returned tuple into 3 separate modules with separate functions, each for a tuple element. That should reduce the amount of memory necessary for compilation. I know that is a bit odd, and would make the module update process less atomic (which is a bad thing), but as long as a separate module is used to call the 3 separate modules (their respective functions for each element) it can handle an error when the data is inconsistent between them. I don't think there is a problem with treating this as a normal module, and I don't think you will be forced to use a beam file for deployment. > Also, forgot to mention, it is better if you deal with the data as integers rather than strings, and that approach saves memory while being more efficient. That is a better change to do, before splitting up the tuple return type, then if necessary, split the tuple return type. Just use the hexadecimal format for integers as necessary. The result may need a list of hexadecimal integers, but that is better than a longer string. -------------- next part -------------- An HTML attachment was scrubbed... URL: From klaus_trainer@REDACTED Sat Mar 26 04:21:29 2016 From: klaus_trainer@REDACTED (Klaus Trainer) Date: Sat, 26 Mar 2016 04:21:29 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: Message-ID: <56F60039.8000503@posteo.de> I'm wondering whether nobody can't see the actual benefit of providing a library that does such a trivial thing, but not more. Is it just people here that can't see through their smugness? Aside from apparently rickrolling Joe Armstrong, Jesper's library is a great demonstration for a minimal solution to the following problem. Provide a library that meets these requirements: * it has a README that explains what problem this library solves * it has a meaningful and clearly recognizable name * it is available under an OSI-approved software license * it is available via the platform's most commonly accepted package repository * one can easily include the provided package in an existing project by using any of the platform's commonly accepted build tools * its source code is provided via a distributed version control system * it uses semantic versioning * it has code documentation explaining the parts in the source code that are not self-explanatory * it has tests * it has type specifications * the type specifications are actually checked when the tests are run * it has code coverage analysis * the code coverage analysis is actually done when the tests are run * the tests are run automatically in a continuous integration system when someone commits code to the main repository or proposes changes to it * one can easily see the status of the continuous integration tests at first glance Some additional things that are important to have in open source projects, but that are still missing from this library (and which would make great contributions, I guess): * end user documentation that shows how to get started and provides a couple of code examples that demonstrate how to use it * API documentation * a CHANGELOG that explains breaking changes and provides upgrade instructions * a code of conduct and information on how it will be enforced and where to report violations * information on how to contribute: e.g. contributing guidelines, how to get support, examples for past contributions, suggestions for future contributions * instructions for reporting issues (where and how etc.) * links to additional resources that are relevant to the topic: e.g. mailing lists, standards, alternative implementations Beside its platform's technical superiority, the only other thing the Erlang community currently seems to be justified to feeling smug about is its own very smugness. It nicely shows how smugness can be implemented in a recursive way (which I hope is tail recursive, though). Well, and there's also some humor, which is often jerky, and sometimes also funny. If the Erlang community would have a reason for feeling smug about a vast ecosystem of projects that meet most of the above standards (or maybe even exceed them), that would be great! From ok@REDACTED Sat Mar 26 06:49:30 2016 From: ok@REDACTED (ok@REDACTED) Date: Sat, 26 Mar 2016 18:49:30 +1300 Subject: [erlang-questions] Beginner trying to figure out idiomatic OTP architecture In-Reply-To: References: Message-ID: <2dbf974f62c13ae42ea3b79894a2d755.squirrel@chasm.otago.ac.nz> Garret Smith wrote that he has "an admitted tendency to be glib about architecture". Whether that is a safe approach depends on the (complexity) scale of the project you are working on, nd how fast you can move from one architecture to another when you discover your present architecture is not ideal, as you inevitably will. The put it another way, you can make a tree-house without worrying about architecture, but not a skyscraper. The answer, of course, is not to build skyscrapers but shanty towns, as long as they get the job one. And that means decouple, decouple, decouple. You can get away with not worrying about architecture much as long as you can test easily, measure easily, and take the bits apart and put them together another way easily. From ok@REDACTED Sat Mar 26 07:42:09 2016 From: ok@REDACTED (ok@REDACTED) Date: Sat, 26 Mar 2016 19:42:09 +1300 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <1940366.BNDHfvLPn8@changa> References: <2531248.X1g9vx8mhL@changa> <1940366.BNDHfvLPn8@changa> Message-ID: <8e07b0df65d6a0aeafcee6869daf6c45.squirrel@chasm.otago.ac.nz> zxq9 wrote > PS: Will anyone second this view? [That JS is the "powerful" and Erlang the "powerless" and that some things richly deserve satire.] Seconded. From ok@REDACTED Sat Mar 26 08:03:53 2016 From: ok@REDACTED (ok@REDACTED) Date: Sat, 26 Mar 2016 20:03:53 +1300 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F60039.8000503@posteo.de> References: <56F60039.8000503@posteo.de> Message-ID: > I'm wondering whether nobody can't see the actual benefit of providing a > library that does such a trivial thing, but not more. When I read the code, I laughed, and then surmised that it was meant as just such an example. I actually thought it was a good one. From bchesneau@REDACTED Sat Mar 26 08:42:44 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 26 Mar 2016 07:42:44 +0000 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: <56F60039.8000503@posteo.de> References: <56F60039.8000503@posteo.de> Message-ID: On Sat, Mar 26, 2016 at 4:21 AM Klaus Trainer wrote: > I'm wondering whether nobody can't see the actual benefit of providing a > library that does such a trivial thing, but not more. Is it just people > here that can't see through their smugness? > > Aside from apparently rickrolling Joe Armstrong, Jesper's library is a > great demonstration for a minimal solution to the following problem. > > Provide a library that meets these requirements: > > * it has a README that explains what problem this library solves > * it has a meaningful and clearly recognizable name > * it is available under an OSI-approved software license > * it is available via the platform's most commonly accepted package > repository > * one can easily include the provided package in an existing project by > using any of the platform's commonly accepted build tools > * its source code is provided via a distributed version control system > * it uses semantic versioning > * it has code documentation explaining the parts in the source code that > are not self-explanatory > * it has tests > * it has type specifications > * the type specifications are actually checked when the tests are run > * it has code coverage analysis > * the code coverage analysis is actually done when the tests are run > * the tests are run automatically in a continuous integration system > when someone commits code to the main repository or proposes changes > to it > * one can easily see the status of the continuous integration tests at > first glance > > Some additional things that are important to have in open source > projects, but that are still missing from this library (and which would > make great contributions, I guess): > > * end user documentation that shows how to get started and provides a > couple of code examples that demonstrate how to use it > * API documentation > * a CHANGELOG that explains breaking changes and provides upgrade > instructions > * a code of conduct and information on how it will be enforced and where > to report violations > * information on how to contribute: e.g. contributing guidelines, how to > get support, examples for past contributions, suggestions for future > contributions > * instructions for reporting issues (where and how etc.) > * links to additional resources that are relevant to the topic: e.g. > mailing lists, standards, alternative implementations > > Beside its platform's technical superiority, the only other thing the > Erlang community currently seems to be justified to feeling smug about > is its own very smugness. It nicely shows how smugness can be > implemented in a recursive way (which I hope is tail recursive, though). > Well, and there's also some humor, which is often jerky, and sometimes > also funny. If the Erlang community would have a reason for feeling > smug about a vast ecosystem of projects that meet most of the above > standards (or maybe even exceed them), that would be great! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions This is a joke right? First it's not a library. A library is a *collection* of similar things. There is no collection with one function. Second, in which world do you live to think we can decently manage thousands of dependencies for a project. Dependencies, on which you don't have any control, forcing you to literally fetch the world to just fix one for your project or adapt it to your own needs. Imo the npm ecosystem is the perfect application of the taylorization forcing each part of the chain depending on each others, introducing as many bottleneck as you have links in it. History proves that taylorization and the choice of using "one best way" had effect on innovation, crystallising some parts of the chain, making really hard to replace some parts of it and holding back the rests. With the final result of stopping any innovation. And this is exactly the issue of the nodejs ecosystem. Without counting the risk that a crucial part of the chain stop to be maintained forcing everyone to move to another. Which may be difficult on rapidly markets. Also this is a very unpleasant process. Anyway back to this one unit code well tested. At the beginning of the century we had these websites collecting snippets. People were able to upload, download, comment and fixe these snippets. This has mostly gone today (letting us with crappy things like stackoverflow). But the idea was good. Maybe rather than having the faulty approach of one code, one package, we could have tools allowing you to easily insert snippets in our code while keeping a link to the source. Besides the copy-paste I mean. It would at least allows us to appropriate this code and keep our authorship. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sat Mar 26 10:31:13 2016 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 26 Mar 2016 09:31:13 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: Is it the .erl -> .beam compilation step that runs out of memory? What happens if you use compile:forms/1,2 instead? Maybe you can write a relatively simple escript that turns your CSV into forms and compiles those (and then writes out the beam file)? See, for example, a script we use to embed binary resources as beam files: https://gist.github.com/rlipscombe/770ce8fc75add11e16f1 On 25 March 2016 at 22:32, Benoit Chesneau wrote: > > > On Fri, Mar 25, 2016 at 11:19 PM Michael Truog wrote: >> >> On 03/25/2016 02:33 PM, Benoit Chesneau wrote: >> >> >> >> On Friday, March 25, 2016, Michael Truog wrote: >>> >>> >>> Having the build process generate the module file and the beam file seems >>> decent. There isn't a need to build the module dynamically (during runtime, >>> upon startup) or store the unicode data in global storage due to the unicode >>> changes being infrequent. Then, if you do need to update due to unicode >>> changes, you can always hot-load a new version of the module, during runtime >>> and the usage of the module shouldn't have problems with that, if it is kept >>> as a simple utility/library module. This problem reminds me of the code at >>> https://github.com/rambocoder/unistring and there might be overlap in the >>> goals of these two repositories. >> >> >> >> this is what the current release (1.2) does. But it doesn't compile in >> containers or machines =< 1GB. The build crash. This is why i'm looking at >> shipping a pre-compiled beam. or maybe include the data in a db. but for now >> my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms using >> maps and a pre-compiled beam. Also maps use less storage compared to simply >> using function pattern matching in the beam. >> >> - beno?t >> >> I think you need to switch to using function pattern matching, when >> keeping it in a module to keep memory usage down. Storing everything in a >> map has to deal with a big chunk of map data, but storing everything in the >> module as function pattern matching cases is just part of the module data >> (should be better for GC due to less heap usage and should be more >> efficient). You probably want to try and keep all the function pattern >> matching cases in-order, though it isn't mentioned as helpful at >> http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might affect >> the compiler execution, if not the efficiency of the pattern matching). If >> you used more formal processing of the unicode CSV data it will be easier, >> perhaps with a python script (instead of awk/shell-utilities, also >> portability is better as a single script), to create the Erlang module. If >> necessary, you could use more than a single Erlang module to deal with >> separate functions, but a single function should require a single module to >> keep its update atomic (not trying to split a function into multiple modules >> based on the input). > > > I agree pattern matching should be probably better than the maps for GC > (they are only 1ms faster on lookup). But the problem is really not > generating the module: > https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl > > The current issue with above is the amount of RAM needed to compile the > beam. If the application is built on a machine with RAM => 1GB it will fail. > I guess I could just generate the beam with pattern matching and ship it > like I do in the "precompiled" branch . Unless some come with a better > idea, i think i will go with it. WWhat do you think? The annoying thing is > having to do the `-on_load` hack (just cause i'm lazy). Using rebar or > erlang.mk i wouldjust generate and ship it in ebin dir. But rebar3 doesn't > copy any content from it to its _build directory :| > > > - beno?t > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Sat Mar 26 13:34:34 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 26 Mar 2016 13:34:34 +0100 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <56F60039.8000503@posteo.de> Message-ID: Hi, I'm going to move myself from around 0.1 nano smug-lisp-weenies to 0 for a little bit and make some rather serious comments: Was this a joke? Yes. I'm not going to change in any way, even if I get flak for making this joke. There is a fine line between what you can say and what you can't, and unless you are willing to walk it now and then, you will not be able to make jokes. I also firmly believe people are able to make their own decisions on the positive/negative impact jokes have on a community. If you can't ignore one jester in the corner, how can you even rule the court? In short: If you evaluate a community on a single incident you don't align with, then you are bound to fail every community in the long run. Dissecting the frog^Wjoke would make it die, but anyway here goes: the joke is not about the writer of the NPM is-positive-integer package, yet it may look like it. If you read what is public on Twitter, the package writer makes some points about this: * It is written in earnest and not as a joke because it is useful * It is useful not because of its contents, but because there is a test suite around the given function. You can easily make the argument that modern software should be small packages with formal proof in Coq for their specifications. The package is a convergence toward that point. * Javascript, in all its insanity of a semantics, has no native integer representation. So you need to handle situations where you get NaN, -Infinity, and other floating point inputs. * The package writer has many other, larger, packages and as such is not a beginner. I'm pretty confident that the writer is a competent person. It is even funnier in Haskell or OCaml, than Erlang, where the type system can implicitly kill most of the corner cases automatically. In Janes St. Core stdlib for OCaml, we have: val sign : int -> Sign.t with Sign.t being type t = Pos | Neg | Zero so 'let positive x = Int.sign x = Pos' is enough to implement it. Most of the scaffolding is already in the standard library in this case. The reason you have many odd-looking libraries in Javascript is due to it having really bad and inconsistent semantics. Asking if something is an array doesn't work in IE8, so you have to resort to hoop-jumping in order to make a fully portable isArray function. And thus, a seemingly insane package is born into NPM. Erlang, in constrast, has a pretty moldy standard library where there is little consistency. But the *semantics* of Erlang are pretty straightforward. Thus we avoid these kinds of special-case libraries. Jokes aside, Klaus Trainer has a point worth following in the long run I think. We don't need much more work on that package before we can run Cuter inside the test suite and we are pretty close to a template. In this case, the library itself *has* to be simple because it is not the desired output: the packaging is. A really nice point was that it took me about 5 minutes to build the initial package and publish it on hex.pm. This turnaround time is something which wouldn't have been possible, had it not been for the work in Elixir around hex.pm. Another interesting point is that this repository has had lots of collaboration. In fact, I'm already being overrun in the commit graph by other people, and their work is better than mine. I find the test suites export-hack beautiful, and wouldn't have come up with that myself :) Benoit says that a library cannot contain a single function, but should be a natural collection of like functions. I would agree a priori. But one problem, which may hit Javascript, is that dead-code-elimination and linking may not be that easy. So if you pull in a large library, you are implicitly forced to download all of that library for your web page. Proper DCE would eliminate that, but alas, Javascript is not going to get it (and neither is Erlang - DCE requires a good static linking phase to get the full benefits). By keeping packages small, you can avoid this problem. The real problem, which is more important to address, is how you handle "evil" or "malicious" packages in a repository. NPM, to my knowledge, does not isolate building and packaging, in the sense that a build step can evaluate arbitrary code. This leads to dangerous situations where the mere build step can take over machines, while the package itself contains no maliciousness. The other problem, of which most people have identified immutability as a solution, is that when you pull the packages at time T, you expect the packages to be the same. But as time goes on, your dependencies might experience a "Temporal Collapse"---A package is being updated in a way that is not compatible with your code, and you cannot reproduce your build. The package manager of the Elm language, while imperfect, uses types to enforce semantic version constraints on packages in an attempt to alleviate this problem. The other temporal problem is when a package is deleted and a new maintainer takes over the the package name. You need a way to tell the dependents they are now using a different package. This is why I really like the idea of locking dependencies in Rebar3, and I hold that we should address our packages by a cryptographic checksum over their content for integrity. Authentication is another subject, and it is quite hard to get right in package managers. It is highly non-trivial and needs to protect against typical attack vectors, for instance the take-over of a central repository by a company. And it has to be simple in the sense that the "Signal" or "WhatsApp" applications are: using a cryptographically secure system should be no harder than using the system without in most cases. Finally, there is HTTP Status Code 451 "Unavailable For Legal Reason", when a package is removed by law. It breaks the immutability solution. I think it would be fair to say that the package should be removed from the repository in such cases, and the name given to the rightful owner. But it is not fair the takeover happens retroactively at the very instant as this leads to Temporal Collapse. Rather, you would like a "sunset" period, in which the earlier version of the package, which people rely on, are still present for download with deprecation warnings. This gives dependents a chance at updating their systems, without leading to immediate breakage. I think we, as programming communities in general, should try to get this into our law-systems such that 3rd parties are not hit by accident. Finally, in full accordance with the toxic person I am, I'll sign this and say I fully stand by everything I did with the joke. And it will not be the last such you see from me, though I think I'll pass on April the 1st now :) On Sat, Mar 26, 2016 at 8:42 AM, Benoit Chesneau wrote: > > > On Sat, Mar 26, 2016 at 4:21 AM Klaus Trainer > wrote: > >> I'm wondering whether nobody can't see the actual benefit of providing a >> library that does such a trivial thing, but not more. Is it just people >> here that can't see through their smugness? >> >> Aside from apparently rickrolling Joe Armstrong, Jesper's library is a >> great demonstration for a minimal solution to the following problem. >> >> Provide a library that meets these requirements: >> >> * it has a README that explains what problem this library solves >> * it has a meaningful and clearly recognizable name >> * it is available under an OSI-approved software license >> * it is available via the platform's most commonly accepted package >> repository >> * one can easily include the provided package in an existing project by >> using any of the platform's commonly accepted build tools >> * its source code is provided via a distributed version control system >> * it uses semantic versioning >> * it has code documentation explaining the parts in the source code that >> are not self-explanatory >> * it has tests >> * it has type specifications >> * the type specifications are actually checked when the tests are run >> * it has code coverage analysis >> * the code coverage analysis is actually done when the tests are run >> * the tests are run automatically in a continuous integration system >> when someone commits code to the main repository or proposes changes >> to it >> * one can easily see the status of the continuous integration tests at >> first glance >> >> Some additional things that are important to have in open source >> projects, but that are still missing from this library (and which would >> make great contributions, I guess): >> >> * end user documentation that shows how to get started and provides a >> couple of code examples that demonstrate how to use it >> * API documentation >> * a CHANGELOG that explains breaking changes and provides upgrade >> instructions >> * a code of conduct and information on how it will be enforced and where >> to report violations >> * information on how to contribute: e.g. contributing guidelines, how to >> get support, examples for past contributions, suggestions for future >> contributions >> * instructions for reporting issues (where and how etc.) >> * links to additional resources that are relevant to the topic: e.g. >> mailing lists, standards, alternative implementations >> >> Beside its platform's technical superiority, the only other thing the >> Erlang community currently seems to be justified to feeling smug about >> is its own very smugness. It nicely shows how smugness can be >> implemented in a recursive way (which I hope is tail recursive, though). >> Well, and there's also some humor, which is often jerky, and sometimes >> also funny. If the Erlang community would have a reason for feeling >> smug about a vast ecosystem of projects that meet most of the above >> standards (or maybe even exceed them), that would be great! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > This is a joke right? > > First it's not a library. A library is a *collection* of similar things. > There is no collection with one function. > > Second, in which world do you live to think we can decently manage > thousands of dependencies for a project. Dependencies, on which you don't > have any control, forcing you to literally fetch the world to just fix one > for your project or adapt it to your own needs. Imo the npm ecosystem is > the perfect application of the taylorization forcing each part of the chain > depending on each others, introducing as many bottleneck as you have links > in it. History proves that taylorization and the choice of using "one best > way" had effect on innovation, crystallising some parts of the chain, > making really hard to replace some parts of it and holding back the rests. > With the final result of stopping any innovation. And this is exactly the > issue of the nodejs ecosystem. Without counting the risk that a crucial > part of the chain stop to be maintained forcing everyone to move to > another. Which may be difficult on rapidly markets. Also this is a very > unpleasant process. > > Anyway back to this one unit code well tested. At the beginning of the > century we had these websites collecting snippets. People were able to > upload, download, comment and fixe these snippets. This has mostly gone > today (letting us with crappy things like stackoverflow). But the idea was > good. Maybe rather than having the faulty approach of one code, one > package, we could have tools allowing you to easily insert snippets in our > code while keeping a link to the source. Besides the copy-paste I mean. It > would at least allows us to appropriate this code and keep our authorship. > > - beno?t > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Mar 26 16:37:01 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 26 Mar 2016 15:37:01 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: On Sat, Mar 26, 2016 at 10:31 AM Roger Lipscombe wrote: > Is it the .erl -> .beam compilation step that runs out of memory? What > happens if you use compile:forms/1,2 instead? Maybe you can write a > relatively simple escript that turns your CSV into forms and compiles > those (and then writes out the beam file)? See, for example, a script > we use to embed binary resources as beam files: > https://gist.github.com/rlipscombe/770ce8fc75add11e16f1 > > On 25 March 2016 at 22:32, Benoit Chesneau wrote: > > > > > > On Fri, Mar 25, 2016 at 11:19 PM Michael Truog > wrote: > >> > >> On 03/25/2016 02:33 PM, Benoit Chesneau wrote: > >> > >> > >> > >> On Friday, March 25, 2016, Michael Truog wrote: > >>> > >>> > >>> Having the build process generate the module file and the beam file > seems > >>> decent. There isn't a need to build the module dynamically (during > runtime, > >>> upon startup) or store the unicode data in global storage due to the > unicode > >>> changes being infrequent. Then, if you do need to update due to > unicode > >>> changes, you can always hot-load a new version of the module, during > runtime > >>> and the usage of the module shouldn't have problems with that, if it > is kept > >>> as a simple utility/library module. This problem reminds me of the > code at > >>> https://github.com/rambocoder/unistring and there might be overlap in > the > >>> goals of these two repositories. > >> > >> > >> > >> this is what the current release (1.2) does. But it doesn't compile in > >> containers or machines =< 1GB. The build crash. This is why i'm looking > at > >> shipping a pre-compiled beam. or maybe include the data in a db. but > for now > >> my tests with a db file (ets) shows it's really slower 30-40ms vs 6ms > using > >> maps and a pre-compiled beam. Also maps use less storage compared to > simply > >> using function pattern matching in the beam. > >> > >> - beno?t > >> > >> I think you need to switch to using function pattern matching, when > >> keeping it in a module to keep memory usage down. Storing everything > in a > >> map has to deal with a big chunk of map data, but storing everything in > the > >> module as function pattern matching cases is just part of the module > data > >> (should be better for GC due to less heap usage and should be more > >> efficient). You probably want to try and keep all the function pattern > >> matching cases in-order, though it isn't mentioned as helpful at > >> http://erlang.org/doc/efficiency_guide/functions.html#id67975 (might > affect > >> the compiler execution, if not the efficiency of the pattern > matching). If > >> you used more formal processing of the unicode CSV data it will be > easier, > >> perhaps with a python script (instead of awk/shell-utilities, also > >> portability is better as a single script), to create the Erlang > module. If > >> necessary, you could use more than a single Erlang module to deal with > >> separate functions, but a single function should require a single > module to > >> keep its update atomic (not trying to split a function into multiple > modules > >> based on the input). > > > > > > I agree pattern matching should be probably better than the maps for GC > > (they are only 1ms faster on lookup). But the problem is really not > > generating the module: > > > https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl > > > > The current issue with above is the amount of RAM needed to compile the > > beam. If the application is built on a machine with RAM => 1GB it will > fail. > > I guess I could just generate the beam with pattern matching and ship it > > like I do in the "precompiled" branch . Unless some come with a better > > idea, i think i will go with it. WWhat do you think? The annoying thing > is > > having to do the `-on_load` hack (just cause i'm lazy). Using rebar or > > erlang.mk i wouldjust generate and ship it in ebin dir. But rebar3 > doesn't > > copy any content from it to its _build directory :| > > > > > > - beno?t > > > > > Michael the idea of using integer is a good idea indded I am making the change so I won't have to make the transformations while running. Which already good. Roger I can do that but how would you link it to the build system? It seems for me I will need to provide a plugin for both rebar3 and erlang.mk. rebar3 is the most problematic there due to the "relative" arch it's using in the _build folder. I will have to find a way to move it in the right ebin.. Any idea? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Mar 26 16:53:41 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 26 Mar 2016 16:53:41 +0100 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: <56F6B085.8090009@ninenines.eu> On 03/26/2016 04:37 PM, Benoit Chesneau wrote: > > > On Sat, Mar 26, 2016 at 10:31 AM Roger Lipscombe > wrote: > > Is it the .erl -> .beam compilation step that runs out of memory? What > happens if you use compile:forms/1,2 instead? Maybe you can write a > relatively simple escript that turns your CSV into forms and compiles > those (and then writes out the beam file)? See, for example, a script > we use to embed binary resources as beam files: > https://gist.github.com/rlipscombe/770ce8fc75add11e16f1 > > On 25 March 2016 at 22:32, Benoit Chesneau > wrote: > > > > > > On Fri, Mar 25, 2016 at 11:19 PM Michael Truog > wrote: > >> > >> On 03/25/2016 02:33 PM, Benoit Chesneau wrote: > >> > >> > >> > >> On Friday, March 25, 2016, Michael Truog > wrote: > >>> > >>> > >>> Having the build process generate the module file and the beam > file seems > >>> decent. There isn't a need to build the module dynamically > (during runtime, > >>> upon startup) or store the unicode data in global storage due > to the unicode > >>> changes being infrequent. Then, if you do need to update due > to unicode > >>> changes, you can always hot-load a new version of the module, > during runtime > >>> and the usage of the module shouldn't have problems with that, > if it is kept > >>> as a simple utility/library module. This problem reminds me of > the code at > >>> https://github.com/rambocoder/unistring and there might be > overlap in the > >>> goals of these two repositories. > >> > >> > >> > >> this is what the current release (1.2) does. But it doesn't > compile in > >> containers or machines =< 1GB. The build crash. This is why i'm > looking at > >> shipping a pre-compiled beam. or maybe include the data in a db. > but for now > >> my tests with a db file (ets) shows it's really slower 30-40ms > vs 6ms using > >> maps and a pre-compiled beam. Also maps use less storage > compared to simply > >> using function pattern matching in the beam. > >> > >> - beno?t > >> > >> I think you need to switch to using function pattern matching, when > >> keeping it in a module to keep memory usage down. Storing > everything in a > >> map has to deal with a big chunk of map data, but storing > everything in the > >> module as function pattern matching cases is just part of the > module data > >> (should be better for GC due to less heap usage and should be more > >> efficient). You probably want to try and keep all the function > pattern > >> matching cases in-order, though it isn't mentioned as helpful at > >> http://erlang.org/doc/efficiency_guide/functions.html#id67975 > (might affect > >> the compiler execution, if not the efficiency of the pattern > matching). If > >> you used more formal processing of the unicode CSV data it will > be easier, > >> perhaps with a python script (instead of awk/shell-utilities, also > >> portability is better as a single script), to create the Erlang > module. If > >> necessary, you could use more than a single Erlang module to > deal with > >> separate functions, but a single function should require a > single module to > >> keep its update atomic (not trying to split a function into > multiple modules > >> based on the input). > > > > > > I agree pattern matching should be probably better than the maps > for GC > > (they are only 1ms faster on lookup). But the problem is really not > > generating the module: > > > https://github.com/benoitc/erlang-idna/blob/v1.x/src/idna_unicode_data1.erl > > > > The current issue with above is the amount of RAM needed to > compile the > > beam. If the application is built on a machine with RAM => 1GB it > will fail. > > I guess I could just generate the beam with pattern matching and > ship it > > like I do in the "precompiled" branch . Unless some come with a > better > > idea, i think i will go with it. WWhat do you think? The annoying > thing is > > having to do the `-on_load` hack (just cause i'm lazy). Using > rebar or > > erlang.mk i wouldjust generate and ship it in > ebin dir. But rebar3 doesn't > > copy any content from it to its _build directory :| > > > > > > - beno?t > > > > > > > Michael the idea of using integer is a good idea indded I am making the > change so I won't have to make the transformations while running. Which > already good. > > Roger I can do that but how would you link it to the build system? It > seems for me I will need to provide a plugin for both rebar3 and > erlang.mk . rebar3 is the most problematic there due > to the "relative" arch it's using in the _build folder. I will have to > find a way to move it in the right ebin.. Any idea? If you follow his advice of writing an escript, then adding support for Erlang.mk should just be a matter of adding app:: ; escript my_escript.erl BEAM_FILES += ebin/generated.beam before the include erlang.mk line. If that's not enough, ping me, because it's probably a bug. Same also works with any other command or script. Alternatively if you do it directly in the Makefile, same advice, except replace the first line with: app:: gen BEAM_FILES += ebin/generated.beam And then define the gen target (before, after, doesn't matter). You could then have rebar just call "make gen". -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roberto.ostinelli@REDACTED Sat Mar 26 19:08:29 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Sat, 26 Mar 2016 19:08:29 +0100 Subject: [erlang-questions] Run in development Message-ID: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> Dear list, When I want to start a node with my code while I develop I use a command such as: erl -pa `rebar3 path` \ -name syn@REDACTED \ +K true \ -eval 'syn:start(),syn:init().' Do you do something similar, or do you have a fancy way to do it? :) Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sat Mar 26 20:26:03 2016 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 26 Mar 2016 19:26:03 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <56F58828.1030906@gmail.com> <56F5B960.40307@gmail.com> Message-ID: On 26 March 2016 at 15:37, Benoit Chesneau wrote: > Roger I can do that but how would you link it to the build system? It seems > for me I will need to provide a plugin for both rebar3 and erlang.mk. rebar3 > is the most problematic there due to the "relative" arch it's using in the > _build folder. I will have to find a way to move it in the right ebin.. Any > idea? Hmmm. Good question. We're still using rebar 2, so the problem doesn't arise. One of my team is in the process of converting our build to use rebar3, so I would ordinarily ask him how he's dealing with it. Except that he's on vacation this week... I'll see if he's pushed his branch recently. From roger@REDACTED Sat Mar 26 20:36:52 2016 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 26 Mar 2016 19:36:52 +0000 Subject: [erlang-questions] Run in development In-Reply-To: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> References: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> Message-ID: On 26 March 2016 at 18:08, Roberto Ostinelli wrote: > When I want to start a node with my code while I develop I use [snip] Ours is pretty similar: erl -pa apps/*/ebin -pa deps/*/ebin -sname foo -config dev -s foo -s bar -s baz ...assuming that we want to run foo, bar and baz top-level applications in a single node (which we do in dev). Then we have (more-or-less): -module(foo). -export([start/0]). start() -> {ok, _} = application:ensure_all_started(foo). Where useful, we'll use (e.g.) https://github.com/rustyio/sync and add "-s sync" From pierrefenoll@REDACTED Sat Mar 26 20:45:50 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Sat, 26 Mar 2016 20:45:50 +0100 Subject: [erlang-questions] Run in development In-Reply-To: References: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> Message-ID: On 26 March 2016 at 20:36, Roger Lipscombe wrote: > erl -pa apps/*/ebin -pa deps/*/ebin -sname foo -config dev -s foo -s bar > -s baz > > ...assuming that we want to run foo, bar and baz top-level > applications in a single node (which we do in dev). Then we have > (more-or-less): > > -module(foo). > -export([start/0]). > start() -> > {ok, _} = application:ensure_all_started(foo). > I often find the need to do that exact same pattern: * erl ? -s myapp * add myapp:start/0 that calls ensure_all_started/1 and matches its output to {ok, _} Would it be a terrible idea to add some CLI option "-app" that would do the same as "-s" except it would call application:ensure_all_started instead of application:start ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Mar 26 20:52:03 2016 From: t@REDACTED (Tristan Sloughter) Date: Sat, 26 Mar 2016 14:52:03 -0500 Subject: [erlang-questions] Run in development In-Reply-To: References: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> Message-ID: <1459021923.1268543.560426154.0EB3F7DA@webmail.messagingengine.com> `rebar3 shell` can start apps and load config instead of creating this custom commands: http://ferd.ca/dev/rebar3-shell.html -- Tristan Sloughter t@REDACTED On Sat, Mar 26, 2016, at 02:45 PM, Pierre Fenoll wrote: > > On 26 March 2016 at 20:36, Roger Lipscombe > wrote: >> erl -pa apps/*/ebin -pa deps/*/ebin -sname foo -config dev -s foo -s >> bar -s baz >> >> ...assuming that we want to run foo, bar and baz top-level >> applications in a single node (which we do in dev). Then we have >> (more-or-less): >> >> -module(foo). >> -export([start/0]). >> start() -> >> {ok, _} = application:ensure_all_started(foo). > > I often find the need to do that exact same pattern: > * erl ? -s myapp > * add myapp:start/0 that calls ensure_all_started/1 and matches its > output to {ok, _} > > Would it be a terrible idea to add some CLI option "-app" that would > do the same as "-s" > except it would call application:ensure_all_started instead of > application:start ? > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat Mar 26 20:53:43 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Sat, 26 Mar 2016 19:53:43 +0000 Subject: [erlang-questions] [ANN] Positive version 13.3.7 In-Reply-To: References: <56F60039.8000503@posteo.de> Message-ID: Geez, I can't believe it's an issue that we're making fun of what happened or about the JavaScript language. Oh, the horror. One thing no one has mentioned is that the reason there is a library like left pad for JS is because JS has to deal with size constraints. Erlang doesn't have that issue. JS is used on the front-end, so forcing a client to download unnecessary code is a no-no, especially on mobile devices. JavaScript is a language in a state of change right now. They're turning a browser language into a server one and they're figuring it out. They'll fix the issues and they'll come out on top, but really services like Github are to blame for npm-gate. I come from the Java world. What I love about the Java world is that we have Apache (and Spring and JBoss) that I can pretty much trust. They have the commons libraries, for example, which are a group of useful utilities that prevent me from having to constantly rewrite the same methods over and over (and left pad is a method within one). I really wish Erlang and Javascript had some foundation like Apache that I could trust and that doesn't just accept any random library. We're missing the filter! It's just not there (or maybe they're not advertising themselves well). I don't trust the code written by some random guy on Github, which is usually just abandoned (I am guilty of this as well). Github is good for what it is, but come on, not everyone is a great programmer and I don't know who is or who isn't based on their Github repo, and I don't have time to muck through their code. I just want to get what I want and not have to worry about it. So when Apache (or JBoss or Spring) releases something, I can certainly feel more confident in the release and that it's going to be supported in some fashion. We're a group of really smart people, so instead of wasting time on discussing the merits of a joke (which I thought was awesome, I mean, go look at the issues in the repo!), we should be looking at this as a learning experience and helping our community grow and prosper. If I was to ask you which JSON library to use, why is it that I'd get different answers? It's crazy. I just want one which I know is going to work and be supported. I know Garret talked about this in one of his talks as a reason for why Erlang is not widely used. Rant over, Andrew On Sat, Mar 26, 2016 at 5:35 AM Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi, > > I'm going to move myself from around 0.1 nano smug-lisp-weenies to 0 for a > little bit and make some rather serious comments: > > Was this a joke? > > Yes. I'm not going to change in any way, even if I get flak for making > this joke. There is a fine line between what you can say and what you > can't, and unless you are willing to walk it now and then, you will not be > able to make jokes. I also firmly believe people are able to make their own > decisions on the positive/negative impact jokes have on a community. If you > can't ignore one jester in the corner, how can you even rule the court? In > short: If you evaluate a community on a single incident you don't align > with, then you are bound to fail every community in the long run. > > Dissecting the frog^Wjoke would make it die, but anyway here goes: the > joke is not about the writer of the NPM is-positive-integer package, yet it > may look like it. If you read what is public on Twitter, the package writer > makes some points about this: > > * It is written in earnest and not as a joke because it is useful > * It is useful not because of its contents, but because there is a test > suite around the given function. You can easily make the argument that > modern software should be small packages with formal proof in Coq for their > specifications. The package is a convergence toward that point. > * Javascript, in all its insanity of a semantics, has no native integer > representation. So you need to handle situations where you get NaN, > -Infinity, and other floating point inputs. > * The package writer has many other, larger, packages and as such is not a > beginner. I'm pretty confident that the writer is a competent person. > > It is even funnier in Haskell or OCaml, than Erlang, where the type system > can implicitly kill most of the corner cases automatically. In Janes St. > Core stdlib for OCaml, we have: > > val sign : int -> Sign.t > > with Sign.t being > > type t = Pos | Neg | Zero > > so 'let positive x = Int.sign x = Pos' is enough to implement it. Most of > the scaffolding is already in the standard library in this case. > > The reason you have many odd-looking libraries in Javascript is due to it > having really bad and inconsistent semantics. Asking if something is an > array doesn't work in IE8, so you have to resort to hoop-jumping in order > to make a fully portable isArray function. And thus, a seemingly insane > package is born into NPM. Erlang, in constrast, has a pretty moldy standard > library where there is little consistency. But the *semantics* of Erlang > are pretty straightforward. Thus we avoid these kinds of special-case > libraries. > > Jokes aside, Klaus Trainer has a point worth following in the long run I > think. We don't need much more work on that package before we can run Cuter > inside the test suite and we are pretty close to a template. In this case, > the library itself *has* to be simple because it is not the desired output: > the packaging is. A really nice point was that it took me about 5 minutes > to build the initial package and publish it on hex.pm. This turnaround > time is something which wouldn't have been possible, had it not been for > the work in Elixir around hex.pm. > > Another interesting point is that this repository has had lots of > collaboration. In fact, I'm already being overrun in the commit graph by > other people, and their work is better than mine. I find the test suites > export-hack beautiful, and wouldn't have come up with that myself :) > > Benoit says that a library cannot contain a single function, but should be > a natural collection of like functions. I would agree a priori. But one > problem, which may hit Javascript, is that dead-code-elimination and > linking may not be that easy. So if you pull in a large library, you are > implicitly forced to download all of that library for your web page. Proper > DCE would eliminate that, but alas, Javascript is not going to get it (and > neither is Erlang - DCE requires a good static linking phase to get the > full benefits). By keeping packages small, you can avoid this problem. > > The real problem, which is more important to address, is how you handle > "evil" or "malicious" packages in a repository. NPM, to my knowledge, does > not isolate building and packaging, in the sense that a build step can > evaluate arbitrary code. This leads to dangerous situations where the mere > build step can take over machines, while the package itself contains no > maliciousness. > > The other problem, of which most people have identified immutability as a > solution, is that when you pull the packages at time T, you expect the > packages to be the same. But as time goes on, your dependencies might > experience a "Temporal Collapse"---A package is being updated in a way that > is not compatible with your code, and you cannot reproduce your build. The > package manager of the Elm language, while imperfect, uses types to enforce > semantic version constraints on packages in an attempt to alleviate this > problem. The other temporal problem is when a package is deleted and a new > maintainer takes over the the package name. You need a way to tell the > dependents they are now using a different package. This is why I really > like the idea of locking dependencies in Rebar3, and I hold that we should > address our packages by a cryptographic checksum over their content for > integrity. > > Authentication is another subject, and it is quite hard to get right in > package managers. It is highly non-trivial and needs to protect against > typical attack vectors, for instance the take-over of a central repository > by a company. And it has to be simple in the sense that the "Signal" or > "WhatsApp" applications are: using a cryptographically secure system should > be no harder than using the system without in most cases. > > Finally, there is HTTP Status Code 451 "Unavailable For Legal Reason", > when a package is removed by law. It breaks the immutability solution. I > think it would be fair to say that the package should be removed from the > repository in such cases, and the name given to the rightful owner. But it > is not fair the takeover happens retroactively at the very instant as this > leads to Temporal Collapse. Rather, you would like a "sunset" period, in > which the earlier version of the package, which people rely on, are still > present for download with deprecation warnings. This gives dependents a > chance at updating their systems, without leading to immediate breakage. I > think we, as programming communities in general, should try to get this > into our law-systems such that 3rd parties are not hit by accident. > > Finally, in full accordance with the toxic person I am, I'll sign this and > say I fully stand by everything I did with the joke. And it will not be the > last such you see from me, though I think I'll pass on April the 1st now :) > > > On Sat, Mar 26, 2016 at 8:42 AM, Benoit Chesneau > wrote: > >> >> >> On Sat, Mar 26, 2016 at 4:21 AM Klaus Trainer >> wrote: >> >>> I'm wondering whether nobody can't see the actual benefit of providing a >>> library that does such a trivial thing, but not more. Is it just people >>> here that can't see through their smugness? >>> >>> Aside from apparently rickrolling Joe Armstrong, Jesper's library is a >>> great demonstration for a minimal solution to the following problem. >>> >>> Provide a library that meets these requirements: >>> >>> * it has a README that explains what problem this library solves >>> * it has a meaningful and clearly recognizable name >>> * it is available under an OSI-approved software license >>> * it is available via the platform's most commonly accepted package >>> repository >>> * one can easily include the provided package in an existing project by >>> using any of the platform's commonly accepted build tools >>> * its source code is provided via a distributed version control system >>> * it uses semantic versioning >>> * it has code documentation explaining the parts in the source code that >>> are not self-explanatory >>> * it has tests >>> * it has type specifications >>> * the type specifications are actually checked when the tests are run >>> * it has code coverage analysis >>> * the code coverage analysis is actually done when the tests are run >>> * the tests are run automatically in a continuous integration system >>> when someone commits code to the main repository or proposes changes >>> to it >>> * one can easily see the status of the continuous integration tests at >>> first glance >>> >>> Some additional things that are important to have in open source >>> projects, but that are still missing from this library (and which would >>> make great contributions, I guess): >>> >>> * end user documentation that shows how to get started and provides a >>> couple of code examples that demonstrate how to use it >>> * API documentation >>> * a CHANGELOG that explains breaking changes and provides upgrade >>> instructions >>> * a code of conduct and information on how it will be enforced and where >>> to report violations >>> * information on how to contribute: e.g. contributing guidelines, how to >>> get support, examples for past contributions, suggestions for future >>> contributions >>> * instructions for reporting issues (where and how etc.) >>> * links to additional resources that are relevant to the topic: e.g. >>> mailing lists, standards, alternative implementations >>> >>> Beside its platform's technical superiority, the only other thing the >>> Erlang community currently seems to be justified to feeling smug about >>> is its own very smugness. It nicely shows how smugness can be >>> implemented in a recursive way (which I hope is tail recursive, though). >>> Well, and there's also some humor, which is often jerky, and sometimes >>> also funny. If the Erlang community would have a reason for feeling >>> smug about a vast ecosystem of projects that meet most of the above >>> standards (or maybe even exceed them), that would be great! >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> This is a joke right? >> >> First it's not a library. A library is a *collection* of similar things. >> There is no collection with one function. >> >> Second, in which world do you live to think we can decently manage >> thousands of dependencies for a project. Dependencies, on which you don't >> have any control, forcing you to literally fetch the world to just fix one >> for your project or adapt it to your own needs. Imo the npm ecosystem is >> the perfect application of the taylorization forcing each part of the chain >> depending on each others, introducing as many bottleneck as you have links >> in it. History proves that taylorization and the choice of using "one best >> way" had effect on innovation, crystallising some parts of the chain, >> making really hard to replace some parts of it and holding back the rests. >> With the final result of stopping any innovation. And this is exactly the >> issue of the nodejs ecosystem. Without counting the risk that a crucial >> part of the chain stop to be maintained forcing everyone to move to >> another. Which may be difficult on rapidly markets. Also this is a very >> unpleasant process. >> >> Anyway back to this one unit code well tested. At the beginning of the >> century we had these websites collecting snippets. People were able to >> upload, download, comment and fixe these snippets. This has mostly gone >> today (letting us with crappy things like stackoverflow). But the idea was >> good. Maybe rather than having the faulty approach of one code, one >> package, we could have tools allowing you to easily insert snippets in our >> code while keeping a link to the source. Besides the copy-paste I mean. It >> would at least allows us to appropriate this code and keep our authorship. >> >> - beno?t >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Sun Mar 27 06:16:16 2016 From: me@REDACTED (Matthew Shapiro) Date: Sun, 27 Mar 2016 00:16:16 -0400 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? Message-ID: In my attempts at learning Erlang I am currently writing a generic tcp_listener application library. One thing I want is for the library to be decoupled from the main project I am writing (simple IRC server) and have it be testable. The way I am handling that is by passing in a MFA for it to pass the accepted socket to, so the tcp_listener gen_server can go back to handling more sockets. I am testing this by creating an EUnit test that connects to the listening socket passed to the listener server and passing in a controlled MFA that sends itself a message back. The code is as follows: ------------------------------------------- handle_cast(accept, State=#state{}) -> {Module, Function, Arguments} = State#state.on_accept_mfa, {ok, AcceptSocket} = gen_tcp:accept(State#state.socket), (State#state.create_acceptor_fun)(), Pid = spawn(Module, Function, [AcceptSocket | Arguments]), io:format(user, "Accept socket valid: ~s~n", [erlang:port_info(AcceptSocket) =/= undefined]), {noreply, State}. -------------------------------------------- %% Tests test_mfa(ListenSocket) -> Mfa = {?MODULE, send_self_message, [self()]}, State = #state{socket = ListenSocket, on_accept_mfa = Mfa, create_acceptor_fun = fun do_nothing/0}, spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) end), {ok, _} = gen_tcp:connect({127,0,0,1}, ?test_port, []), receive success -> ?_assert(true); X -> ?_assert(X) after 1000 -> ?_assert(false) end. %% Stubs do_nothing() -> ok. send_self_message(AcceptSocket, TestPid) -> case erlang:port_info(AcceptSocket) of undefined -> TestPid ! bad_accept_socket; _ -> TestPid ! success end. ------------------------------------------------ Now I am 100% sure that my handle_cast is wrong, in that I should be passing ownership of the accepted socket to the spawned process. Adding that makes everything work honkey dorey. Before I realized that though, I got stuck for quite a while with a failing test that I couldn't figure out why. Before I added that io:format call in handle_cast/2 my tests were failing with erlang:port_info/1 returning undefined. I was going crazy in the shell trying to figure out why and adding debug statements all over the place when I added the io:format/3 call you see now. Suddenly everything works and the test passes. Why does that io:format/3 call (only when after the spawn/3 call) keep the socket open and active for the spawned process? I wouldn't expect it to be garbage collected because the spawned process holds a reference to it (and even adding it to the State in the return doesn't fix it). Only other thing I could think of was that handle_cast is run in it's own process which closes the socket when it dies (since it has ownership), but that doesn't make sense from what I've read about OTP. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Sun Mar 27 06:45:46 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Sun, 27 Mar 2016 00:45:46 -0400 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: I don't know about your garbage-collection question, but: > Why does that io:format/3 call (only when after the spawn/3 call) keep the socket open and active for the spawned process? I wouldn't expect it to be garbage collected because the spawned process holds a reference to it (and even adding it to the State in the return doesn't fix it). Only other thing I could think of was that handle_cast is run in it's own process which closes the socket when it dies (since it has ownership), but that doesn't make sense from what I've read about OTP. Normally handle_cast would be run in the gen_server's process (since that's the whole point), but it looks to me like in your test: > spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) end), you are bypassing gen_server and instead ... telling handle_cast to run in its own one-off process. Do you mean to be doing this? In your test code, you could call gen_server:start/1 and then gen_server:cast/2 and get the actual gen_server behavior. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From vimal7370@REDACTED Sun Mar 27 11:31:49 2016 From: vimal7370@REDACTED (Vimal Kumar) Date: Sun, 27 Mar 2016 15:01:49 +0530 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: Matthew, Like Nathaniel said, the eunit test should involve gen_server. Do not test handle_cast/2 directly. Instead, run your test on an exported function (say: Module:accept/0) which in turn should execute: gen_server:cast(?MODULE, accept) You can start a gen_server process, execute such tests and shut it down properly using 'setup' Fixture: http://learnyousomeerlang.com/eunit#fixtures Regards, Vimal On Sun, Mar 27, 2016 at 10:15 AM, Nathaniel Waisbrot wrote: > I don't know about your garbage-collection question, but: > > > Why does that io:format/3 call (only when after the spawn/3 call) keep > the socket open and active for the spawned process? I wouldn't expect it > to be garbage collected because the spawned process holds a reference to it > (and even adding it to the State in the return doesn't fix it). Only other > thing I could think of was that handle_cast is run in it's own process > which closes the socket when it dies (since it has ownership), but that > doesn't make sense from what I've read about OTP. > > > Normally handle_cast would be run in the gen_server's process (since > that's the whole point), but it looks to me like in your test: > > > > spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) > end), > > > you are bypassing gen_server and instead ... telling handle_cast to run in > its own one-off process. > > > Do you mean to be doing this? In your test code, you could call > gen_server:start/1 and then gen_server:cast/2 and get the actual gen_server > behavior. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Sun Mar 27 12:26:16 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 27 Mar 2016 12:26:16 +0200 Subject: [erlang-questions] Run in development In-Reply-To: <1459021923.1268543.560426154.0EB3F7DA@webmail.messagingengine.com> References: <317FEEC6-674F-4058-A545-EFF449269553@widetag.com> <1459021923.1268543.560426154.0EB3F7DA@webmail.messagingengine.com> Message-ID: <7E82029A-2EF8-4A76-A7CF-77A2F35A6AC0@widetag.com> Thank you all. Just ensuring we're all on the same spot ^^_ > On 26 mar 2016, at 20:52, Tristan Sloughter wrote: > > `rebar3 shell` can start apps and load config instead of creating this custom commands: http://ferd.ca/dev/rebar3-shell.html > > -- > Tristan Sloughter > t@REDACTED > > > >> On Sat, Mar 26, 2016, at 02:45 PM, Pierre Fenoll wrote: >> >> On 26 March 2016 at 20:36, Roger Lipscombe wrote: >> erl -pa apps/*/ebin -pa deps/*/ebin -sname foo -config dev -s foo -s bar -s baz >> >> ...assuming that we want to run foo, bar and baz top-level >> applications in a single node (which we do in dev). Then we have >> (more-or-less): >> >> -module(foo). >> -export([start/0]). >> start() -> >> {ok, _} = application:ensure_all_started(foo). >> >> I often find the need to do that exact same pattern: >> * erl ? -s myapp >> * add myapp:start/0 that calls ensure_all_started/1 and matches its output to {ok, _} >> >> Would it be a terrible idea to add some CLI option "-app" that would do the same as "-s" >> except it would call application:ensure_all_started instead of application:start ? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Sun Mar 27 15:13:31 2016 From: me@REDACTED (Matthew Shapiro) Date: Sun, 27 Mar 2016 09:13:31 -0400 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: Well duh, now I feel like an idiot. Thanks, On Sun, Mar 27, 2016 at 5:31 AM, Vimal Kumar wrote: > Matthew, > > Like Nathaniel said, the eunit test should involve gen_server. Do not test > handle_cast/2 directly. Instead, run your test on an exported function > (say: Module:accept/0) which in turn should execute: > > gen_server:cast(?MODULE, accept) > > You can start a gen_server process, execute such tests and shut it down > properly using 'setup' Fixture: > http://learnyousomeerlang.com/eunit#fixtures > > Regards, > Vimal > > On Sun, Mar 27, 2016 at 10:15 AM, Nathaniel Waisbrot < > nathaniel@REDACTED> wrote: > >> I don't know about your garbage-collection question, but: >> >> > Why does that io:format/3 call (only when after the spawn/3 call) keep >> the socket open and active for the spawned process? I wouldn't expect it >> to be garbage collected because the spawned process holds a reference to it >> (and even adding it to the State in the return doesn't fix it). Only other >> thing I could think of was that handle_cast is run in it's own process >> which closes the socket when it dies (since it has ownership), but that >> doesn't make sense from what I've read about OTP. >> >> >> Normally handle_cast would be run in the gen_server's process (since >> that's the whole point), but it looks to me like in your test: >> >> >> > spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) >> end), >> >> >> you are bypassing gen_server and instead ... telling handle_cast to run >> in its own one-off process. >> >> >> Do you mean to be doing this? In your test code, you could call >> gen_server:start/1 and then gen_server:cast/2 and get the actual gen_server >> behavior. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Sun Mar 27 17:01:46 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Sun, 27 Mar 2016 16:01:46 +0100 Subject: [erlang-questions] erlang in docker from scratch Message-ID: TLDR; I package each release in a docker container - until recently using ?FROM debian? (or other distribution of choice) for each one, resulting in at least a ~125MB footprint. I have written a sample (https://github.com/shortishly/erlang-in-docker-from-scratch) application which automatically includes dynamic library dependencies from ERTS packaged in a docker container ?FROM scratch? with a minimal footprint of around 17MB. When packaging an application as a Docker container it is too easy to just be lazy and put "FROM debian" (other distributions are available, replace ?debian" with your distribution of choice). For sure it is going to work, but you have just included dozens of libraries and binaries that your application just does not need. An image that could be tens of megabytes is now at least several hundred ? we are building containers not virtual machines here! One of the things I like about Go is that typical application binaries are small with no runtime dependencies. Fewer dependencies mean less patching and security headaches. The less friction in the CI build cycle, the better. Go achieves this by having statically linked applications meaning that just one binary is necessary in ADD, and they are typically built from scratch (etcd as a good example): FROM scratch ADD etcd / ADD etcdctl / EXPOSE 4001 7001 2379 2380 ENTRYPOINT ["/etcd?] Assuming Linux and docker is already running - clone and build https://github.com/shortishly/erlang-in-docker-from-scratch repository, which contains a minimal erlang application that builds a release into the _rel directory (more detail with a slightly more Mac centric commands are https://blog.shortishly.com/2016/03/erlang-in-docker-from-scratch/): git clone https://github.com/shortishly/erlang-in-docker-from-scratch.git eidfs cd eidfs make At the end of make a standard erlang release for the eidfs application is now present in the _rel directory. To make it run inside a scratch container we need to include any runtime dependencies too - this is where mkimage comes in: ./bin/mkimage The ./bin/mkimage script copies in any dynamic libraries that ERTS needs to run the erlang release: '/lib/x86_64-linux-gnu/libc.so.6' -> '_rel/eidfs/lib/x86_64-linux-gnu/libc.so.6' '/lib/x86_64-linux-gnu/libdl.so.2' -> '_rel/eidfs/lib/x86_64-linux-gnu/libdl.so.2' '/lib/x86_64-linux-gnu/libm.so.6' -> '_rel/eidfs/lib/x86_64-linux-gnu/libm.so.6' '/lib/x86_64-linux-gnu/libpthread.so.0' -> '_rel/eidfs/lib/x86_64-linux-gnu/libpthread.so.0' '/lib/x86_64-linux-gnu/librt.so.1' -> '_rel/eidfs/lib/x86_64-linux-gnu/librt.so.1' '/lib/x86_64-linux-gnu/libtinfo.so.5' -> '_rel/eidfs/lib/x86_64-linux-gnu/libtinfo.so.5' '/lib/x86_64-linux-gnu/libutil.so.1' -> '_rel/eidfs/lib/x86_64-linux-gnu/libutil.so.1' '/lib/x86_64-linux-gnu/libz.so.1' -> '_rel/eidfs/lib/x86_64-linux-gnu/libz.so.1' It also copies /bin/sh so that we can run the release too. We can build a docker image for the release using the following command: docker build --build-arg REL_NAME=$(bin/release_name) --build-arg ERTS_VSN=$(bin/system_version) --pull=true --no-cache=true --force-rm=true -t $(bin/release_name):$(bin/version) . The $(bin/release_name), $(bin/system_version) and $(bin/version) are short escripts that respond with the release name, system ERTS version and the application version respectively. "docker images" and look at the size of the resultant container: REPOSITORY TAG IMAGE ID CREATED SIZE eidfs 0.0.1 6748931f94e4 4 seconds ago 16.74 MB We have a docker packaged erlang release in ~17MB. Lets run it: docker run --name $(bin/release_name) -d $(bin/release_name):$(bin/version) Check the logs using docker logs $(bin/release_name) and you will see the usual application startup messages from SASL. You might notice that the ENTRYPOINT used in the Dockerfile directly invokes erlexec. I have done this to reduce dependencies further so that the release, ERTS dynamic libraries, and /bin/bash only are present in the container. FROM scratch MAINTAINER Peter Morgan ARG REL_NAME ARG REL_VSN=1 ARG ERTS_VSN ENV BINDIR /erts-${ERTS_VSN}/bin ENV BOOT /releases/1/${REL_NAME} ENV CONFIG /releases/${REL_VSN}/sys.config ENV ARGS_FILE /releases/${REL_VSN}/vm.args ENV TZ=GMT ENTRYPOINT exec ${BINDIR}/erlexec -boot_var /lib -boot ${BOOT} -noinput -config ${CONFIG} -args_file ${ARGS_FILE} ADD _rel/${REL_NAME}/ / Much of the above is completely boilerplate, perhaps a erlang.mk ?bootstrap-docker? makes sense? Or are people using other/better methods to package applications in Docker that I have missed? Regards, Peter. From essen@REDACTED Sun Mar 27 17:37:01 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sun, 27 Mar 2016 17:37:01 +0200 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: References: Message-ID: <56F7FE1D.5010609@ninenines.eu> On 03/27/2016 05:01 PM, Peter Morgan wrote: > Much of the above is completely boilerplate, perhaps a erlang.mk ?bootstrap-docker? makes sense? Or are people using other/better methods to package applications in Docker that I have missed? I am totally open to that. What would be great, would be to be able to build on top of the release handling. If the project uses Docker, then "make" should build the container and "make run" should run the container (instead of building the release and running it). Does that make sense? If we do that, then the only other important thing is the bootstrap-docker target. Looking at your code, it sounds like you would benefit greatly from having Relx configuration built into the Makefile instead of as a separate file. But that can come as a second step. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From nathaniel@REDACTED Sun Mar 27 18:44:58 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Sun, 27 Mar 2016 12:44:58 -0400 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: References: Message-ID: > When packaging an application as a Docker container it is too easy to just be lazy and put "FROM debian" (other distributions are available, replace ?debian" with your distribution of choice). For sure it is going to work, but you have just included dozens of libraries and binaries that your application just does not need. An image that could be tens of megabytes is now at least several hundred ? we are building containers not virtual machines here! There was a news story last month about Docker moving to Alpine Linux as its default base for this reason. I just don't see it, though. I'm using Docker _because_ I'm lazy. If I wanted to not be lazy, I'd build a proper RPM or Debian package. Just for example: you've got libc on the host machine to run Docker, but you're also going to have libc on your image. So if saving a few MB is a big deal, then Docker was a bad decision. My thinking with Docker is, "I want to take a bunch of short-cuts, and I am willing to pay for them in disk space and a little CPU performance." So why should I care if my Erlang release has to drag along 500 MB of extra junk? Particularly when Docker's got this whole layers thing that works pretty nicely. I've got containers from 5 different images (Erlang, Erlang, Python 3, Node.js, Nginx) on one host and they're all lazily based on some 500 MB base image, which means that each image only has an amortized cost of 100 MB garbage. Are you doing embedded work? I saw a presentation about https://resin.io that uses Docker on IoT devices and I could see where you'd worry about space when you're running a single Docker container on a Raspberry Pi. But, again, Docker seems like a strange choice there. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From rvirding@REDACTED Sun Mar 27 23:43:58 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 27 Mar 2016 23:43:58 +0200 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: To be even more specific all the 6 gen_server callbacks[*] must always be run within the gen_server process, this irrespective of whether you are testing or not. Robert [*] init/1, terminate/2, handle_call/3, handle-cast/2, handle_info/2, code_change/3. On 27 March 2016 at 15:13, Matthew Shapiro wrote: > Well duh, now I feel like an idiot. > > Thanks, > > On Sun, Mar 27, 2016 at 5:31 AM, Vimal Kumar wrote: > >> Matthew, >> >> Like Nathaniel said, the eunit test should involve gen_server. Do not >> test handle_cast/2 directly. Instead, run your test on an exported function >> (say: Module:accept/0) which in turn should execute: >> >> gen_server:cast(?MODULE, accept) >> >> You can start a gen_server process, execute such tests and shut it down >> properly using 'setup' Fixture: >> http://learnyousomeerlang.com/eunit#fixtures >> >> Regards, >> Vimal >> >> On Sun, Mar 27, 2016 at 10:15 AM, Nathaniel Waisbrot < >> nathaniel@REDACTED> wrote: >> >>> I don't know about your garbage-collection question, but: >>> >>> > Why does that io:format/3 call (only when after the spawn/3 call) keep >>> the socket open and active for the spawned process? I wouldn't expect it >>> to be garbage collected because the spawned process holds a reference to it >>> (and even adding it to the State in the return doesn't fix it). Only other >>> thing I could think of was that handle_cast is run in it's own process >>> which closes the socket when it dies (since it has ownership), but that >>> doesn't make sense from what I've read about OTP. >>> >>> >>> Normally handle_cast would be run in the gen_server's process (since >>> that's the whole point), but it looks to me like in your test: >>> >>> >>> > spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) >>> end), >>> >>> >>> you are bypassing gen_server and instead ... telling handle_cast to run >>> in its own one-off process. >>> >>> >>> Do you mean to be doing this? In your test code, you could call >>> gen_server:start/1 and then gen_server:cast/2 and get the actual gen_server >>> behavior. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Mon Mar 28 03:34:16 2016 From: me@REDACTED (Matthew Shapiro) Date: Sun, 27 Mar 2016 21:34:16 -0400 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: Why is that? I was under the impression that one of the advantages was that you can test the handle call/cast/info functions without having to stand up a full server to do a small surface area unit test. On Sun, Mar 27, 2016 at 5:43 PM, Robert Virding wrote: > To be even more specific all the 6 gen_server callbacks[*] must always be > run within the gen_server process, this irrespective of whether you are > testing or not. > > Robert > > [*] init/1, terminate/2, handle_call/3, handle-cast/2, handle_info/2, > code_change/3. > > On 27 March 2016 at 15:13, Matthew Shapiro wrote: > >> Well duh, now I feel like an idiot. >> >> Thanks, >> >> On Sun, Mar 27, 2016 at 5:31 AM, Vimal Kumar wrote: >> >>> Matthew, >>> >>> Like Nathaniel said, the eunit test should involve gen_server. Do not >>> test handle_cast/2 directly. Instead, run your test on an exported function >>> (say: Module:accept/0) which in turn should execute: >>> >>> gen_server:cast(?MODULE, accept) >>> >>> You can start a gen_server process, execute such tests and shut it down >>> properly using 'setup' Fixture: >>> http://learnyousomeerlang.com/eunit#fixtures >>> >>> Regards, >>> Vimal >>> >>> On Sun, Mar 27, 2016 at 10:15 AM, Nathaniel Waisbrot < >>> nathaniel@REDACTED> wrote: >>> >>>> I don't know about your garbage-collection question, but: >>>> >>>> > Why does that io:format/3 call (only when after the spawn/3 call) >>>> keep the socket open and active for the spawned process? I wouldn't expect >>>> it to be garbage collected because the spawned process holds a reference to >>>> it (and even adding it to the State in the return doesn't fix it). Only >>>> other thing I could think of was that handle_cast is run in it's own >>>> process which closes the socket when it dies (since it has ownership), but >>>> that doesn't make sense from what I've read about OTP. >>>> >>>> >>>> Normally handle_cast would be run in the gen_server's process (since >>>> that's the whole point), but it looks to me like in your test: >>>> >>>> >>>> > spawn_link(fun () -> tcp_listener_server:handle_cast(accept, State) >>>> end), >>>> >>>> >>>> you are bypassing gen_server and instead ... telling handle_cast to run >>>> in its own one-off process. >>>> >>>> >>>> Do you mean to be doing this? In your test code, you could call >>>> gen_server:start/1 and then gen_server:cast/2 and get the actual gen_server >>>> behavior. >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Mon Mar 28 04:13:03 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 28 Mar 2016 11:13:03 +0900 Subject: [erlang-questions] Why does adding an io:format call make my accept socket remain valid? In-Reply-To: References: Message-ID: <1856922.K0tsxklpqD@changa> On 2016?3?27? ??? 21:34:16 Matthew Shapiro wrote: > Why is that? I was under the impression that one of the advantages was > that you can test the handle call/cast/info functions without having to > stand up a full server to do a small surface area unit test. You could do test the callbacks in isolation, but the callbacks themselves should be so tiny that they don't mean anything without the OTP machinery. Most of the time you shouldn't be putting a lot of logic into your handle_* function clauses. You want the handle_* functions to be doing just one thing: handling the message. Plagiarizing from a marginally related post I wrote last week: spawn_mob(Options) -> gen_server:cast(?MODULE, {spawn_mob, Options}). %... handle_cast({spawn_mob, Options}, State) -> ok = do_spawn_mob(Options, State), {noreply, State}; % ... do_spawn_mob(Options, State = #s{stuff = Stuff}) -> % Actually do work in the `do_*` functions down here % Side effects go here, OtherStuff = some_pure_function(Stuff) the_target_effect(OtherStuff). You do NOT want this: handle_cast({spawn_mob, Options}, State = #s{stuff = Stuff}) -> % A few lines of stuff % Shoving everything into *radically* different clauses % across your handle_*, making it a constantly morphing % and difficult to navigate structure of its own over time. OtherStuff = select_something(fun(X) -> Continue = case check_mob(X, State) of good -> % WHY DO PEOPLE ok; % DO THIS TO bad -> % THEMSELVES?!? reset(X) end, Options, Stuff), ok = the_target_effect(OtherStuff), {noreply, State}; handle_cast({register, Player}, State) -> % A bunch of lines of details completely unrelated to % mob spawning. ... Why this breakdown? Because in the first case we have broken the concerns of each function down into separate elements, concentrated the side effects in a single spot (the do_* parts), can now test each pure function on its own, know for sure what has a side effect that needs to be tested in the context of a more complete environment, and have the handle_* functions doing exactly one thing: dispatching based on received messages. This way is testable. The other way isn't. A few quick hacks with long fun definitions in argument lists which themselves contain cases or other funs within the the handle_* clauses is all it takes to turn an easily understandable and maintainable gen_server into the kind of code that actually makes you *feel* bad when you open it to make an edit. As for testing... [Digression ahead! Heresy warning!] I occasionally go as far as to write a test/0 that returns ok or crashes when called, and all it does is call tests for each pure function in the module -- just the pure functions. This gives me a motivation to isolate side effects, and prevents the case I see so often in older commercial products where the a test/ or tests/ directory is chock full of test modules that don't really describe the code at all anymore, and nobody has the time to go pick through there and sort out what is useful, what is dead code and what tests are just flat out useless. By including tests only for pure functions within the module itself I know they are tests of value, not effect, and that they should be written to define clear bounds to the function and all the other strict things you can do with a pure function. Outside of that I can also know that everything in a test/ directory should test the *module* as it would exist within a running system, to include the OTP stuff above it, since that is part of the process the module represents. On that note, testing beyond the module can usually only tell you a little bit of what you want to know about the system running as a whole in production. I've found that integration testing is generally not as useful as actual user testing once the big pieces reach that "works for me" stage throughout the dev team. But this means you have to stand up an actual (gasp!) test infrastructure that is identical to production and stands parallel to it. Which most folks seem to not have the patience for ("We have tests, I don't see how that buys us much" and so on), and requires that you are prepared for people to do things in your system you didn't expect (which was going to be the case once its live anyway, so wtf?!?). [/digression /heresy] -Craig From askjuise@REDACTED Mon Mar 28 11:02:54 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Mon, 28 Mar 2016 12:02:54 +0300 Subject: [erlang-questions] rebar3 and not nightly build Message-ID: Hi! I wonder, why I can download only nightly build of rebar3, instead of getting binary stable version without building it from source? -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Mon Mar 28 11:15:07 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Mon, 28 Mar 2016 12:15:07 +0300 Subject: [erlang-questions] rebar3 and ct In-Reply-To: References: Message-ID: Unfortunately it's doesn't work: -{ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, > all}]}. > +{ct_opts, [{dir, "./test"}]}. I have the same problem: ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam > /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl > /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml > "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") > failed with return code 1 and the following output: > cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such > file or directory and looks like it becasu the file TEST-file_myapp.app.xml have name TEST-file_"myapp.app".xml 2016-03-24 17:37 GMT+03:00 alisdair sullivan : > rebar3 doesn?t currently support the setting on test configuration via > `ct_opts` like that. changing your `ct_opts` section to: `[{dir, ?test?}, > {logdir, ?logs?}]` should work > > On Mar 24, 2016, at 2:34 AM, Alexander Petrovsky > wrote: > > Hi! > > I have the follow ct section in rebar.config: > > {ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, >> all}]}. >> {ct_compile_opts, [{i, "./include/"}]}. > > > On the "rebar 3.0.0-beta.3+build.327.ref97ae8d6 on Erlang/OTP 18 Erts 7.0" > it's work perfectly, but on the "rebar 3.0.0+build.273.refd2de55d on > Erlang/OTP 18 Erts 7.0" it says: > > ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam >> /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl >> /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml >> "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") >> failed with return code 1 and the following output: >> cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such >> file or directory > > > and after "rm -rf ./test/TEST*.xml" it says: > > ./rebar3 ct skip_deps=true verbose=3 >> ===> Verifying dependencies... >> ===> Member `{alias,myapp,"./test/"}' of option `ct_opts' must be a >> 2-tuple >> make: *** [test] Error 1 > > > How can I fix it? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Mon Mar 28 14:35:44 2016 From: t@REDACTED (Tristan Sloughter) Date: Mon, 28 Mar 2016 07:35:44 -0500 Subject: [erlang-questions] rebar3 and not nightly build In-Reply-To: References: Message-ID: <1459168544.1676027.561369458.2000ECF9@webmail.messagingengine.com> Sorry, not yet. I've been meaning to get the travis script updated to build and upload tags like it does each commits so we can link to the current stable version. I'll try to get that done soon. -- Tristan Sloughter t@REDACTED On Mon, Mar 28, 2016, at 04:02 AM, Alexander Petrovsky wrote: > Hi! > > I wonder, why I can download only nightly build of rebar3, instead of > getting binary stable version without building it from source? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisdairsullivan@REDACTED Mon Mar 28 14:58:36 2016 From: alisdairsullivan@REDACTED (alisdair s) Date: Mon, 28 Mar 2016 05:58:36 -0700 Subject: [erlang-questions] rebar3 and ct In-Reply-To: References: Message-ID: <551460D4-59FB-4B62-81B5-A6F440891459@yahoo.ca> what os are you using? osx? this looks like a bug handling files with quotes in the name, i will investigate and open an issue Sent from my iPad > On Mar 28, 2016, at 2:15 AM, Alexander Petrovsky wrote: > > Unfortunately it's doesn't work: > >> -{ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, all}]}. >> +{ct_opts, [{dir, "./test"}]}. > > > I have the same problem: > >> ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") >> failed with return code 1 and the following output: >> cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such file or directory > > > and looks like it becasu the file TEST-file_myapp.app.xml have name TEST-file_"myapp.app".xml > > 2016-03-24 17:37 GMT+03:00 alisdair sullivan : >> rebar3 doesn?t currently support the setting on test configuration via `ct_opts` like that. changing your `ct_opts` section to: `[{dir, ?test?}, {logdir, ?logs?}]` should work >> >>> On Mar 24, 2016, at 2:34 AM, Alexander Petrovsky wrote: >>> >>> Hi! >>> >>> I have the follow ct section in rebar.config: >>> >>>> {ct_opts, [{alias, myapp, "./test/"}, {logdir, "./logs/"}, {suites, myapp, all}]}. >>>> {ct_compile_opts, [{i, "./include/"}]}. >>> >>> On the "rebar 3.0.0-beta.3+build.327.ref97ae8d6 on Erlang/OTP 18 Erts 7.0" it's work perfectly, but on the "rebar 3.0.0+build.273.refd2de55d on Erlang/OTP 18 Erts 7.0" it says: >>> >>>> ===> sh(cp -Rp /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.beam /Users/juise/Documents/myapp/test/myapp_shaper_SUITE.erl /Users/juise/Documents/myapp/test/TEST-file_"myapp.app".xml "/Users/juise/Documents/myapp/_build/test/lib/myapp/test") >>>> failed with return code 1 and the following output: >>>> cp: /Users/juise/Documents/myapp/test/TEST-file_myapp.app.xml: No such file or directory >>> >>> >>> and after "rm -rf ./test/TEST*.xml" it says: >>> >>>> ./rebar3 ct skip_deps=true verbose=3 >>>> ===> Verifying dependencies... >>>> ===> Member `{alias,myapp,"./test/"}' of option `ct_opts' must be a 2-tuple >>>> make: *** [test] Error 1 >>> >>> >>> How can I fix it? >>> >>> -- >>> ?????????? ????????? / Alexander Petrovsky, >>> >>> Skype: askjuise >>> Phone: +7 914 8 820 815 >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Mon Mar 28 17:51:18 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Mon, 28 Mar 2016 16:51:18 +0100 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: <56F7FE1D.5010609@ninenines.eu> References: <56F7FE1D.5010609@ninenines.eu> Message-ID: <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> > On 27 Mar 2016, at 16:37, Lo?c Hoguin wrote: > > On 03/27/2016 05:01 PM, Peter Morgan wrote: >> Much of the above is completely boilerplate, perhaps a erlang.mk ?bootstrap-docker? makes sense? Or are people using other/better methods to package applications in Docker that I have missed? > > I am totally open to that. > > What would be great, would be to be able to build on top of the release handling. If the project uses Docker, then "make" should build the container and "make run" should run the container (instead of building the release and running it). Does that make sense? > > If we do that, then the only other important thing is the bootstrap-docker target. Awesome - I?ve forked. I have some changes, they need a bunch more testing - they?re not going to work on anything other than Linux at the moment. I?ll look at OSX, but that will mean getting a Linux ERTS from somewhere into the container somehow - probably ?FROM erlang? - but that is enormous. I?ve added a docker plugin which is weaved it into the ?rel? target, with a ?bootstrap-docker? to bootstrap.mk - it probably could do with a ?.dockerignore? in there too, and some help. The dynamic libraries probably should be copied using something like http://stackoverflow.com/questions/16929445/makefile-dynamic-rules-based-on-a-file - but I?ve struggled with that - everything being .PHONY seems wrong? The below works for me on Fedora: mkdir demo cd demo # fetch my fork wget -q https://github.com/shortishly/erlang.mk/raw/master/erlang.mk # initial bootstrap, release and docker make -f erlang.mk bootstrap make bootstrap-rel make bootstrap-docker # build the release and the docker image make ===> Starting relx build process ... ===> Resolving OTP Applications from directories: /home/pmorgan/src/git/demo/ebin /home/pmorgan/opt/erlang/18.2.1/lib /home/pmorgan/src/git/demo/apps /home/pmorgan/src/git/demo/deps ===> Resolved demo_release-1 ===> Including Erts from /home/pmorgan/opt/erlang/18.2.1 ===> release successfully created! GEN docker-scratch-cp-dynamic-libs GEN docker-scratch-cp-link-loader GEN docker-scratch-cp-sh GEN docker-strip-erts-binaries GEN docker-build sha256:76a9383ada7b25dce142880bf321f6399e738a82a0bb4c31811b2040b2219077 # docker image built from scratch image docker images REPOSITORY TAG IMAGE ID CREATED SIZE demo_release 0.0.1 76a9383ada7b 2 minutes ago 22.02 MB # nothing running in docker docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES # ?docker-run? will kill any existing run for that application and start a new one: make docker-run GEN docker-rm GEN docker-run 9842e2831927e2717c461c6b0a71ac6acf6f364c1077ce2e42805d360cef496d # container is running named after the release docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9842e2831927 demo_release:0.0.1 "/bin/sh -c 'exec ${B" 2 minutes ago Up 2 minutes demo_release # ?docker-logs? is a shortcut to get the logs for the container: make docker-logs GEN docker-logs heart_beat_kill_pid = 1 > Looking at your code, it sounds like you would benefit greatly from having Relx configuration built into the Makefile instead of as a separate file. But that can come as a second step. Yes, that would make sense too. Having some more PROJECT_XYZ variable that describe the release, would be great. Also - should ?-heart? be default in vm.args? Heart is kind of redundant with docker ?restart=always and breaks the one-process-per-container rule (similar for epmd). Thanks, Peter. From roberto@REDACTED Mon Mar 28 19:32:19 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 28 Mar 2016 19:32:19 +0200 Subject: [erlang-questions] Erlang's Logo license Message-ID: All, Can someone point me where I can find the Erlang's logo license? I'd like to know how it can (eventually) be used, and how it cannot. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From son@REDACTED Mon Mar 28 21:31:51 2016 From: son@REDACTED (Son Tran-Nguyen) Date: Mon, 28 Mar 2016 14:31:51 -0500 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> References: <56F7FE1D.5010609@ninenines.eu> <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> Message-ID: Having tried Docker with Erlang, I have a question about hot code swapping. Does using Docker get rid of that benefit? It seems to me that every Docker image will be a new release. On Mon, Mar 28, 2016 at 10:51 AM, Peter Morgan wrote: > > > On 27 Mar 2016, at 16:37, Lo?c Hoguin wrote: > > > > On 03/27/2016 05:01 PM, Peter Morgan wrote: > >> Much of the above is completely boilerplate, perhaps a erlang.mk > ?bootstrap-docker? makes sense? Or are people using other/better methods to > package applications in Docker that I have missed? > > > > I am totally open to that. > > > > What would be great, would be to be able to build on top of the release > handling. If the project uses Docker, then "make" should build the > container and "make run" should run the container (instead of building the > release and running it). Does that make sense? > > > > If we do that, then the only other important thing is the > bootstrap-docker target. > > > Awesome - I?ve forked. I have some changes, they need a bunch more testing > - they?re not going to work on anything other than Linux at the moment. > I?ll look at OSX, but that will mean getting a Linux ERTS from somewhere > into the container somehow - probably ?FROM erlang? - but that is enormous. > > I?ve added a docker plugin which is weaved it into the ?rel? target, with > a ?bootstrap-docker? to bootstrap.mk - it probably could do with a > ?.dockerignore? in there too, and some help. The dynamic libraries probably > should be copied using something like > http://stackoverflow.com/questions/16929445/makefile-dynamic-rules-based-on-a-file > - but I?ve struggled with that - everything being .PHONY seems wrong? > > The below works for me on Fedora: > > mkdir demo > cd demo > > # fetch my fork > wget -q https://github.com/shortishly/erlang.mk/raw/master/erlang.mk > > # initial bootstrap, release and docker > make -f erlang.mk bootstrap > make bootstrap-rel > make bootstrap-docker > > # build the release and the docker image > make > ===> Starting relx build process ... > ===> Resolving OTP Applications from directories: > /home/pmorgan/src/git/demo/ebin > /home/pmorgan/opt/erlang/18.2.1/lib > /home/pmorgan/src/git/demo/apps > /home/pmorgan/src/git/demo/deps > ===> Resolved demo_release-1 > ===> Including Erts from /home/pmorgan/opt/erlang/18.2.1 > ===> release successfully created! > GEN docker-scratch-cp-dynamic-libs > GEN docker-scratch-cp-link-loader > GEN docker-scratch-cp-sh > GEN docker-strip-erts-binaries > GEN docker-build > sha256:76a9383ada7b25dce142880bf321f6399e738a82a0bb4c31811b2040b2219077 > > # docker image built from scratch image > docker images > REPOSITORY TAG IMAGE ID > CREATED SIZE > demo_release 0.0.1 76a9383ada7b 2 > minutes ago 22.02 MB > > # nothing running in docker > docker ps -a > CONTAINER ID IMAGE COMMAND CREATED > STATUS PORTS NAMES > > # ?docker-run? will kill any existing run for that application and start a > new one: > make docker-run > GEN docker-rm > GEN docker-run > 9842e2831927e2717c461c6b0a71ac6acf6f364c1077ce2e42805d360cef496d > > # container is running named after the release > docker ps -a > CONTAINER ID IMAGE COMMAND CREATED > STATUS PORTS NAMES > 9842e2831927 demo_release:0.0.1 "/bin/sh -c 'exec ${B" 2 > minutes ago Up 2 minutes demo_release > > # ?docker-logs? is a shortcut to get the logs for the container: > make docker-logs > GEN docker-logs > heart_beat_kill_pid = 1 > > > Looking at your code, it sounds like you would benefit greatly from > having Relx configuration built into the Makefile instead of as a separate > file. But that can come as a second step. > > Yes, that would make sense too. Having some more PROJECT_XYZ variable that > describe the release, would be great. > > Also - should ?-heart? be default in vm.args? Heart is kind of redundant > with docker ?restart=always and breaks the one-process-per-container rule > (similar for epmd). > > Thanks, > Peter. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Mar 28 21:37:32 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 28 Mar 2016 21:37:32 +0200 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> References: <56F7FE1D.5010609@ninenines.eu> <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> Message-ID: <56F987FC.9010204@ninenines.eu> On 03/28/2016 05:51 PM, Peter Morgan wrote: > >> On 27 Mar 2016, at 16:37, Lo?c Hoguin wrote: >> >> On 03/27/2016 05:01 PM, Peter Morgan wrote: >>> Much of the above is completely boilerplate, perhaps a erlang.mk ?bootstrap-docker? makes sense? Or are people using other/better methods to package applications in Docker that I have missed? >> >> I am totally open to that. >> >> What would be great, would be to be able to build on top of the release handling. If the project uses Docker, then "make" should build the container and "make run" should run the container (instead of building the release and running it). Does that make sense? >> >> If we do that, then the only other important thing is the bootstrap-docker target. > > > Awesome - I?ve forked. I have some changes, they need a bunch more testing - they?re not going to work on anything other than Linux at the moment. I?ll look at OSX, but that will mean getting a Linux ERTS from somewhere into the container somehow - probably ?FROM erlang? - but that is enormous. > > I?ve added a docker plugin which is weaved it into the ?rel? target, with a ?bootstrap-docker? to bootstrap.mk - it probably could do with a ?.dockerignore? in there too, and some help. The dynamic libraries probably should be copied using something like http://stackoverflow.com/questions/16929445/makefile-dynamic-rules-based-on-a-file - but I?ve struggled with that - everything being .PHONY seems wrong? Don't be worried about details. Most important is having tests and that they pass. We can then improve things easily. > The below works for me on Fedora: Must be some issues, got: strip $(file _rel/*/erts-*/bin/*|grep "not stripped"|awk '{print $1}'|cut -d: -f1) &>/dev/null erlang.mk:6634: recipe for target 'docker-strip-erts-binaries' failed make: [docker-strip-erts-binaries] Error 1 (ignored) And: $ make docker-run GEN docker-rm GEN docker-run 5af98425557abc276483dca047bb711fceb86e929a6cd022ca93d8344893ed6f $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5af98425557a dockerdemo_release:0.0.1 "/bin/sh -c 'exec ${B" 3 seconds ago Exited (127) 3 seconds ago dockerdemo_release > Also - should ?-heart? be default in vm.args? Heart is kind of redundant with docker ?restart=always and breaks the one-process-per-container rule (similar for epmd). -heart makes sense for releases, although it really depends on how you run them. It's better to have it as a default and have people remove it knowingly I think. -heart should on the other hand always be stripped when using Docker I guess. Not too sure what's the best for epmd though. It can probably be ironed out with user feedback. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From santiagopoli@REDACTED Tue Mar 29 01:42:43 2016 From: santiagopoli@REDACTED (Santiago Ignacio Poli) Date: Mon, 28 Mar 2016 20:42:43 -0300 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: References: <56F7FE1D.5010609@ninenines.eu> <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> Message-ID: I thought the same thing when I was considering using Erlang in Docker, and I think it won't be possible to do hot code swapping with this approach. The only solution is to mount the code as an external volume and hot swapping it. In that case you will need to have an image for your environment and another one for your code. 2016-03-28 16:31 GMT-03:00 Son Tran-Nguyen : > Having tried Docker with Erlang, I have a question about hot code > swapping. > Does using Docker get rid of that benefit? It seems to me that every Docker > image will be a new release. > > On Mon, Mar 28, 2016 at 10:51 AM, Peter Morgan < > peter.james.morgan@REDACTED> wrote: > >> >> > On 27 Mar 2016, at 16:37, Lo?c Hoguin wrote: >> > >> > On 03/27/2016 05:01 PM, Peter Morgan wrote: >> >> Much of the above is completely boilerplate, perhaps a erlang.mk >> ?bootstrap-docker? makes sense? Or are people using other/better methods to >> package applications in Docker that I have missed? >> > >> > I am totally open to that. >> > >> > What would be great, would be to be able to build on top of the release >> handling. If the project uses Docker, then "make" should build the >> container and "make run" should run the container (instead of building the >> release and running it). Does that make sense? >> > >> > If we do that, then the only other important thing is the >> bootstrap-docker target. >> >> >> Awesome - I?ve forked. I have some changes, they need a bunch more >> testing - they?re not going to work on anything other than Linux at the >> moment. I?ll look at OSX, but that will mean getting a Linux ERTS from >> somewhere into the container somehow - probably ?FROM erlang? - but that is >> enormous. >> >> I?ve added a docker plugin which is weaved it into the ?rel? target, with >> a ?bootstrap-docker? to bootstrap.mk - it probably could do with a >> ?.dockerignore? in there too, and some help. The dynamic libraries probably >> should be copied using something like >> http://stackoverflow.com/questions/16929445/makefile-dynamic-rules-based-on-a-file >> - but I?ve struggled with that - everything being .PHONY seems wrong? >> >> The below works for me on Fedora: >> >> mkdir demo >> cd demo >> >> # fetch my fork >> wget -q https://github.com/shortishly/erlang.mk/raw/master/erlang.mk >> >> # initial bootstrap, release and docker >> make -f erlang.mk bootstrap >> make bootstrap-rel >> make bootstrap-docker >> >> # build the release and the docker image >> make >> ===> Starting relx build process ... >> ===> Resolving OTP Applications from directories: >> /home/pmorgan/src/git/demo/ebin >> /home/pmorgan/opt/erlang/18.2.1/lib >> /home/pmorgan/src/git/demo/apps >> /home/pmorgan/src/git/demo/deps >> ===> Resolved demo_release-1 >> ===> Including Erts from /home/pmorgan/opt/erlang/18.2.1 >> ===> release successfully created! >> GEN docker-scratch-cp-dynamic-libs >> GEN docker-scratch-cp-link-loader >> GEN docker-scratch-cp-sh >> GEN docker-strip-erts-binaries >> GEN docker-build >> sha256:76a9383ada7b25dce142880bf321f6399e738a82a0bb4c31811b2040b2219077 >> >> # docker image built from scratch image >> docker images >> REPOSITORY TAG IMAGE ID >> CREATED SIZE >> demo_release 0.0.1 76a9383ada7b 2 >> minutes ago 22.02 MB >> >> # nothing running in docker >> docker ps -a >> CONTAINER ID IMAGE COMMAND CREATED >> STATUS PORTS NAMES >> >> # ?docker-run? will kill any existing run for that application and start >> a new one: >> make docker-run >> GEN docker-rm >> GEN docker-run >> 9842e2831927e2717c461c6b0a71ac6acf6f364c1077ce2e42805d360cef496d >> >> # container is running named after the release >> docker ps -a >> CONTAINER ID IMAGE COMMAND >> CREATED STATUS PORTS NAMES >> 9842e2831927 demo_release:0.0.1 "/bin/sh -c 'exec ${B" 2 >> minutes ago Up 2 minutes demo_release >> >> # ?docker-logs? is a shortcut to get the logs for the container: >> make docker-logs >> GEN docker-logs >> heart_beat_kill_pid = 1 >> >> > Looking at your code, it sounds like you would benefit greatly from >> having Relx configuration built into the Makefile instead of as a separate >> file. But that can come as a second step. >> >> Yes, that would make sense too. Having some more PROJECT_XYZ variable >> that describe the release, would be great. >> >> Also - should ?-heart? be default in vm.args? Heart is kind of redundant >> with docker ?restart=always and breaks the one-process-per-container rule >> (similar for epmd). >> >> Thanks, >> Peter. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Santiago Poli -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Mar 29 07:38:00 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 29 Mar 2016 06:38:00 +0100 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: References: <56F7FE1D.5010609@ninenines.eu> <05997A68-66B7-42E4-AA7E-3945FBC19AAA@gmail.com> Message-ID: <089A8633-AB0C-4EC4-9337-3BF0EBE7FEA6@gmail.com> > On 29 Mar 2016, at 00:42, Santiago Ignacio Poli wrote: > > I thought the same thing when I was considering using Erlang in Docker, and I think it won't be possible to do hot code swapping with this approach. The only solution is to mount the code as an external volume and hot swapping it. In that case you will need to have an image for your environment and another one for your code. > > 2016-03-28 16:31 GMT-03:00 Son Tran-Nguyen >: > Having tried Docker with Erlang, I have a question about hot code swapping. > Does using Docker get rid of that benefit? It seems to me that every Docker > image will be a new release. My workflow: in local development I sit and write code in a ?make shell", until it mostly works. Turning thoughts into modules, when they get more mature, I start using make:all([load]) together with an Emakefile - quick and low ceremony. Once I have a ?feature? - I commit, my CI automatically builds a release, creates and pushes a new docker image to a registry - part of the CI also pings the docker API on the integration environment to deploy and bounce the new services individually which are each hidden behind a HTTP software load balancer. The few volume mounts are for persistent data in those services, but mostly they are stateless and rely on Kafka topics to replenish their state or some other SOR. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Mar 29 08:46:34 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 29 Mar 2016 07:46:34 +0100 Subject: [erlang-questions] erlang in docker from scratch In-Reply-To: References: Message-ID: <77AA1AA5-5A69-4DAC-98BB-33B812B084C9@gmail.com> > On 27 Mar 2016, at 17:44, Nathaniel Waisbrot wrote: > >> When packaging an application as a Docker container it is too easy to just be lazy and put "FROM debian" (other distributions are available, replace ?debian" with your distribution of choice). For sure it is going to work, but you have just included dozens of libraries and binaries that your application just does not need. An image that could be tens of megabytes is now at least several hundred ? we are building containers not virtual machines here! > > > There was a news story last month about Docker moving to Alpine Linux as its default base for this reason. > > I just don't see it, though. I'm using Docker _because_ I'm lazy. If I wanted to not be lazy, I'd build a proper RPM or Debian package. Just for example: you've got libc on the host machine to run Docker, but you're also going to have libc on your image. So if saving a few MB is a big deal, then Docker was a bad decision. > > My thinking with Docker is, "I want to take a bunch of short-cuts, and I am willing to pay for them in disk space and a little CPU performance." So why should I care if my Erlang release has to drag along 500 MB of extra junk? Particularly when Docker's got this whole layers thing that works pretty nicely. I've got containers from 5 different images (Erlang, Erlang, Python 3, Node.js, Nginx) on one host and they're all lazily based on some 500 MB base image, which means that each image only has an amortized cost of 100 MB garbage. In development I completely agree - do the simplest thing that works, and it doesn?t matter if it is 50MB or 500MB - it is more important to demonstrate that it can be done and quickly. However, I think for the reasons that you outlined in the Alpine Linux story (https://news.ycombinator.com/item?id=10998667) - it quickly matters when people start picking a web tier from (say) Nginx that also includes a whole lump of ubuntu/debian/?, which is also then combined with a local middle tier that is also includes a whole lump of rhel/centos/sles/oracle linux . Containers quickly become like virtual machines, with a whole bunch of issues around patching and security that any Corporate is going to care deeply about. In my original post I said that I was influenced by the way that Go applications are packaged. A move to Alpine will help make images smaller, but I wasn?t sure about support for musl versus glibc (http://erlang.org/pipermail/erlang-questions/2015-July/085344.html) with erlang? The official erlang docker image (https://hub.docker.com/_/erlang/) is based on debian:jessie at 747MB (the same size as Go?s) . Go statically links, so you can usually just copy a single binary into a ?FROM scratch? image. I?ve not tried - but static linking erlang doesn't seem to be an option (http://erlang.org/pipermail/erlang-questions/2011-June/059231.html) and probably doesn?t make sense because there is more than one binary anyway (erlexec, beam, beam.smp, etc). So, in Erlang you either need to ?FROM debian:jessie? to ensure you get the same runtime libraries you built with - which means an image size of at least 278MB (until Alpine comes along) or ?FROM scratch? and copy with the runtime libraries with the release? Or not use Docker... > Are you doing embedded work? I saw a presentation about https://resin.io that uses Docker on IoT devices and I could see where you'd worry about space when you're running a single Docker container on a Raspberry Pi. But, again, Docker seems like a strange choice there. I thinking of the opposite scale - pushing a 50MB versus 500MB image to tens/hundreds of ephemeral nodes in a Swarm. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Mar 29 13:27:42 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 29 Mar 2016 13:27:42 +0200 Subject: [erlang-questions] Erlang's Logo license In-Reply-To: References: Message-ID: To add to this: the Erlang logo can be found on the WikiMedia page here: https://commons.wikimedia.org/wiki/File:Erlang_logo.png That pages states that: This image only consists of simple geometric shapes or text. It does not meet the threshold of originality needed for copyright protection, and is therefore in the public domain. Although it is free of copyright restrictions, this image may still be subject to other restrictions. See WP:PD#Fonts and typefaces or Template talk:PD-textlogo for more information. However, it may still be protected. Anyone knows who to contact? Best, r. On Mon, Mar 28, 2016 at 7:32 PM, Roberto Ostinelli wrote: > All, > Can someone point me where I can find the Erlang's logo license? > I'd like to know how it can (eventually) be used, and how it cannot. > > Thank you, > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Mar 29 13:51:22 2016 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 29 Mar 2016 12:51:22 +0100 Subject: [erlang-questions] Erlang's Logo license In-Reply-To: References: Message-ID: On 29 March 2016 at 12:27, Roberto Ostinelli wrote: > To add to this: the Erlang logo can be found on the WikiMedia page here: > https://commons.wikimedia.org/wiki/File:Erlang_logo.png > > That pages states that: > > This image only consists of simple geometric shapes or text. It does not > meet the threshold of originality needed for copyright protection, and is > therefore in the public domain. Although it is free of copyright > restrictions, this image may still be subject to other restrictions. See > WP:PD#Fonts and typefaces or Template talk:PD-textlogo for more information. That merely covers copyright. It doesn't cover trademark. From roger@REDACTED Tue Mar 29 13:55:35 2016 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 29 Mar 2016 12:55:35 +0100 Subject: [erlang-questions] Erlang's Logo license In-Reply-To: References: Message-ID: I found this: http://bugs.erlang.org/browse/ERL-41 "The name "Erlang" is a trademark of Ericsson while the logos have not yet been registered. It is okay for you to use the name Erlang and the unaltered Erlang logo in your presentation as long as it is a nominative use, i.e. the user does nothing to suggest sponsorship or endorsement by the trademark holder." On 29 March 2016 at 12:51, Roger Lipscombe wrote: > On 29 March 2016 at 12:27, Roberto Ostinelli wrote: >> To add to this: the Erlang logo can be found on the WikiMedia page here: >> https://commons.wikimedia.org/wiki/File:Erlang_logo.png >> >> That pages states that: >> >> This image only consists of simple geometric shapes or text. It does not >> meet the threshold of originality needed for copyright protection, and is >> therefore in the public domain. Although it is free of copyright >> restrictions, this image may still be subject to other restrictions. See >> WP:PD#Fonts and typefaces or Template talk:PD-textlogo for more information. > > That merely covers copyright. It doesn't cover trademark. From roberto@REDACTED Tue Mar 29 15:34:24 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 29 Mar 2016 15:34:24 +0200 Subject: [erlang-questions] Erlang's Logo license In-Reply-To: References: Message-ID: Thank you Roger! On Tue, Mar 29, 2016 at 1:55 PM, Roger Lipscombe wrote: > I found this: http://bugs.erlang.org/browse/ERL-41 > > "The name "Erlang" is a trademark of Ericsson while the logos have not > yet been registered. It is okay for you to use the name Erlang and the > unaltered Erlang logo in your presentation as long as it is a > nominative use, i.e. the user does nothing to suggest sponsorship or > endorsement by the trademark holder." > > > On 29 March 2016 at 12:51, Roger Lipscombe wrote: > > On 29 March 2016 at 12:27, Roberto Ostinelli > wrote: > >> To add to this: the Erlang logo can be found on the WikiMedia page here: > >> https://commons.wikimedia.org/wiki/File:Erlang_logo.png > >> > >> That pages states that: > >> > >> This image only consists of simple geometric shapes or text. It does not > >> meet the threshold of originality needed for copyright protection, and > is > >> therefore in the public domain. Although it is free of copyright > >> restrictions, this image may still be subject to other restrictions. See > >> WP:PD#Fonts and typefaces or Template talk:PD-textlogo for more > information. > > > > That merely covers copyright. It doesn't cover trademark. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Mar 29 15:44:39 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 29 Mar 2016 15:44:39 +0200 Subject: [erlang-questions] Freeze gen_server Message-ID: Dear list, I realize how this sounds but: is there a way to temporarily "freeze" a gen_server, so that messages sent to it are not received and its message box stays intact until it is "unfrozen"? What I am trying to achieve: I need to provide a "global lock" mechanism for a specific use case related to cluster conflict resolution. While that conflict resolution is ongoing, I want to ensure that no other changes happen anywhere else. Any ideas? Thanks, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.a.vinogradov@REDACTED Tue Mar 29 15:51:14 2016 From: g.a.vinogradov@REDACTED (Gleb Vinogradov) Date: Tue, 29 Mar 2016 19:51:14 +0600 Subject: [erlang-questions] Fwd: Freeze gen_server In-Reply-To: References: Message-ID: Hi, Roberto. Have you tried sys:suspend/1, sys:resume/1 ? Gleb. 2016-03-29 19:44 GMT+06:00 Roberto Ostinelli : > Dear list, > I realize how this sounds but: is there a way to temporarily "freeze" a > gen_server, so that messages sent to it are not received and its message > box stays intact until it is "unfrozen"? > > What I am trying to achieve: I need to provide a "global lock" mechanism > for a specific use case related to cluster conflict resolution. While that > conflict resolution is ongoing, I want to ensure that no other changes > happen anywhere else. > > Any ideas? > > Thanks, > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Mar 29 16:16:58 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 29 Mar 2016 16:16:58 +0200 Subject: [erlang-questions] Fwd: Freeze gen_server In-Reply-To: References: Message-ID: <20160329141658.GA24345@erix.ericsson.se> On Tue, Mar 29, 2016 at 07:51:14PM +0600, Gleb Vinogradov wrote: > Hi, Roberto. > > Have you tried sys:suspend/1, sys:resume/1 ? That is used by when upgrading an application. First all servers belonging to the application is suspended using sys:suspend/1, then all modules are upgraded, then the code upgrade function is called on all servers and lastly the servers are resumed using sys:resume/1. While suspended the servers will still respond to system messages, otherwise sys:resume/1 would not work. That is done using a selective receive for {system,From,Msg} and {'EXIT',Parent,Reason} so an application can still be stopped while servers are suspended. > > Gleb. > > 2016-03-29 19:44 GMT+06:00 Roberto Ostinelli : > > > Dear list, > > I realize how this sounds but: is there a way to temporarily "freeze" a > > gen_server, so that messages sent to it are not received and its message > > box stays intact until it is "unfrozen"? > > > > What I am trying to achieve: I need to provide a "global lock" mechanism > > for a specific use case related to cluster conflict resolution. While that > > conflict resolution is ongoing, I want to ensure that no other changes > > happen anywhere else. > > > > Any ideas? > > > > Thanks, > > r. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From roberto@REDACTED Tue Mar 29 17:28:51 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 29 Mar 2016 17:28:51 +0200 Subject: [erlang-questions] Fwd: Freeze gen_server In-Reply-To: <20160329141658.GA24345@erix.ericsson.se> References: <20160329141658.GA24345@erix.ericsson.se> Message-ID: Thank you all. These are extremely helpful and can't thank you enough. For anyone interested, I needed to ensure that during Syn's MNESIA auto-conflict resolution I would not allow other processes to modify the MNESIA tables entries before the resolution is over. I've therefore used the suspend / resume schema, this can be seen here: https://github.com/ostinelli/syn/blob/eeeda5ba80fb3c8e5bf55ec48beb3216f349eea5/src/syn_consistency.erl#L192 Again, thank you Gleb, Ulf and Raimo for the great answers. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal@REDACTED Tue Mar 29 14:01:04 2016 From: michal@REDACTED (=?UTF-8?B?TWljaGHFgiBNdXNrYcWCYQ==?=) Date: Tue, 29 Mar 2016 14:01:04 +0200 Subject: [erlang-questions] High memory consumption inside docker Message-ID: Hello everybody, This is a followup from the elixir mailing list: https://groups.google.com/forum/#!msg/elixir-lang-talk/TqIcSVkHBxs/N1nWRWW9BwAJ There were several issues involved in that particular problem, but one of them might be of interest to the people here as well, as it's generally erlang-related rather than tied to elixir or any package in particular. The issue comes down to unusually high memory consumption for erlang, specifically the system part from erlang:memory/0. The output below shows values when running outside docker and inside: Outside docker: erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,15182640},{processes,3799768},{processes_used,3797720},{system,11382872},{atom,194289},{atom_used,169621},{binary,78344},{code,3868184},{ets,229416}] Inside docker's official erlang image (debian based) docker run --rm -it erlang:18-slim erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,111303160},{processes,3799768},{processes_used,3797720},{system,107503392},{atom,194289},{atom_used,169584},{binary,595608},{code,3860673},{ets,239464}] Inside minimal alpine linux erlang image docker run --rm -it msaraiva/erlang erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,111432888},{processes,3748312},{processes_used,3746312},{system,107684576},{atom,194289},{atom_used,169869},{binary,596568},{code,3847723},{ets,185296}] As you can see all the values except for system are very similar, with system being almost 10x as large. The issue is also consistent across different images. While outside docker a simple shell uses around 15MB, inside docker it grows to 80MB. I tried to investigate the issue further using recon, and it looks like the excessive memory is allocated with ll_alloc, although I have to admit I'm out of my depth here. I'll be thankful for any help or indications where I could dig deeper. Micha?. From lukas@REDACTED Tue Mar 29 18:44:19 2016 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 29 Mar 2016 18:44:19 +0200 Subject: [erlang-questions] High memory consumption inside docker In-Reply-To: References: Message-ID: Hello, You want to start the emulator with the "+Mim true +Mis true" flags and then use instrument (http://erlang.org/doc/man/instrument.html) to inspect what is going on. The most interesting for you should be something like: lists:reverse(lists:keysort(2,instrument:memory_status(types))). For example: erl +Mis true +Mim true Erlang/OTP 19 [DEVELOPMENT] [erts-8.0] [source-9946a17] [64-bit] [smp:16:16] [async-threads:10] [hipe] [kernel-poll:false] 1> lists:reverse(lists:keysort(2,instrument:memory_status(types))). [{timer_wheel,[{sizes,8391792,8391792,8391792}, {blocks,16,16,16}]}, {code,[{sizes,3463220,3463220,3469945},{blocks,84,84,84}]}, {proc_tab,[{sizes,3145791,3145791,3145791},{blocks,1,1,1}]}, {heap,[{sizes,1882920,1932208,3456120},{blocks,26,28,28}]}, {pre_alloc_data,[{sizes,1032303,1032303,1032303}, {blocks,17,17,17}]}, {port_tab,[{sizes,786495,786495,786495},{blocks,1,1,1}]}, {old_heap,[{sizes,679872,682880,2941656},{blocks,10,11,11}]}, {scheduler_data,[{sizes,673141,673141,673141}, {blocks,3,3,3}]}, {export_entry,[{sizes,462528,462528,462528}, {blocks,2628,2628,2628}]}, {literal,[{sizes,379648,379648,379648},{blocks,84,84,84}]}, {ethread_long_lived,[{sizes,326016,326016,326016}, {blocks,283,283,283}]}, {atom_entry,[{sizes,304840,304840,304840}, {blocks,7621,7621,7621}]}, {beam_register,[{sizes,264544,264544,264544}, {blocks,32,32,32}]}, {export_tab,[{sizes,162888,162888,162888}, {blocks,15,15,15}]}, {hipe_data,[{sizes,148464,148464,148464},{blocks,18,18,18}]}, {ethread_standard,[{sizes,133487,133487,133487}, {blocks,6,6,6}]}, {db_term,[{sizes,119568,119568,119568}, {blocks,388,388,388}]}, {atom_tab,[{sizes,112232,112232,121472},{blocks,10,10,10}]}, {atom_text,[{sizes,98328,98328,98328},{blocks,3,3,3}]}, {db_tabs,[{sizes,82023,82023,82023},{blocks,3,3,3}]}, {drv_internal,[{sizes,79352,79352,79608},{blocks,4,4,19}]}, {db_segment,[{sizes,51744,51744,53824},{blocks,19,19,...}]}, {fun_entry,[{sizes,51568,51568,...},{blocks,586,...}]}, {drv_binary,[{sizes,47774,...},{blocks,...}]}, {driver_event_state,[{sizes,...},{...}]}, {module_tab,[{...}|...]}, {module_entry,[...]}, {proc,...}, {...}|...] So in the above we can see that the timer wheel structures take the most memory and then code, proc tab etc etc. Lukas On Tue, Mar 29, 2016 at 2:01 PM, Micha? Muska?a wrote: > Hello everybody, > > This is a followup from the elixir mailing list: > > https://groups.google.com/forum/#!msg/elixir-lang-talk/TqIcSVkHBxs/N1nWRWW9BwAJ > There were several issues involved in that particular problem, but one > of them might be of interest to the people here as well, as it's > generally erlang-related rather than tied to elixir or any package in > particular. > > The issue comes down to unusually high memory consumption for erlang, > specifically the system part from erlang:memory/0. The output below > shows values when running outside docker and inside: > > Outside docker: > erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell > > [{total,15182640},{processes,3799768},{processes_used,3797720},{system,11382872},{atom,194289},{atom_used,169621},{binary,78344},{code,3868184},{ets,229416}] > > Inside docker's official erlang image (debian based) > docker run --rm -it erlang:18-slim erl -eval 'io:format("~w~n", > [erlang:memory()]), init:stop().' -noshell > > [{total,111303160},{processes,3799768},{processes_used,3797720},{system,107503392},{atom,194289},{atom_used,169584},{binary,595608},{code,3860673},{ets,239464}] > > Inside minimal alpine linux erlang image > docker run --rm -it msaraiva/erlang erl -eval 'io:format("~w~n", > [erlang:memory()]), init:stop().' -noshell > > [{total,111432888},{processes,3748312},{processes_used,3746312},{system,107684576},{atom,194289},{atom_used,169869},{binary,596568},{code,3847723},{ets,185296}] > > As you can see all the values except for system are very similar, with > system being almost 10x as large. The issue is also consistent across > different images. While outside docker a simple shell uses around > 15MB, inside docker it grows to 80MB. > I tried to investigate the issue further using recon, and it looks > like the excessive memory is allocated with ll_alloc, although I have > to admit I'm out of my depth here. > > I'll be thankful for any help or indications where I could dig deeper. > > Micha?. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Mar 29 18:59:28 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 29 Mar 2016 12:59:28 -0400 (EDT) Subject: [erlang-questions] Printed list to Erlang function Message-ID: <1459270768.794130702@apps.rackspace.com> Hello, So, I have a printed list of stop words: http://www.ranks.nl/stopwords I'd like to turn this list into an Erlang function that I can query--- stopwords() -> ["word1", "word2" ... "wordN"]. is_stopword(Word) -> List = stopwords(), lists_member(Word, List). All my efforts so far have evolved into ugly kludges. Seems to me there must be an elegant method that I'm overlooking. Some kind soul point the way? Many thanks, LRP ********************************************* My books: THE GOSPEL OF ASHES http://thegospelofashes.com Strength is not enough. Do they have the courage and the cunning? Can they survive long enough to save the lives of millions? FREEIN' PANCHO http://freeinpancho.com A community of misfits help a troubled boy find his way AYA TAKEO http://ayatakeo.com Star-crossed love, war and power in an alternative universe Available through Amazon or by request from your favorite bookstore ********************************************** From felixgallo@REDACTED Tue Mar 29 19:14:30 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 29 Mar 2016 10:14:30 -0700 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1459270768.794130702@apps.rackspace.com> References: <1459270768.794130702@apps.rackspace.com> Message-ID: if you don't mind massaging your text file into erlang term format, then you could use file:consult() to pull it into erlang in a fairly nice and seamless way. If your list might get big, your disk storage is slow, or you might check it at a high rate of speed, you might cache the result of file:consult in ETS and check that before loading from disk. F. On Tue, Mar 29, 2016 at 9:59 AM, wrote: > Hello, > > So, I have a printed list of stop words: > > http://www.ranks.nl/stopwords > > I'd like to turn this list into an Erlang function that I can query--- > > stopwords() -> > ["word1", "word2" ... "wordN"]. > > is_stopword(Word) -> > List = stopwords(), > lists_member(Word, List). > > All my efforts so far have evolved into ugly kludges. Seems to me there > must be an elegant method that I'm overlooking. > > Some kind soul point the way? > > Many thanks, > > LRP > > ********************************************* > My books: > > THE GOSPEL OF ASHES > http://thegospelofashes.com > > Strength is not enough. Do they have the courage > and the cunning? Can they survive long enough to > save the lives of millions? > > FREEIN' PANCHO > http://freeinpancho.com > > A community of misfits help a troubled boy find his way > > AYA TAKEO > http://ayatakeo.com > > Star-crossed love, war and power in an alternative > universe > > Available through Amazon or by request from your > favorite bookstore > > > ********************************************** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Mar 29 19:13:00 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 30 Mar 2016 02:13 +0900 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1459270768.794130702@apps.rackspace.com> References: <1459270768.794130702@apps.rackspace.com> Message-ID: <7018608.paWjv6mZvg@changa> On 2016?3?29? ??? 12:59:28 lloyd@REDACTED wrote: > Hello, > > So, I have a printed list of stop words: > > http://www.ranks.nl/stopwords > > I'd like to turn this list into an Erlang function that I can query--- > > stopwords() -> > ["word1", "word2" ... "wordN"]. > > is_stopword(Word) -> > List = stopwords(), > lists_member(Word, List). > > All my efforts so far have evolved into ugly kludges. Seems to me there must be an elegant method that I'm overlooking. > > Some kind soul point the way? Hi Lloyd. I must not quite be understanding the requirement. stopwords/0 returns your list of words. lists:member/2 does indeed return 'true' if the first argument is a member of the second. As written, lists_member/2 above could be lists:member/2 instead and work as expected: ceverett@REDACTED:~/Code/erlang$ cat member_example.erl -module(member_example). -export([go/0]). stopwords() -> ["word1", "word2", "word3"]. is_stopword(Word) -> List = stopwords(), lists:member(Word, List). go() -> First = "word2", Second = "caboose", ok = io:format("~p is in stopwords: ~p~n", [First, is_stopword(First)]), ok = io:format("~p is in stopwords: ~p~n", [Second, is_stopword(Second)]). ceverett@REDACTED:~/Code/erlang$ erl Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Eshell V7.2 (abort with ^G) 1> c(member_example). {ok,member_example} 2> member_example:go(). "word2" is in stopwords: true "caboose" is in stopwords: false ok Either a typo tripped you up or I'm not quite understanding what your goal is. Can you elaborate? -Craig From sergej.jurecko@REDACTED Tue Mar 29 19:19:13 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Tue, 29 Mar 2016 19:19:13 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1459270768.794130702@apps.rackspace.com> References: <1459270768.794130702@apps.rackspace.com> Message-ID: <7F62D2D7-4029-467D-A05F-067362C6723C@gmail.com> > I'd like to turn this list into an Erlang function that I can query--- > Do you mean you want to compile those words into a module so they are always accessible? From zxq9@REDACTED Tue Mar 29 19:21:42 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 30 Mar 2016 02:21:42 +0900 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <7018608.paWjv6mZvg@changa> References: <1459270768.794130702@apps.rackspace.com> <7018608.paWjv6mZvg@changa> Message-ID: <9297606.nXPQ9rW6PY@changa> Wow. Some serious copy-paste snafu my mail client was smart enough to render invisible to me, but dumb enough to pass on to the world. I intended to paste the following: On 2016?3?30? ??? 02:13:00 zxq9 wrote: ... > ceverett@REDACTED:~/Code/erlang$ cat member_example.erl > -module(member_example). > -export([go/0]). > > stopwords() -> > ["word1", "word2", "word3"]. > > is_stopword(Word) -> > List = stopwords(), > lists:member(Word, List). > > go() -> > First = "word2", > Second = "caboose", > ok = io:format("~p is in stopwords: ~p~n", [First, is_stopword(First)]), > ok = io:format("~p is in stopwords: ~p~n", [Second, is_stopword(Second)]). > ceverett@REDACTED:~/Code/erlang$ erl > Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] > > Eshell V7.2 (abort with ^G) > 1> c(member_example). > {ok,member_example} > 2> member_example:go(). > "word2" is in stopwords: true > "caboose" is in stopwords: false > ok Ugh. -Craig From sergej.jurecko@REDACTED Tue Mar 29 19:46:44 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Tue, 29 Mar 2016 19:46:44 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1459273224.06583710@apps.rackspace.com> References: <1459270768.794130702@apps.rackspace.com> <7F62D2D7-4029-467D-A05F-067362C6723C@gmail.com> <1459273224.06583710@apps.rackspace.com> Message-ID: I suggest copying this function (with better formatting): https://github.com/biokoda/bkdcore/blob/master/src/bkdcore.erl#L214 Sergej > On 29 Mar 2016, at 19:40, lloyd@REDACTED wrote: > > Hi Sergej, > > Yes. Sorry I wasn't more explicit. > > Thanks, > > L. > > -----Original Message----- > From: "Sergej Jure?ko" > Sent: Tuesday, March 29, 2016 1:19pm > To: lloyd@REDACTED > Cc: "Erlang Questions Mailing List" > Subject: Re: [erlang-questions] Printed list to Erlang function > >> I'd like to turn this list into an Erlang function that I can query--- >> > > Do you mean you want to compile those words into a module so they are always accessible? > > > > From lloyd@REDACTED Tue Mar 29 19:51:10 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 29 Mar 2016 13:51:10 -0400 (EDT) Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <7F62D2D7-4029-467D-A05F-067362C6723C@gmail.com> <1459273224.06583710@apps.rackspace.com> Message-ID: <1459273870.34630872@apps.rackspace.com> Yeaoh, Sergej, Now I'm really lost. Thanks, LRP -----Original Message----- From: "Sergej Jure?ko" Sent: Tuesday, March 29, 2016 1:46pm To: lloyd@REDACTED Cc: "Erlang Questions Mailing List" Subject: Re: [erlang-questions] Printed list to Erlang function I suggest copying this function (with better formatting): https://github.com/biokoda/bkdcore/blob/master/src/bkdcore.erl#L214 Sergej > On 29 Mar 2016, at 19:40, lloyd@REDACTED wrote: > > Hi Sergej, > > Yes. Sorry I wasn't more explicit. > > Thanks, > > L. > > -----Original Message----- > From: "Sergej Jure?ko" > Sent: Tuesday, March 29, 2016 1:19pm > To: lloyd@REDACTED > Cc: "Erlang Questions Mailing List" > Subject: Re: [erlang-questions] Printed list to Erlang function > >> I'd like to turn this list into an Erlang function that I can query--- >> > > Do you mean you want to compile those words into a module so they are always accessible? > > > > From lloyd@REDACTED Tue Mar 29 19:52:18 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 29 Mar 2016 13:52:18 -0400 (EDT) Subject: [erlang-questions] Printed list to Erlang function Message-ID: <1459273938.51921326@apps.rackspace.com> Hi zxq9, As noted by Sergej, the issue for me was how to transform the list on the printed page into an Erlang list that I could process further; e.g. how to get from the printed list to stopwords/0 with minimum steps along the way. My efforts were taking from copy into file, read back into binary, convert to list, use string:tokens/2 to transform newlines of commas, yada yada... until I had a list in the Erlang shell. But, then, how do I move that list into module? I could do it with another search and replace, but in all, it felt ugly. Thanks, LRP -----Original Message----- From: "zxq9" Sent: Tuesday, March 29, 2016 1:21pm To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Printed list to Erlang function Wow. Some serious copy-paste snafu my mail client was smart enough to render invisible to me, but dumb enough to pass on to the world. I intended to paste the following: On 2016?3?30? ??? 02:13:00 zxq9 wrote: ... > ceverett@REDACTED:~/Code/erlang$ cat member_example.erl > -module(member_example). > -export([go/0]). > > stopwords() -> > ["word1", "word2", "word3"]. > > is_stopword(Word) -> > List = stopwords(), > lists:member(Word, List). > > go() -> > First = "word2", > Second = "caboose", > ok = io:format("~p is in stopwords: ~p~n", [First, is_stopword(First)]), > ok = io:format("~p is in stopwords: ~p~n", [Second, is_stopword(Second)]). > ceverett@REDACTED:~/Code/erlang$ erl > Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] > > Eshell V7.2 (abort with ^G) > 1> c(member_example). > {ok,member_example} > 2> member_example:go(). > "word2" is in stopwords: true > "caboose" is in stopwords: false > ok Ugh. -Craig _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From felixgallo@REDACTED Tue Mar 29 20:35:58 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 29 Mar 2016 11:35:58 -0700 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1459273938.51921326@apps.rackspace.com> References: <1459273938.51921326@apps.rackspace.com> Message-ID: why do you want the list in a module, rather than loaded dynamically on startup? The list is data, which is probably best kept separate from the code. consider, e.g.: https://gist.github.com/anonymous/7dd00fc5fba177f6dd2041edfe9ad09e not a ton of code, and loads a list with a string per line. And you can change the list without recompiling and re-shipping the code. A lot to like about that. F. On Tue, Mar 29, 2016 at 10:52 AM, wrote: > Hi zxq9, > > As noted by Sergej, the issue for me was how to transform the list on the > printed page into an Erlang list that I could process further; e.g. how to > get from the printed list to stopwords/0 with minimum steps along the way. > > My efforts were taking from copy into file, read back into binary, convert > to list, use string:tokens/2 to transform newlines of commas, yada yada... > until I had a list in the Erlang shell. But, then, how do I move that list > into module? I could do it with another search and replace, but in all, it > felt ugly. > > Thanks, > > LRP > > > > -----Original Message----- > From: "zxq9" > Sent: Tuesday, March 29, 2016 1:21pm > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Printed list to Erlang function > > Wow. Some serious copy-paste snafu my mail client was smart enough to > render invisible to me, but dumb enough to pass on to the world. > > I intended to paste the following: > > On 2016?3?30? ??? 02:13:00 zxq9 wrote: > ... > > ceverett@REDACTED:~/Code/erlang$ cat member_example.erl > > -module(member_example). > > -export([go/0]). > > > > stopwords() -> > > ["word1", "word2", "word3"]. > > > > is_stopword(Word) -> > > List = stopwords(), > > lists:member(Word, List). > > > > go() -> > > First = "word2", > > Second = "caboose", > > ok = io:format("~p is in stopwords: ~p~n", [First, > is_stopword(First)]), > > ok = io:format("~p is in stopwords: ~p~n", [Second, > is_stopword(Second)]). > > ceverett@REDACTED:~/Code/erlang$ erl > > Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] > [kernel-poll:false] > > > > Eshell V7.2 (abort with ^G) > > 1> c(member_example). > > {ok,member_example} > > 2> member_example:go(). > > "word2" is in stopwords: true > > "caboose" is in stopwords: false > > ok > > Ugh. > > -Craig > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Mar 30 02:50:08 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 30 Mar 2016 13:50:08 +1300 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> Message-ID: On 30/03/16 6:14 am, Felix Gallo wrote: > if you don't mind massaging your text file into erlang term format, > then you could use file:consult() to pull it into erlang in a fairly > nice and seamless way. If your list might get big, It's a list of stop-words. The stop-word list for any language is going to be at most a few hundred words, and is likely to be changed less often than anything else in the program. On the other hand, is_stopword/1 is going to be called for *EVERY* (or almost every) word in the data that's processed; this is a function you want to be FAST. From ok@REDACTED Wed Mar 30 02:57:39 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 30 Mar 2016 13:57:39 +1300 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <7F62D2D7-4029-467D-A05F-067362C6723C@gmail.com> <1459273224.06583710@apps.rackspace.com> Message-ID: <878c0a9d-efe5-2b7e-a46b-22a87223fe18@cs.otago.ac.nz> On 30/03/16 6:46 am, Sergej Jure?ko wrote: > I suggest copying this function (with better formatting): SInce the list is short (175 entries), and unlikely to change very often, and turning it into efficient Erlang takes two (count 'em, two) VI commands, it seems unnecessary to do anything tricky. From ok@REDACTED Wed Mar 30 02:58:52 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 30 Mar 2016 13:58:52 +1300 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459273938.51921326@apps.rackspace.com> Message-ID: On 30/03/16 7:35 am, Felix Gallo wrote: > why do you want the list in a module, rather than loaded dynamically > on startup? Just a suggestion, but since it's a stop-word list, because the amount of data is small and is_stopword/1 should be fast (which the version using lists:member/2 definitely would not be). From ok@REDACTED Wed Mar 30 03:16:36 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 30 Mar 2016 14:16:36 +1300 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: Message-ID: <613f5b28-186b-9ba1-5a61-77052e2720c7@cs.otago.ac.nz> I notice a lot of lines like ,"0001" => { "0", "", ""} ,"0002" => { "0", "", ""} ,"0003" => { "0", "", ""} ,"0004" => { "0", "", ""} ,"0005" => { "0", "", ""} ,"0006" => { "0", "", ""} ,"0007" => { "0", "", ""} I just wonder whether factoring {"0","",""} out would help at all. I also notice that the lines are all "" => {"", "", ""} and wonder whether having three maps => "" => "" => "" and then lookup(Codepoint) -> N = list_to_integer(Codepoint, 16), A = maps:get(N, ?BY_CODE_TO_DEC, false), B = maps:get(N, ?BY_CODE_TO_STUFF, false), C = maps:get(N, ?BY_CODE_TO_HEX, false), if A == false -> false ; true -> {A, B, C} end. might give the compiler less trouble. I mean, it seems a bit odd to do lookup(Codepoint) -> idna_unicode_data:lookup(hex(Codepoint)). when you could do lookup(Codepoint) -> idna_unicode_data:lookup(Codepoint). In fact, since different functions in idna_unicode.erl are interested in different bits of the triples, I don't understand why these weren't separate maps to begin with. From lloyd@REDACTED Wed Mar 30 04:05:03 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 29 Mar 2016 22:05:03 -0400 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> Message-ID: <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Wow! What a cool idea. Thanks, Richard. Best wishes, LRP Sent from my iPad > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" wrote: > > >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: >> So, I have a printed list of stop words: >> >> http://www.ranks.nl/stopwords >> >> I'd like to turn this list into an Erlang function that I can query--- >> >> stopwords() -> >> ["word1", "word2" ... "wordN"]. >> >> is_stopword(Word) -> >> List = stopwords(), >> lists_member(Word, List). > Even if there is some arcane reason why you want the collection of words > as a list, I strongly suggest generating > > is_stopword("a") -> true; > is_stopword("about") -> true; > ... > is_stopword("yourselves") -> true; > is_stopword(_) -> false. > > Open the list of stopwords in vi. > :1,$s/^.*$/is_stopword("&") -> true;/ > :$a > is_stopword(_) -> false. > > > The Erlang compiler will turn this into a trie, roughly speaking. > This will be *dizzyingly* faster than the code you outlined. > > > > >> >> All my efforts so far have evolved into ugly kludges. Seems to me there must be an elegant method that I'm overlooking. >> >> Some kind soul point the way? >> >> Many thanks, >> >> LRP >> >> ********************************************* >> My books: >> >> THE GOSPEL OF ASHES >> http://thegospelofashes.com >> >> Strength is not enough. Do they have the courage >> and the cunning? Can they survive long enough to >> save the lives of millions? >> >> FREEIN' PANCHO >> http://freeinpancho.com >> >> A community of misfits help a troubled boy find his way >> >> AYA TAKEO >> http://ayatakeo.com >> >> Star-crossed love, war and power in an alternative >> universe >> >> Available through Amazon or by request from your >> favorite bookstore >> >> >> ********************************************** >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From mremond@REDACTED Wed Mar 30 18:50:23 2016 From: mremond@REDACTED (=?UTF-8?B?TWlja2HDq2wgUsOpbW9uZA==?=) Date: Wed, 30 Mar 2016 16:50:23 +0000 Subject: [erlang-questions] ejabberd 16.03 released Message-ID: Hello, We have been very busy on preparing new ejabberd release lately. This new release of the famous Erlang based XMPP server brings experimental MIX group chat support, LDAP, SQL and Riak improvements and many more. I hope you will enjoy it. You can learn more here: https://blog.process-one.net/ejabberd-16-03-experimental-mix-support-ldap-sql-and-riak-improvements/ Enjoy ! -- Micka?l R?mond http://www.process-one.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 30 20:12:20 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 30 Mar 2016 20:12:20 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: Every time I read a claim about how fast it will be I have urge test it. I had an idea that constant map in a module could be faster than function clause co I test it. I was wrong and RAO is right as usual. Function using function clause seems to be three times faster than using map. x clause + map +--------------------------------------------------------------------------+ |xxxxx +++++ +| |xxxx ++++ | |xxxx +++ | |xxxx ++ | |xxx ++ | |xxx ++ | |xx ++ | |xx ++ | |xx ++ | |xx + | |xx + | |xx + | |xx + | |xx + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | + | | + | | + | | + | | + | | + | | + | | + | | + | ||A| | | |_MA_| | +--------------------------------------------------------------------------+ Dataset: x N=50 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 3490.00 1st Qu. 3551.00 Median: 3591.00 3rd Qu. 3679.00 Max: 3945.00 Average: 3630.16 [ 0.137534] ( 3602.82 ? 3664.56) Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? 141.539) Outliers: 0/4 = 4 (?=3630.30, ?=111.587) Outlier variance: 0.151802 (moderate) ------ Dataset: + N=50 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 1.09500e+4 1st Qu. 1.10160e+4 Median: 1.10400e+4 3rd Qu. 1.11270e+4 Max: 1.28270e+4 Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? 1.12491e+4) Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? 582.629) Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) Outlier variance: 9.45082e-2 (slight) Difference at 95.0% confidence 7475.36 ? 80.8533 205.924% ? 2.22726% (Student's t, pooled s = 203.763) ------ It's about 31 million stopwords_clause:is_stopword/1 per second and 10 million stopwords_map:is_stopword/1 per second. You can find code in gist https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 Pichi On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice wrote: > Wow! What a cool idea. > > Thanks, Richard. > > Best wishes, > > LRP > > Sent from my iPad > > > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" > wrote: > > > > > >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: > >> So, I have a printed list of stop words: > >> > >> http://www.ranks.nl/stopwords > >> > >> I'd like to turn this list into an Erlang function that I can query--- > >> > >> stopwords() -> > >> ["word1", "word2" ... "wordN"]. > >> > >> is_stopword(Word) -> > >> List = stopwords(), > >> lists_member(Word, List). > > Even if there is some arcane reason why you want the collection of words > > as a list, I strongly suggest generating > > > > is_stopword("a") -> true; > > is_stopword("about") -> true; > > ... > > is_stopword("yourselves") -> true; > > is_stopword(_) -> false. > > > > Open the list of stopwords in vi. > > :1,$s/^.*$/is_stopword("&") -> true;/ > > :$a > > is_stopword(_) -> false. > > > > > > The Erlang compiler will turn this into a trie, roughly speaking. > > This will be *dizzyingly* faster than the code you outlined. > > > > > > > > > >> > >> All my efforts so far have evolved into ugly kludges. Seems to me there > must be an elegant method that I'm overlooking. > >> > >> Some kind soul point the way? > >> > >> Many thanks, > >> > >> LRP > >> > >> ********************************************* > >> My books: > >> > >> THE GOSPEL OF ASHES > >> http://thegospelofashes.com > >> > >> Strength is not enough. Do they have the courage > >> and the cunning? Can they survive long enough to > >> save the lives of millions? > >> > >> FREEIN' PANCHO > >> http://freeinpancho.com > >> > >> A community of misfits help a troubled boy find his way > >> > >> AYA TAKEO > >> http://ayatakeo.com > >> > >> Star-crossed love, war and power in an alternative > >> universe > >> > >> Available through Amazon or by request from your > >> favorite bookstore > >> > >> > >> ********************************************** > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Wed Mar 30 20:33:00 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 30 Mar 2016 14:33:00 -0400 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: Hi Pichi, Since I haven't learned yet how to design and conduct performance tests, results like these are both interesting and comforting. The long stop words list in http://www.ranks.nl/stopwords has something less than 700 words. So from these results it looks like either method would do the job in most applications, unless you are filtering stop words out of a huge archive of long documents. Many thanks, Pichi. Best wishes, LRP Sent from my iPad > On Mar 30, 2016, at 2:12 PM, Hynek Vychodil wrote: > > Every time I read a claim about how fast it will be I have urge test it. I had an idea that constant map in a module could be faster than function clause co I test it. > > I was wrong and RAO is right as usual. Function using function clause seems to be three times faster than using map. > > x clause > + map > +--------------------------------------------------------------------------+ > |xxxxx +++++ +| > |xxxx ++++ | > |xxxx +++ | > |xxxx ++ | > |xxx ++ | > |xxx ++ | > |xx ++ | > |xx ++ | > |xx ++ | > |xx + | > |xx + | > |xx + | > |xx + | > |xx + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | x + | > | + | > | + | > | + | > | + | > | + | > | + | > | + | > | + | > | + | > ||A| | > | |_MA_| | > +--------------------------------------------------------------------------+ > Dataset: x N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 3490.00 > 1st Qu. 3551.00 > Median: 3591.00 > 3rd Qu. 3679.00 > Max: 3945.00 > Average: 3630.16 [ 0.137534] ( 3602.82 ? 3664.56) > Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? 141.539) > > Outliers: 0/4 = 4 (?=3630.30, ?=111.587) > Outlier variance: 0.151802 (moderate) > > ------ > > Dataset: + N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 1.09500e+4 > 1st Qu. 1.10160e+4 > Median: 1.10400e+4 > 3rd Qu. 1.11270e+4 > Max: 1.28270e+4 > Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? 1.12491e+4) > Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? 582.629) > > Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) > Outlier variance: 9.45082e-2 (slight) > > Difference at 95.0% confidence > 7475.36 ? 80.8533 > 205.924% ? 2.22726% > (Student's t, pooled s = 203.763) > ------ > > It's about 31 million stopwords_clause:is_stopword/1 per second and 10 million stopwords_map:is_stopword/1 per second. > > You can find code in gist https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 > > Pichi > >> On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice wrote: >> Wow! What a cool idea. >> >> Thanks, Richard. >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" wrote: >> > >> > >> >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: >> >> So, I have a printed list of stop words: >> >> >> >> http://www.ranks.nl/stopwords >> >> >> >> I'd like to turn this list into an Erlang function that I can query--- >> >> >> >> stopwords() -> >> >> ["word1", "word2" ... "wordN"]. >> >> >> >> is_stopword(Word) -> >> >> List = stopwords(), >> >> lists_member(Word, List). >> > Even if there is some arcane reason why you want the collection of words >> > as a list, I strongly suggest generating >> > >> > is_stopword("a") -> true; >> > is_stopword("about") -> true; >> > ... >> > is_stopword("yourselves") -> true; >> > is_stopword(_) -> false. >> > >> > Open the list of stopwords in vi. >> > :1,$s/^.*$/is_stopword("&") -> true;/ >> > :$a >> > is_stopword(_) -> false. >> > >> > >> > The Erlang compiler will turn this into a trie, roughly speaking. >> > This will be *dizzyingly* faster than the code you outlined. >> > >> > >> > >> > >> >> >> >> All my efforts so far have evolved into ugly kludges. Seems to me there must be an elegant method that I'm overlooking. >> >> >> >> Some kind soul point the way? >> >> >> >> Many thanks, >> >> >> >> LRP >> >> >> >> ********************************************* >> >> My books: >> >> >> >> THE GOSPEL OF ASHES >> >> http://thegospelofashes.com >> >> >> >> Strength is not enough. Do they have the courage >> >> and the cunning? Can they survive long enough to >> >> save the lives of millions? >> >> >> >> FREEIN' PANCHO >> >> http://freeinpancho.com >> >> >> >> A community of misfits help a troubled boy find his way >> >> >> >> AYA TAKEO >> >> http://ayatakeo.com >> >> >> >> Star-crossed love, war and power in an alternative >> >> universe >> >> >> >> Available through Amazon or by request from your >> >> favorite bookstore >> >> >> >> >> >> ********************************************** >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ynikityshev@REDACTED Wed Mar 30 18:09:39 2016 From: ynikityshev@REDACTED (Yaroslav Nikityshev) Date: Wed, 30 Mar 2016 19:09:39 +0300 Subject: [erlang-questions] [ANN] NoiseIDE v0.847 (One more IDE for erlang) Message-ID: Hey. My coworkers asked me to share this tool we use internally as IDE to make game servers using erlang. Github repo is public for more than a year. https://github.com/IDNoise/NoiseIDE Checkout https://github.com/IDNoise/NoiseIDE/wiki for setup steps and some screenshots. Written using wxPython and Erlang. Wanted to do it 100% erlang, but wxErlang don't have all api calls i need (most of them are for STC component) and i had no luck with updating wx version and recompilation. Code is pretty scary but it works :) Didn't test it on Linux/Mac. We work all the time on windows. - Yaroslav -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 30 22:04:52 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 30 Mar 2016 22:04:52 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: There is result for long list (667 words): x clause + map +--------------------------------------------------------------------------+ | xxx x x +++ + + +| | xxx x +++ | | xxx +++ | | xxx +++ | | xx +++ | | xx ++ | | xx ++ | | xx ++ | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | xx + | | x + | | x + | | x + | | x + | | x + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | ||_A_| | | |___MA____| | +--------------------------------------------------------------------------+ Dataset: x N=50 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 5087.00 1st Qu. 5113.00 Median: 5137.00 3rd Qu. 5188.00 Max: 7081.00 Average: 5205.64 [ 0.729718] ( 5157.08 ? 5372.30) Std. Dev: 287.752 [ -33.4550] ( 81.6038 ? 633.923) Outliers: 0/4 = 4 (?=5206.37, ?=254.297) Outlier variance: 0.365232 (moderate) ------ Dataset: + N=50 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 1.13720e+4 1st Qu. 1.14450e+4 Median: 1.14890e+4 3rd Qu. 1.15510e+4 Max: 1.51180e+4 Average: 1.16464e+4 [ -0.578036] ( 1.15250e+4 ? 1.19671e+4) Std. Dev: 661.815 [ -48.2839] ( 336.017 ? 1217.81) Outliers: 0/3 = 3 (?=1.16458e+4, ?=613.531) Outlier variance: 0.384516 (moderate) Difference at 95.0% confidence 6440.78 ? 202.485 123.727% ? 3.88972% (Student's t, pooled s = 510.294) ------ It is still faster when using function clause and performs nice 22 million calls per second. Pichi On Wed, Mar 30, 2016 at 8:33 PM, Lloyd R. Prentice wrote: > Hi Pichi, > > Since I haven't learned yet how to design and conduct performance tests, > results like these are both interesting and comforting. > > The long stop words list in http://www.ranks.nl/stopwords has something > less than 700 words. So from these results it looks like either method > would do the job in most applications, unless you are filtering stop words > out of a huge archive of long documents. > > Many thanks, Pichi. > > Best wishes, > > LRP > > Sent from my iPad > > On Mar 30, 2016, at 2:12 PM, Hynek Vychodil > wrote: > > Every time I read a claim about how fast it will be I have urge test it. I > had an idea that constant map in a module could be faster than function > clause co I test it. > > I was wrong and RAO is right as usual. Function using function clause > seems to be three times faster than using map. > > x clause > + map > > +--------------------------------------------------------------------------+ > |xxxxx +++++ > +| > |xxxx ++++ > | > |xxxx +++ > | > |xxxx ++ > | > |xxx ++ > | > |xxx ++ > | > |xx ++ > | > |xx ++ > | > |xx ++ > | > |xx + > | > |xx + > | > |xx + > | > |xx + > | > |xx + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > ||A| > | > | |_MA_| > | > > +--------------------------------------------------------------------------+ > Dataset: x N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 3490.00 > 1st Qu. 3551.00 > Median: 3591.00 > 3rd Qu. 3679.00 > Max: 3945.00 > Average: 3630.16 [ 0.137534] ( 3602.82 ? 3664.56) > Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? 141.539) > > Outliers: 0/4 = 4 (?=3630.30, ?=111.587) > Outlier variance: 0.151802 (moderate) > > ------ > > Dataset: + N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 1.09500e+4 > 1st Qu. 1.10160e+4 > Median: 1.10400e+4 > 3rd Qu. 1.11270e+4 > Max: 1.28270e+4 > Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? 1.12491e+4) > Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? 582.629) > > Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) > Outlier variance: 9.45082e-2 (slight) > > Difference at 95.0% confidence > 7475.36 ? 80.8533 > 205.924% ? 2.22726% > (Student's t, pooled s = 203.763) > ------ > > It's about 31 million stopwords_clause:is_stopword/1 per second and 10 > million stopwords_map:is_stopword/1 per second. > > You can find code in gist > https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 > > Pichi > > On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice > wrote: > >> Wow! What a cool idea. >> >> Thanks, Richard. >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" >> wrote: >> > >> > >> >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: >> >> So, I have a printed list of stop words: >> >> >> >> http://www.ranks.nl/stopwords >> >> >> >> I'd like to turn this list into an Erlang function that I can query--- >> >> >> >> stopwords() -> >> >> ["word1", "word2" ... "wordN"]. >> >> >> >> is_stopword(Word) -> >> >> List = stopwords(), >> >> lists_member(Word, List). >> > Even if there is some arcane reason why you want the collection of words >> > as a list, I strongly suggest generating >> > >> > is_stopword("a") -> true; >> > is_stopword("about") -> true; >> > ... >> > is_stopword("yourselves") -> true; >> > is_stopword(_) -> false. >> > >> > Open the list of stopwords in vi. >> > :1,$s/^.*$/is_stopword("&") -> true;/ >> > :$a >> > is_stopword(_) -> false. >> > >> > >> > The Erlang compiler will turn this into a trie, roughly speaking. >> > This will be *dizzyingly* faster than the code you outlined. >> > >> > >> > >> > >> >> >> >> All my efforts so far have evolved into ugly kludges. Seems to me >> there must be an elegant method that I'm overlooking. >> >> >> >> Some kind soul point the way? >> >> >> >> Many thanks, >> >> >> >> LRP >> >> >> >> ********************************************* >> >> My books: >> >> >> >> THE GOSPEL OF ASHES >> >> http://thegospelofashes.com >> >> >> >> Strength is not enough. Do they have the courage >> >> and the cunning? Can they survive long enough to >> >> save the lives of millions? >> >> >> >> FREEIN' PANCHO >> >> http://freeinpancho.com >> >> >> >> A community of misfits help a troubled boy find his way >> >> >> >> AYA TAKEO >> >> http://ayatakeo.com >> >> >> >> Star-crossed love, war and power in an alternative >> >> universe >> >> >> >> Available through Amazon or by request from your >> >> favorite bookstore >> >> >> >> >> >> ********************************************** >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Wed Mar 30 22:07:16 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Wed, 30 Mar 2016 13:07:16 -0700 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: If Dr. O'Keefe were here, he would say that you should get it right first, and then worry about perfor... ...oh. :) F. On Wed, Mar 30, 2016 at 1:04 PM, Hynek Vychodil wrote: > There is result for long list (667 words): > > x clause > + map > > +--------------------------------------------------------------------------+ > | xxx x x +++ + + > +| > | xxx x +++ > | > | xxx +++ > | > | xxx +++ > | > | xx +++ > | > | xx ++ > | > | xx ++ > | > | xx ++ > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | xx + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > ||_A_| > | > | |___MA____| > | > > +--------------------------------------------------------------------------+ > Dataset: x N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 5087.00 > 1st Qu. 5113.00 > Median: 5137.00 > 3rd Qu. 5188.00 > Max: 7081.00 > Average: 5205.64 [ 0.729718] ( 5157.08 ? 5372.30) > Std. Dev: 287.752 [ -33.4550] ( 81.6038 ? 633.923) > > Outliers: 0/4 = 4 (?=5206.37, ?=254.297) > Outlier variance: 0.365232 (moderate) > > ------ > > Dataset: + N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 1.13720e+4 > 1st Qu. 1.14450e+4 > Median: 1.14890e+4 > 3rd Qu. 1.15510e+4 > Max: 1.51180e+4 > Average: 1.16464e+4 [ -0.578036] ( 1.15250e+4 ? 1.19671e+4) > Std. Dev: 661.815 [ -48.2839] ( 336.017 ? 1217.81) > > Outliers: 0/3 = 3 (?=1.16458e+4, ?=613.531) > Outlier variance: 0.384516 (moderate) > > Difference at 95.0% confidence > 6440.78 ? 202.485 > 123.727% ? 3.88972% > (Student's t, pooled s = 510.294) > ------ > > It is still faster when using function clause and performs nice 22 million > calls per second. > > Pichi > > On Wed, Mar 30, 2016 at 8:33 PM, Lloyd R. Prentice > wrote: > >> Hi Pichi, >> >> Since I haven't learned yet how to design and conduct performance tests, >> results like these are both interesting and comforting. >> >> The long stop words list in http://www.ranks.nl/stopwords has something >> less than 700 words. So from these results it looks like either method >> would do the job in most applications, unless you are filtering stop words >> out of a huge archive of long documents. >> >> Many thanks, Pichi. >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> On Mar 30, 2016, at 2:12 PM, Hynek Vychodil >> wrote: >> >> Every time I read a claim about how fast it will be I have urge test it. >> I had an idea that constant map in a module could be faster than function >> clause co I test it. >> >> I was wrong and RAO is right as usual. Function using function clause >> seems to be three times faster than using map. >> >> x clause >> + map >> >> +--------------------------------------------------------------------------+ >> |xxxxx +++++ >> +| >> |xxxx ++++ >> | >> |xxxx +++ >> | >> |xxxx ++ >> | >> |xxx ++ >> | >> |xxx ++ >> | >> |xx ++ >> | >> |xx ++ >> | >> |xx ++ >> | >> |xx + >> | >> |xx + >> | >> |xx + >> | >> |xx + >> | >> |xx + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | x + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> | + >> | >> ||A| >> | >> | |_MA_| >> | >> >> +--------------------------------------------------------------------------+ >> Dataset: x N=50 CI=95.0000 >> Statistic Value [ Bias] (Bootstrapped LB?UB) >> Min: 3490.00 >> 1st Qu. 3551.00 >> Median: 3591.00 >> 3rd Qu. 3679.00 >> Max: 3945.00 >> Average: 3630.16 [ 0.137534] ( 3602.82 ? 3664.56) >> Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? 141.539) >> >> Outliers: 0/4 = 4 (?=3630.30, ?=111.587) >> Outlier variance: 0.151802 (moderate) >> >> ------ >> >> Dataset: + N=50 CI=95.0000 >> Statistic Value [ Bias] (Bootstrapped LB?UB) >> Min: 1.09500e+4 >> 1st Qu. 1.10160e+4 >> Median: 1.10400e+4 >> 3rd Qu. 1.11270e+4 >> Max: 1.28270e+4 >> Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? 1.12491e+4) >> Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? 582.629) >> >> Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) >> Outlier variance: 9.45082e-2 (slight) >> >> Difference at 95.0% confidence >> 7475.36 ? 80.8533 >> 205.924% ? 2.22726% >> (Student's t, pooled s = 203.763) >> ------ >> >> It's about 31 million stopwords_clause:is_stopword/1 per second and 10 >> million stopwords_map:is_stopword/1 per second. >> >> You can find code in gist >> https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 >> >> Pichi >> >> On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice > > wrote: >> >>> Wow! What a cool idea. >>> >>> Thanks, Richard. >>> >>> Best wishes, >>> >>> LRP >>> >>> Sent from my iPad >>> >>> > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" >>> wrote: >>> > >>> > >>> >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: >>> >> So, I have a printed list of stop words: >>> >> >>> >> http://www.ranks.nl/stopwords >>> >> >>> >> I'd like to turn this list into an Erlang function that I can query--- >>> >> >>> >> stopwords() -> >>> >> ["word1", "word2" ... "wordN"]. >>> >> >>> >> is_stopword(Word) -> >>> >> List = stopwords(), >>> >> lists_member(Word, List). >>> > Even if there is some arcane reason why you want the collection of >>> words >>> > as a list, I strongly suggest generating >>> > >>> > is_stopword("a") -> true; >>> > is_stopword("about") -> true; >>> > ... >>> > is_stopword("yourselves") -> true; >>> > is_stopword(_) -> false. >>> > >>> > Open the list of stopwords in vi. >>> > :1,$s/^.*$/is_stopword("&") -> true;/ >>> > :$a >>> > is_stopword(_) -> false. >>> > >>> > >>> > The Erlang compiler will turn this into a trie, roughly speaking. >>> > This will be *dizzyingly* faster than the code you outlined. >>> > >>> > >>> > >>> > >>> >> >>> >> All my efforts so far have evolved into ugly kludges. Seems to me >>> there must be an elegant method that I'm overlooking. >>> >> >>> >> Some kind soul point the way? >>> >> >>> >> Many thanks, >>> >> >>> >> LRP >>> >> >>> >> ********************************************* >>> >> My books: >>> >> >>> >> THE GOSPEL OF ASHES >>> >> http://thegospelofashes.com >>> >> >>> >> Strength is not enough. Do they have the courage >>> >> and the cunning? Can they survive long enough to >>> >> save the lives of millions? >>> >> >>> >> FREEIN' PANCHO >>> >> http://freeinpancho.com >>> >> >>> >> A community of misfits help a troubled boy find his way >>> >> >>> >> AYA TAKEO >>> >> http://ayatakeo.com >>> >> >>> >> Star-crossed love, war and power in an alternative >>> >> universe >>> >> >>> >> Available through Amazon or by request from your >>> >> favorite bookstore >>> >> >>> >> >>> >> ********************************************** >>> >> >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Mar 30 22:11:41 2016 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 30 Mar 2016 22:11:41 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: I had same intuition so why I tried maps first. If you know how it is implemented (You can look at compiled bytecode) and know how BEAM works it shows how well is BEAM written or there is something fishy in maps implementation. (It's 18.3.) Pichi On Wed, Mar 30, 2016 at 8:23 PM, Jason Douglas wrote: > At very large sizes though, wouldn?t the map (with O(1) access) become > faster than the trie built by the compiler (which will be proportional to > lookup term length, when sufficiently dense)? I used to deal a lot with > pre-compiled tries and hash tables in C++, and while I loved tries for > space efficiency, a hash table always had the upper hand in performance. > > I can see though why for a small set like this, a carefully tuned > implementation like Erlang?s would beat out the map. > > Jason > > On Mar 30, 2016, at 11:12 AM, Hynek Vychodil > wrote: > > Every time I read a claim about how fast it will be I have urge test it. I > had an idea that constant map in a module could be faster than function > clause co I test it. > > I was wrong and RAO is right as usual. Function using function clause > seems to be three times faster than using map. > > x clause > + map > > +--------------------------------------------------------------------------+ > |xxxxx +++++ > +| > |xxxx ++++ > | > |xxxx +++ > | > |xxxx ++ > | > |xxx ++ > | > |xxx ++ > | > |xx ++ > | > |xx ++ > | > |xx ++ > | > |xx + > | > |xx + > | > |xx + > | > |xx + > | > |xx + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | x + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > | + > | > ||A| > | > | |_MA_| > | > > +--------------------------------------------------------------------------+ > Dataset: x N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 3490.00 > 1st Qu. 3551.00 > Median: 3591.00 > 3rd Qu. 3679.00 > Max: 3945.00 > Average: 3630.16 [ 0.137534] ( 3602.82 ? 3664.56) > Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? 141.539) > > Outliers: 0/4 = 4 (?=3630.30, ?=111.587) > Outlier variance: 0.151802 (moderate) > > ------ > > Dataset: + N=50 CI=95.0000 > Statistic Value [ Bias] (Bootstrapped LB?UB) > Min: 1.09500e+4 > 1st Qu. 1.10160e+4 > Median: 1.10400e+4 > 3rd Qu. 1.11270e+4 > Max: 1.28270e+4 > Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? 1.12491e+4) > Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? 582.629) > > Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) > Outlier variance: 9.45082e-2 (slight) > > Difference at 95.0% confidence > 7475.36 ? 80.8533 > 205.924% ? 2.22726% > (Student's t, pooled s = 203.763) > ------ > > It's about 31 million stopwords_clause:is_stopword/1 per second and 10 > million stopwords_map:is_stopword/1 per second. > > You can find code in gist > https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 > > Pichi > > On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice > wrote: > >> Wow! What a cool idea. >> >> Thanks, Richard. >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" >> wrote: >> > >> > >> >> On 30/03/16 5:59 am, lloyd@REDACTED wrote: >> >> So, I have a printed list of stop words: >> >> >> >> http://www.ranks.nl/stopwords >> >> >> >> I'd like to turn this list into an Erlang function that I can query--- >> >> >> >> stopwords() -> >> >> ["word1", "word2" ... "wordN"]. >> >> >> >> is_stopword(Word) -> >> >> List = stopwords(), >> >> lists_member(Word, List). >> > Even if there is some arcane reason why you want the collection of words >> > as a list, I strongly suggest generating >> > >> > is_stopword("a") -> true; >> > is_stopword("about") -> true; >> > ... >> > is_stopword("yourselves") -> true; >> > is_stopword(_) -> false. >> > >> > Open the list of stopwords in vi. >> > :1,$s/^.*$/is_stopword("&") -> true;/ >> > :$a >> > is_stopword(_) -> false. >> > >> > >> > The Erlang compiler will turn this into a trie, roughly speaking. >> > This will be *dizzyingly* faster than the code you outlined. >> > >> > >> > >> > >> >> >> >> All my efforts so far have evolved into ugly kludges. Seems to me >> there must be an elegant method that I'm overlooking. >> >> >> >> Some kind soul point the way? >> >> >> >> Many thanks, >> >> >> >> LRP >> >> >> >> ********************************************* >> >> My books: >> >> >> >> THE GOSPEL OF ASHES >> >> http://thegospelofashes.com >> >> >> >> Strength is not enough. Do they have the courage >> >> and the cunning? Can they survive long enough to >> >> save the lives of millions? >> >> >> >> FREEIN' PANCHO >> >> http://freeinpancho.com >> >> >> >> A community of misfits help a troubled boy find his way >> >> >> >> AYA TAKEO >> >> http://ayatakeo.com >> >> >> >> Star-crossed love, war and power in an alternative >> >> universe >> >> >> >> Available through Amazon or by request from your >> >> favorite bookstore >> >> >> >> >> >> ********************************************** >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.azedo@REDACTED Wed Mar 30 22:01:49 2016 From: luis.azedo@REDACTED (Luis Azedo) Date: Wed, 30 Mar 2016 21:01:49 +0100 Subject: [erlang-questions] function clauses with maps (18.3) Message-ID: Hi, this doesn't compile in 18.3. is it supposed to work with the maps spec ? maybe in a later release ? -spec my_match(atom(), atom(), map()) -> any(). my_match(Classification, CarType, #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := Map}}}}) -> .... variable 'Classification' is unbound variable 'CarType' is unbound this works but we're not taking advantage of function clauses. my_match(Classification, CarType, M ) -> #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := Map}}}} = M, Map. .... TIA -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Wed Mar 30 22:54:58 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Wed, 30 Mar 2016 22:54:58 +0200 Subject: [erlang-questions] function clauses with maps (18.3) In-Reply-To: References: Message-ID: This is a current limitation with maps and bitsyntax. Variables has to be bound before being used as an expression in a pattern. 2016-03-30 22:01 GMT+02:00 Luis Azedo : > Hi, > > this doesn't compile in 18.3. is it supposed to work with the maps spec ? > maybe in a later release ? > > -spec my_match(atom(), atom(), map()) -> any(). > my_match(Classification, CarType, #{my_key := #{ Classification := #{ > <<"cars">> := #{ CarType := Map}}}}) -> > .... > variable 'Classification' is unbound > variable 'CarType' is unbound > > > this works but we're not taking advantage of function clauses. > > my_match(Classification, CarType, M ) -> > #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := > Map}}}} = M, > Map. > .... > > TIA > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Wed Mar 30 23:04:53 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Wed, 30 Mar 2016 23:04:53 +0200 Subject: [erlang-questions] function clauses with maps (18.3) In-Reply-To: References: Message-ID: OTP 20.0 at the earliest. However, no definitive decision has been made about when or even if. 2016-03-30 22:59 GMT+02:00 Luis Azedo : > Thanks for the reply. > > so in the future in should work, right ? do you know which version/release > we should wait before testing this ? > > Thanks > > > On Wed, Mar 30, 2016 at 9:54 PM, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> This is a current limitation with maps and bitsyntax. Variables has to be >> bound before being used as an expression in a pattern. >> >> 2016-03-30 22:01 GMT+02:00 Luis Azedo : >> >>> Hi, >>> >>> this doesn't compile in 18.3. is it supposed to work with the maps spec >>> ? maybe in a later release ? >>> >>> -spec my_match(atom(), atom(), map()) -> any(). >>> my_match(Classification, CarType, #{my_key := #{ Classification := #{ >>> <<"cars">> := #{ CarType := Map}}}}) -> >>> .... >>> variable 'Classification' is unbound >>> variable 'CarType' is unbound >>> >>> >>> this works but we're not taking advantage of function clauses. >>> >>> my_match(Classification, CarType, M ) -> >>> #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := >>> Map}}}} = M, >>> Map. >>> .... >>> >>> TIA >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Wed Mar 30 23:59:25 2016 From: rvirding@REDACTED (Robert Virding) Date: Wed, 30 Mar 2016 23:59:25 +0200 Subject: [erlang-questions] function clauses with maps (18.3) In-Reply-To: References: Message-ID: Personally I think it would be very dubious to change it as you would be changing the meaning of matching for those variables. Robert On 30 March 2016 at 23:04, Bj?rn-Egil Dahlberg wrote: > OTP 20.0 at the earliest. However, no definitive decision has been made > about when or even if. > > 2016-03-30 22:59 GMT+02:00 Luis Azedo : > >> Thanks for the reply. >> >> so in the future in should work, right ? do you know which >> version/release we should wait before testing this ? >> >> Thanks >> >> >> On Wed, Mar 30, 2016 at 9:54 PM, Bj?rn-Egil Dahlberg < >> wallentin.dahlberg@REDACTED> wrote: >> >>> This is a current limitation with maps and bitsyntax. Variables has to >>> be bound before being used as an expression in a pattern. >>> >>> 2016-03-30 22:01 GMT+02:00 Luis Azedo : >>> >>>> Hi, >>>> >>>> this doesn't compile in 18.3. is it supposed to work with the maps spec >>>> ? maybe in a later release ? >>>> >>>> -spec my_match(atom(), atom(), map()) -> any(). >>>> my_match(Classification, CarType, #{my_key := #{ Classification := #{ >>>> <<"cars">> := #{ CarType := Map}}}}) -> >>>> .... >>>> variable 'Classification' is unbound >>>> variable 'CarType' is unbound >>>> >>>> >>>> this works but we're not taking advantage of function clauses. >>>> >>>> my_match(Classification, CarType, M ) -> >>>> #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := >>>> Map}}}} = M, >>>> Map. >>>> .... >>>> >>>> TIA >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Thu Mar 31 00:08:12 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Thu, 31 Mar 2016 00:08:12 +0200 Subject: [erlang-questions] function clauses with maps (18.3) In-Reply-To: References: Message-ID: You are being vague in your doubt. Do you mean it would impose a matching order? Which it would. 2016-03-30 23:59 GMT+02:00 Robert Virding : > Personally I think it would be very dubious to change it as you would be > changing the meaning of matching for those variables. > > Robert > > > On 30 March 2016 at 23:04, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> OTP 20.0 at the earliest. However, no definitive decision has been made >> about when or even if. >> >> 2016-03-30 22:59 GMT+02:00 Luis Azedo : >> >>> Thanks for the reply. >>> >>> so in the future in should work, right ? do you know which >>> version/release we should wait before testing this ? >>> >>> Thanks >>> >>> >>> On Wed, Mar 30, 2016 at 9:54 PM, Bj?rn-Egil Dahlberg < >>> wallentin.dahlberg@REDACTED> wrote: >>> >>>> This is a current limitation with maps and bitsyntax. Variables has to >>>> be bound before being used as an expression in a pattern. >>>> >>>> 2016-03-30 22:01 GMT+02:00 Luis Azedo : >>>> >>>>> Hi, >>>>> >>>>> this doesn't compile in 18.3. is it supposed to work with the maps >>>>> spec ? maybe in a later release ? >>>>> >>>>> -spec my_match(atom(), atom(), map()) -> any(). >>>>> my_match(Classification, CarType, #{my_key := #{ Classification := #{ >>>>> <<"cars">> := #{ CarType := Map}}}}) -> >>>>> .... >>>>> variable 'Classification' is unbound >>>>> variable 'CarType' is unbound >>>>> >>>>> >>>>> this works but we're not taking advantage of function clauses. >>>>> >>>>> my_match(Classification, CarType, M ) -> >>>>> #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := >>>>> Map}}}} = M, >>>>> Map. >>>>> .... >>>>> >>>>> TIA >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Mar 31 00:43:39 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 31 Mar 2016 11:43:39 +1300 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: On 31/03/16 9:11 am, Hynek Vychodil wrote: > On Wed, Mar 30, 2016 at 8:23 PM, Jason Douglas > wrote: > > At very large sizes though, wouldn?t the map (with O(1) access) > become faster than the trie built by the compiler (which will be > proportional to lookup term length, when sufficiently dense)? > Suppose you are looking up a key of length L in a map, and that everything goes right. The hash function will (if it is a *good* one) examine every part of the key, so hashing will take O(L). Assuming everything goes right, it will take O(1) time to find the right bucket and there will be only one item in it, but it will take another O(L) to confirm that you have indeed found the right key. So maps are *NOT* O(1), they are O(L). What about a trie? A trie over a binary alphabet takes O(1) time per element of the key, for a total of O(L). The constant factor could go either way depending on the implementation. But of course this *isn't* a binary alphabet. Let B be the average branching factor of the trie; the cost will be O(L.B) with a linear search at each node, O(L.log(B)) with a binary search at each node, or O(L) with some sort of local hashing. And this depends on exactly what the compiler does, and may change from release to release. For the record, I never said that the trie would beat a map. I said it would beat lists:member/2. Concerning good hash functions, see "Hashing in Smalltalk: Theory and Practice" by Andr?s Valloud. It's not just the best book I know that explains how to actually construct and evaluate hash functions for a variety of data structures, it's the *only* one. (I have a paper about how to compute hash functions of sets and dictionaries; we can do a lot better than the simplistic combine-using-plus-or-xor that's usually done. Haven't found a journal to take it yet.) From ok@REDACTED Thu Mar 31 03:59:10 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 31 Mar 2016 14:59:10 +1300 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: <171AEF8E-EB9A-4BEC-B3E3-DC376569CC8C@icloud.com> References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> <171AEF8E-EB9A-4BEC-B3E3-DC376569CC8C@icloud.com> Message-ID: On 31/03/16 1:45 pm, Jason Douglas wrote: > Part of the reason I?d expect this, is that as the data set grows in > size, the time to access memory should become the dominant factor, as > opposed to CPU work (like hashing the key). A hash table with minimal > collisions should always be about 1 or 2 memory accesses. No, no, NO! A hash table has to access ALL THE MEMORY OF THE KEY. Suppose a list is a sequence of pairs, a pair just being a head cell and a tail cell with no header, so that a list of N elements takes 2N words plus any extra space for elements. Then looking up "1...N" in a map is going to need *at least* 1 memory reference to determine that the map is a map 1 memory reference to determine the size of the map 2N memory references to compute the key's hash 1 memory reference to pick up a matched key from a bucket 4N memory references to check that the key sought and the key found are the same 1 memory reference to pick up the associated value for a total of 4 + 6N memory references. If the keys are small integers, replace 2N by 1, for a total of 7. (This is why I recommended working with Unicode codepoints as integers yesterday, instead of as lists of hex digits.) If you use binaries <<"1...N">> instead of lists, that's going to take something like 4 + ceiling(N/4) words on a 32-bit machine so we get 16 + N/2 memory references. If the average word is 6 letters, then we're looking at FORTY memory references for a list or NINETEEN for a binary and this is the *best* case. NOT "1 or 2". Hash tables are *not* magic. Key hashing and comparison take real work, not clairvoyance. > A trie on the other hand will typically be /at least/ 1 or 2, often many more. Recall that a hash table has to do 3N character visits: 1N to compute the hash, and 2N to verify equality. A trie just does 1N character visits; no equality test at the end is needed. It's *really* not clear which will win; in any given case I would not presume to guess but would benchmark. Also, tries *might* win on space by sharing prefixes. "food", "fool", "foolish", "foot", "footfall" will all pass through the *same* nodes as the trie steps through "foo". This can be taken further, to Directed Acyclic Word Graphs (DAWGs; I've always loved that acronym) which can share suffixes as well as prefixes. There are also data structures like suffix arrays. You'll find a range of suitable data structures described in information retrieval textbooks, especially Baeza-Yates. From luis.azedo@REDACTED Wed Mar 30 22:59:44 2016 From: luis.azedo@REDACTED (Luis Azedo) Date: Wed, 30 Mar 2016 21:59:44 +0100 Subject: [erlang-questions] function clauses with maps (18.3) In-Reply-To: References: Message-ID: Thanks for the reply. so in the future in should work, right ? do you know which version/release we should wait before testing this ? Thanks On Wed, Mar 30, 2016 at 9:54 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > This is a current limitation with maps and bitsyntax. Variables has to be > bound before being used as an expression in a pattern. > > 2016-03-30 22:01 GMT+02:00 Luis Azedo : > >> Hi, >> >> this doesn't compile in 18.3. is it supposed to work with the maps spec ? >> maybe in a later release ? >> >> -spec my_match(atom(), atom(), map()) -> any(). >> my_match(Classification, CarType, #{my_key := #{ Classification := #{ >> <<"cars">> := #{ CarType := Map}}}}) -> >> .... >> variable 'Classification' is unbound >> variable 'CarType' is unbound >> >> >> this works but we're not taking advantage of function clauses. >> >> my_match(Classification, CarType, M ) -> >> #{my_key := #{ Classification := #{ <<"cars">> := #{ CarType := >> Map}}}} = M, >> Map. >> .... >> >> TIA >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Thu Mar 31 11:05:28 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Thu, 31 Mar 2016 11:05:28 +0200 Subject: [erlang-questions] [ANN] Contribution needed - bugs.erlang.org Message-ID: Hi Since the launch of bugs.erlang.org, many issues have been reported by the community. You might have noticed that some issues are labelled as *"Contribution Needed"*: http://bugs.erlang.org/issues/?jql=status%3D%22Contribution%20Needed%22 This means that the OTP team has looked at the issue and would like help from the community. You are encouraged to contribute to Erlang by joining the discussion and/or helping solving these issues. Contribute to Erlang/OTP: 1. Look for issues labelled as "Contribution Needed". 2. Discuss in the issue comment. 3. Agree on a solution/implementation. 4. Submit the contribution as a pull request on Github. 5. An OTP team member reviews the implementation and merges the change. You are also encouraged to contribute by joining the discussion on issues which are "Open". Best regards *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Mar 31 12:54:42 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 31 Mar 2016 10:54:42 +0000 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: <613f5b28-186b-9ba1-5a61-77052e2720c7@cs.otago.ac.nz> References: <613f5b28-186b-9ba1-5a61-77052e2720c7@cs.otago.ac.nz> Message-ID: On Wed, Mar 30, 2016 at 3:16 AM Richard A. O'Keefe wrote: > I notice a lot of lines like > > ,"0001" => { "0", "", ""} > ,"0002" => { "0", "", ""} > ,"0003" => { "0", "", ""} > ,"0004" => { "0", "", ""} > ,"0005" => { "0", "", ""} > ,"0006" => { "0", "", ""} > ,"0007" => { "0", "", ""} > > I just wonder whether factoring {"0","",""} out > would help at all. > > I also notice that the lines are all > "" => {"", "", ""} > and wonder whether having three maps > => "" > => "" > => "" > So I changed the code to use integers everywhere. It makes the processing a lot faster: https://github.com/benoitc/erlang-idna/commit/a40fedb1ac2933a4d668b4813bf2cfab408ef6a4 For now the code is creating one module with pattern matching and still has the tuple. Since the size of the file is smaller, it compile a little faster and use less memory. > and then > lookup(Codepoint) -> > N = list_to_integer(Codepoint, 16), > A = maps:get(N, ?BY_CODE_TO_DEC, false), > B = maps:get(N, ?BY_CODE_TO_STUFF, false), > C = maps:get(N, ?BY_CODE_TO_HEX, false), > if A == false -> false > ; true -> {A, B, C} > end. > might give the compiler less trouble. > > I mean, it seems a bit odd to do > > lookup(Codepoint) -> > idna_unicode_data:lookup(hex(Codepoint)). > > when you could do > > lookup(Codepoint) -> > idna_unicode_data:lookup(Codepoint). > > In fact, since different functions in idna_unicode.erl > are interested in different bits of the triples, I > don't understand why these weren't separate maps to > begin with. > > Do you mean having different functions in the same module using a map each? What is the cost of using a map compared to use function pattern-matching if the code is called by different processes? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Mar 31 13:29:26 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 31 Mar 2016 13:29:26 +0200 Subject: [erlang-questions] [ANN] Contribution needed - bugs.erlang.org In-Reply-To: References: Message-ID: <56FD0A16.3020506@ninenines.eu> Not much to look at here. Would love to have access to the OTP tickets rather than just the community reports. In particular for the SSL application. I'm probably not the only one who would like to have the full list of issues for that one. :-) On 03/31/2016 11:05 AM, Bruce Yinhe wrote: > Hi > > Since the launch of bugs.erlang.org , many > issues have been reported by the community. You might have noticed that > some issues are labelled as *"Contribution Needed"*: > > http://bugs.erlang.org/issues/?jql=status%3D%22Contribution%20Needed%22 > > > This means that the OTP team has looked at the issue and would like help > from the community. You are encouraged to contribute to Erlang by > joining the discussion and/or helping solving these issues. > > Contribute to Erlang/OTP: > > 1. Look for issues labelled as "Contribution Needed". > 2. Discuss in the issue comment. > 3. Agree on a solution/implementation. > 4. Submit the contribution as a pull request on Github. > 5. An OTP team member reviews the implementation and merges the change. > > > You are also encouraged to contribute by joining the discussion on > issues which are "Open". > > Best regards > > *Bruce Yinhe* > Community Manager > Industrial Erlang User Group > +46 72 311 43 89 > community-manager@REDACTED > > -- > Visit our Erlang community site ErlangCentral.org > | @ErlangCentral > |Industrial Erlang User Group > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From michael.santos@REDACTED Thu Mar 31 16:27:13 2016 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 31 Mar 2016 10:27:13 -0400 Subject: [erlang-questions] [erlang-bugs] [ANN] Contribution needed - bugs.erlang.org In-Reply-To: References: Message-ID: <20160331142713.GA10502@brk> On Thu, Mar 31, 2016 at 11:05:28AM +0200, Bruce Yinhe wrote: > Hi > > Since the launch of bugs.erlang.org, many issues have been reported by the > community. You might have noticed that some issues are labelled as > *"Contribution > Needed"*: > > http://bugs.erlang.org/issues/?jql=status%3D%22Contribution%20Needed%22 > > > This means that the OTP team has looked at the issue and would like help > from the community. You are encouraged to contribute to Erlang by joining > the discussion and/or helping solving these issues. > > Contribute to Erlang/OTP: > > 1. Look for issues labelled as "Contribution Needed". > 2. Discuss in the issue comment. > 3. Agree on a solution/implementation. > 4. Submit the contribution as a pull request on Github. > 5. An OTP team member reviews the implementation and merges the change. > > > You are also encouraged to contribute by joining the discussion on issues > which are "Open". Why does logging in with your github account require full access (write access) to your github account personal details? Is there a way of monitoring for new issues? For example, could new issues be forwarded to the old erlang-bugs mail list? > > Best regards > > *Bruce Yinhe* > Community Manager > Industrial Erlang User Group > +46 72 311 43 89 > community-manager@REDACTED > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs From roger@REDACTED Thu Mar 31 17:33:59 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 31 Mar 2016 16:33:59 +0100 Subject: [erlang-questions] SSL connections (in Common Test) intermittently fail with "unknown ca" Message-ID: I've got a custom ranch protocol -- it's based on ranch_ssl, but it adds a custom verify_fun, with configuration options. I'm attempting to test it in Common Test, and I'm seeing intermittent "unknown ca" failures. I suspect, though I'm not sure, that it might be due to the fact that each test starts a ranch listener with different SSL options, in particular the 'cacertfile' option varies. Is there a race condition in the 'ssl' application which might get confused by this? I've attempted to clean up by calling ssl:clear_pem_cache from end_per_testcase, but it doesn't appear to make any difference. Help...? I can tidy up the test suite for public consumption if anyone thinks that would be useful. -- Roger. From michael.nisi@REDACTED Thu Mar 31 16:38:08 2016 From: michael.nisi@REDACTED (Michael Nisi) Date: Thu, 31 Mar 2016 16:38:08 +0200 Subject: [erlang-questions] [ANN] Contribution needed - bugs.erlang.org In-Reply-To: References: Message-ID: Two, maybe na?ve, questions. How does one get access to this? Why isn't everything on GitHub? Michael > On 31.03.2016, at 11:05, Bruce Yinhe wrote: > > Hi > > Since the launch of bugs.erlang.org, many issues have been reported by the community. You might have noticed that some issues are labelled as "Contribution Needed": > > http://bugs.erlang.org/issues/?jql=status%3D%22Contribution%20Needed%22 > > This means that the OTP team has looked at the issue and would like help from the community. You are encouraged to contribute to Erlang by joining the discussion and/or helping solving these issues. > > Contribute to Erlang/OTP: > > 1. Look for issues labelled as "Contribution Needed". > 2. Discuss in the issue comment. > 3. Agree on a solution/implementation. > 4. Submit the contribution as a pull request on Github. > 5. An OTP team member reviews the implementation and merges the change. > > You are also encouraged to contribute by joining the discussion on issues which are "Open". > > Best regards > > Bruce Yinhe > Community Manager > Industrial Erlang User Group > +46 72 311 43 89 > community-manager@REDACTED > > -- > Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From betabrain.erlangquestions@REDACTED Thu Mar 31 19:59:56 2016 From: betabrain.erlangquestions@REDACTED (T A) Date: Thu, 31 Mar 2016 17:59:56 +0000 Subject: [erlang-questions] Newbie questions about development since "A History of Erlang" Message-ID: <4a2f2859f08e15e1d15de0e34b62ec3d@node1.gmailbox.org> An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Thu Mar 31 21:31:44 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 31 Mar 2016 21:31:44 +0200 Subject: [erlang-questions] Printed list to Erlang function In-Reply-To: References: <1459270768.794130702@apps.rackspace.com> <082646ea-14d5-a1ee-b24d-40ea0a98d9dc@cs.otago.ac.nz> <1F3463C7-C355-4F6D-B007-25784EDAEDB9@writersglen.com> Message-ID: <56FD7B20.7020807@ericsson.com> Small maps uses an array with linear search for key. Large maps (> 32 keys) uses a HAMT tree with a maximum branching factor of 16. https://en.wikipedia.org/wiki/Hash_array_mapped_trie /Sverker, Erlang/OTP On 03/30/2016 10:11 PM, Hynek Vychodil wrote: > I had same intuition so why I tried maps first. If you know how it is > implemented (You can look at compiled bytecode) and know how BEAM > works it shows how well is BEAM written or there is something fishy in > maps implementation. (It's 18.3.) > > Pichi > > On Wed, Mar 30, 2016 at 8:23 PM, Jason Douglas > wrote: > > At very large sizes though, wouldn?t the map (with O(1) access) > become faster than the trie built by the compiler (which will be > proportional to lookup term length, when sufficiently dense)? I > used to deal a lot with pre-compiled tries and hash tables in C++, > and while I loved tries for space efficiency, a hash table always > had the upper hand in performance. > > I can see though why for a small set like this, a carefully tuned > implementation like Erlang?s would beat out the map. > > Jason > >> On Mar 30, 2016, at 11:12 AM, Hynek Vychodil >> > wrote: >> >> Every time I read a claim about how fast it will be I have urge >> test it. I had an idea that constant map in a module could be >> faster than function clause co I test it. >> >> I was wrong and RAO is right as usual. Function using function >> clause seems to be three times faster than using map. >> >> x clause >> + map >> +--------------------------------------------------------------------------+ >> |xxxxx +++++ +| >> |xxxx ++++ | >> |xxxx +++ | >> |xxxx ++ | >> |xxx ++ | >> |xxx ++ | >> |xx ++ | >> |xx ++ | >> |xx ++ | >> |xx + | >> |xx + | >> |xx + | >> |xx + | >> |xx + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | x + | >> | + | >> | + | >> | + | >> | + | >> | + | >> | + | >> | + | >> | + | >> | + | >> ||A| | >> | |_MA_| | >> +--------------------------------------------------------------------------+ >> Dataset: x N=50 CI=95.0000 >> Statistic Value [ Bias] (Bootstrapped LB?UB) >> Min: 3490.00 >> 1st Qu. 3551.00 >> Median: 3591.00 >> 3rd Qu. 3679.00 >> Max: 3945.00 >> Average: 3630.16 [ 0.137534] ( 3602.82 ? >> 3664.56) >> Std. Dev: 113.400 [ -1.81311] ( 90.8425 ? >> 141.539) >> >> Outliers: 0/4 = 4 (?=3630.30, ?=111.587) >> Outlier variance: 0.151802 (moderate) >> >> ------ >> >> Dataset: + N=50 CI=95.0000 >> Statistic Value [ Bias] (Bootstrapped LB?UB) >> Min: 1.09500e+4 >> 1st Qu. 1.10160e+4 >> Median: 1.10400e+4 >> 3rd Qu. 1.11270e+4 >> Max: 1.28270e+4 >> Average: 1.11055e+4 [ 0.297998] ( 1.10611e+4 ? >> 1.12491e+4) >> Std. Dev: 264.914 [ -31.0673] ( 84.7956 ? >> 582.629) >> >> Outliers: 0/2 = 2 (?=1.11058e+4, ?=233.847) >> Outlier variance: 9.45082e-2 (slight) >> >> Difference at 95.0% confidence >> 7475.36 ? 80.8533 >> 205.924% ? 2.22726% >> (Student's t, pooled s = 203.763) >> ------ >> >> It's about 31 million stopwords_clause:is_stopword/1 per second >> and 10 million stopwords_map:is_stopword/1 per second. >> >> You can find code in gist >> https://gist.github.com/pichi/2d10c93242d5057913d026a607f07dd4 >> >> Pichi >> >> On Wed, Mar 30, 2016 at 4:05 AM, Lloyd R. Prentice >> > wrote: >> >> Wow! What a cool idea. >> >> Thanks, Richard. >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> > On Mar 29, 2016, at 8:47 PM, "Richard A. O'Keefe" >> > wrote: >> > >> > >> >> On 30/03/16 5:59 am, lloyd@REDACTED >> wrote: >> >> So, I have a printed list of stop words: >> >> >> >> http://www.ranks.nl/stopwords >> >> >> >> I'd like to turn this list into an Erlang function that I >> can query--- >> >> >> >> stopwords() -> >> >> ["word1", "word2" ... "wordN"]. >> >> >> >> is_stopword(Word) -> >> >> List = stopwords(), >> >> lists_member(Word, List). >> > Even if there is some arcane reason why you want the >> collection of words >> > as a list, I strongly suggest generating >> > >> > is_stopword("a") -> true; >> > is_stopword("about") -> true; >> > ... >> > is_stopword("yourselves") -> true; >> > is_stopword(_) -> false. >> > >> > Open the list of stopwords in vi. >> > :1,$s/^.*$/is_stopword("&") -> true;/ >> > :$a >> > is_stopword(_) -> false. >> > >> > >> > The Erlang compiler will turn this into a trie, roughly >> speaking. >> > This will be *dizzyingly* faster than the code you outlined. >> > >> > >> > >> > >> >> >> >> All my efforts so far have evolved into ugly kludges. >> Seems to me there must be an elegant method that I'm overlooking. >> >> >> >> Some kind soul point the way? >> >> >> >> Many thanks, >> >> >> >> LRP >> >> >> >> ********************************************* >> >> My books: >> >> >> >> THE GOSPEL OF ASHES >> >> http://thegospelofashes.com >> >> >> >> Strength is not enough. Do they have the courage >> >> and the cunning? Can they survive long enough to >> >> save the lives of millions? >> >> >> >> FREEIN' PANCHO >> >> http://freeinpancho.com >> >> >> >> A community of misfits help a troubled boy find his way >> >> >> >> AYA TAKEO >> >> http://ayatakeo.com >> >> >> >> Star-crossed love, war and power in an alternative >> >> universe >> >> >> >> Available through Amazon or by request from your >> >> favorite bookstore >> >> >> >> >> >> ********************************************** >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Mar 31 23:10:09 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 31 Mar 2016 23:10:09 +0200 Subject: [erlang-questions] jinterface maven artifact Message-ID: Hello everybody, Sorry to send this to everyone, but I don't have another list for people using jinterface. I have published a Maven artifact for the latest released jinterface (1.6.1, from OTP 18.3). Since it's my first time doing that, there is a good chance that something went wrong, so if someone else could test it I would be grateful. I will wait a few days before making a full release. Michael Truog made a good point in the bug issue at http://bugs.erlang.org/browse/ERL-67, in that this is not an official OTP artifact. For now the released version is 1.6.1-SNAPSHOT and is available from the repository at https://oss.sonatype.org/content/repositories/snapshots/ best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Thu Mar 31 23:12:24 2016 From: hq@REDACTED (Adam Rutkowski) Date: Thu, 31 Mar 2016 23:12:24 +0200 Subject: [erlang-questions] [ANN] Contribution needed - bugs.erlang.org In-Reply-To: <56FD0A16.3020506@ninenines.eu> References: <56FD0A16.3020506@ninenines.eu> Message-ID: <1459458744.2391267.565147346.4E61833A@webmail.messagingengine.com> On Thu, Mar 31, 2016, at 13:29, Lo?c Hoguin wrote: > Not much to look at here. > > Would love to have access to the OTP tickets rather than just the > community reports. In particular for the SSL application. I'm probably > not the only one who would like to have the full list of issues for that > one. :-) Seconded. -- /A.