From mail@REDACTED Thu Jan 1 17:24:40 2015 From: mail@REDACTED (=?utf-8?Q?Philip_M=C3=BCller?=) Date: Thu, 01 Jan 2015 17:24:40 +0100 Subject: [erlang-questions] Dialyzer failing to find simple type error Message-ID: Hi there, a few days ago I wanted to demonstrate the dialyzer to a friend who is unfamiliar with erlang/otp. Unfortunately, my simplest demo case did not work. I made a github project for it at [1] which documents the steps taken. For me, the first case (test.erl) looks like dialyzer is ignoring -Wunmatched_returns completely. The second case (test2.erl) I don't understand at all. Can someone explain what's going on there? Or should I file a bug report? Best regards Philip [1]: https://github.com/exterm/dialyzer-fail-example From ulf@REDACTED Thu Jan 1 22:15:15 2015 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 1 Jan 2015 22:15:15 +0100 Subject: [erlang-questions] how to dynamically create function to run in a different node In-Reply-To: <997694066.3161864.1420048153567.JavaMail.yahoo@jws106124.mail.bf1.yahoo.com> References: <24A572A4-2B88-435B-B25D-F8C2A842D5C9@feuerlabs.com> <997694066.3161864.1420048153567.JavaMail.yahoo@jws106124.mail.bf1.yahoo.com> Message-ID: > On 31 Dec 2014, at 18:49, Dror Mein wrote: > > Thank you! > > so parse_transform is responsible of returning {M,F,A} where A is something I don't yet understand :). > is there a way to turn {M,F,A} to fun? Well, if you want to dispatch it to another node, using a (compiled) fun is problematic. This is the main reason why I used {erl_eval, exprs, [Exprs, Bindings]}. > Is there a simpler way to create a fun that can be run on a different node? A fun can be run on a different node if the module it was compiled in has a compatible version (it doesn?t have to be the exact same version). BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.lindahl@REDACTED Thu Jan 1 23:57:21 2015 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Thu, 1 Jan 2015 23:57:21 +0100 Subject: [erlang-questions] Dialyzer failing to find simple type error In-Reply-To: References: Message-ID: Hi Philip, You are confusing Dialyzer with a static type checker (which it isn't [1]). In one of your examples, there is no problem, and in the other one, the analysis would have to be pretty strong (and have a different type domain) to find the problem. In test.erl, you have the success typings (written as specs) -spec bar() -> 1 | 0. -spec foo(1) -> 'ok' So applying foo(bar()) _can_ succeed, which makes it perfectly fine in Dialyzer's eyes. Similarly, in test2.erl you have the success typings -spec bar(any()) -> 'foo' | 'bar'. -spec foo('foo') -> 'ok'. So applying ok = foo(bar(1)), ok = foo(bar(2)). once again _can_ succeed (based on the success typings) so Dialyzer remains quiet. Arguably, if Dialyzer's analysis was strong enough to infer the type -spec bar (1) -> 'foo'; () -> 'bar', it could warn, but the above typing is way beyond Dialyzer. Tobias [1] http://www.it.uu.se/research/group/hipe/dialyzer/publications/succ_types.pdf 2015-01-01 17:24 GMT+01:00 Philip M?ller : > Hi there, > > a few days ago I wanted to demonstrate the dialyzer to a friend who is > unfamiliar with erlang/otp. > > Unfortunately, my simplest demo case did not work. I made a github project > for it at [1] which documents the steps taken. > > For me, the first case (test.erl) looks like dialyzer is ignoring > -Wunmatched_returns completely. The second case (test2.erl) I don't > understand at all. > > Can someone explain what's going on there? Or should I file a bug report? > > Best regards > Philip > > [1]: https://github.com/exterm/dialyzer-fail-example > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rj@REDACTED Fri Jan 2 12:26:59 2015 From: rj@REDACTED (Richard Jones) Date: Fri, 2 Jan 2015 11:26:59 +0000 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> Message-ID: Great to hear the IEUG and the OTP team are giving this some attention - please post a summary of what you consider the important points are, once the dust settles :) Two main concerns from me: the nature of the repo/api/tools, and appup related stuff. My instinct would be to first copy or design a well behaved HTTP api for a package repo, along with defining the relevant package metadata format (an extension of .app files, perhaps). With any luck, this will mostly be just a bunch of files on disk in a special directory structure. Then expose an HTTP search interface that indexes the metadata. Make sure it's easy to clone/mirror and run your own private package repository or a copy of the main one (eg: rsync, then trigger a reindex) Then build tools that use the aforementioned HTTP API. Pushing a new version of a package I author should make it publicly visible without undue delay. Although I'm reasonably content with things like rebar that depend on stuff on github, copying the packages to a central repo is still preferable, since it can easily work offline and deal with private repos and mirrors. Globally installed packages for erlang projects doesn't make sense to me, so a traditional package manager seems redundant - hopefully whatever tools emerge will be concerned with installing packages to a per-project deps directory. Like how rebar get-deps and npm install do. A robust API, well defined repo structure, and well maintained bunch of erlang code that uses it, would allow integration with rebar, rebar3, erlang.mk, hex, future tools, etc. I tend to deploy releases by upgrading running systems (irccloud stuff), so I'm concerned with making sure my deps are well behaved in the .app and .appup department. The main reason i have to fork and patch erlang projects is to fix app and appup related mistakes or omissions, to make release upgrades work; at the moment, that means i change the git url to my fork - I'm not sure if there's a better way. When I make a new release, and I'm feeling particularly masochistic, I try upgrading the versions of various third-party deps. Unfortunately, not many projects have valid .appups between versions. I end up forking and writing appups, or just not upgrading things. My fantasy package manager would have a command like: $ epkgm "upgrade any of my deps as long as they have an appup file from my current version, so release upgrades will work fine" ...preferably with a flag for "don't do major version bumps that break the api". But unless the package repo comes with a giant integration-testing CI system, I don't think we can enforce or guarantee something like semver. I'd need to give this more thought, but for now I'm happy to continue trusting package authors to version things appropriately (and I'd read some changelogs/readmes..). I'd be content to review which new version of any dep to upgrade to once I run that command. If I then go and fork a dep I'm using to add an appup file, it would be nice if it was easy to find by other people using that dep. This is kind of related to my ongoing struggle with figuring out which github fork is the "best" for any given erlang project. Perhaps a new package repo for erlang could address this be collecting some package usage statistics. I'm aware this appup stuff is an unnecessary burden for people not concerned with release-upgrades. I certainly don't care about it when trying something new, or building something that could tolerate being restarted for upgrades. I don't want to feel pressured into writing and testing appups before publishing a useful new library. Per-package metadata is one way, but perhaps some sort of release-upgrade-friendly channel vs. main, like how apt does stable/unstable/testing - if you have to support hot-upgrading releases, stick to the appropriate channel. RJ On 31 December 2014 at 14:39, Palmer wrote: > Joe Armstrong gmail.com> writes: > > > > Interesting - I read the papers and was impressed - I downloaded NiX > > on my MacBook and the first command in the tutorial I was following > > failed. > > > > Can anybody point me to a good tutorial other than the manual page > > and various academic papers? > > Hi Joe, > > The OS X version of Nix has been under development recently. You should > try > using the testing branch. > > Setup instructions here: > > https://nixos.org/wiki/Nix_on_OS_X > > -Palmer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger.qiu@REDACTED Fri Jan 2 18:39:40 2015 From: roger.qiu@REDACTED (Roger Qiu) Date: Sat, 3 Jan 2015 04:39:40 +1100 Subject: [erlang-questions] Using Nix as a package manager In-Reply-To: References: Message-ID: Hey all, I posted this before but it didn't go through. I saw some discussion on using Nix as a package manager and wanyed to share some info on Nix. As some have said Nix of course doesn't currently work on Windows, unless you use Cygwin. Most other language package managers when inside a NixOS or Nix system become hard to use, because they allow mutation. This has meant that the Nix has to create adapters like cabal2nix or npm2nix... etc. These convert an appropriate configuration file into a Nix file so it can be used inside the Nix system. Nix is of course language independent, so its purpose is far more than just building language libraries, it can build apps and configure the entire OS of NixOS. Some aspects of Nix I've realised over the course of my usage. The language is a full turing complete functional language. This means there's more things to learn and understand when it comes to Nix. It is not at the level of things like JSON or TOML. This of course gives you power at the cost of less simplicity. However it is a pretty useful language designed for configuration and manipulating things like attribute sets. During a build Nix still relies on the internal build scripts of the particular package. This could mean makefiles and bash files. I tend to find changing the properties of existing package nix files to be somewhat difficult. In some cases you can override properties, in others you pass parameters to the functions, and in other cases you completely create your package nix file. This problem appears whenever the package you're building has a lot of different parameters and configuration. One really cool thing is the concept of the Nix channel. So one could possibly add all of Erlangs nix files into the erlang channel instead putting it all into the official nixpkgs channel. Furthermore you also have build farms which can ship the binary to the OS instead of building it from source. Using Nix in development is quite different from using Nix in production. In development you generally want to be able to mutate the source code easily, even possibly the source code of your dependencies. But you can't really do this if you just use Nix normally to install packages. Instead you have to use Nix in a Nix-Shell or a nix container or build everything in the current directory. Some other people who are more expert in Nix might be pitching in later. This shows the difference of doing dev on Nix vs normal Nix: http://blog.lastlog.de/posts/nodejs_on_nixos_status/ Thanks, Roger -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Fri Jan 2 19:52:39 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Fri, 2 Jan 2015 19:52:39 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> Message-ID: I would think the Erlang community would want to own their package manager so it can be made to fit, potentially written in the language the community uses. But if you want to use an existing wheel, why not a wheel that runs on all planned roads, like f.ex. NPM (Node Package Manager) vs Nix that doesn't run on Windows and I understood just started on MacOS. NPM has bee used by a larger, much more varied community then the Erlang community so if not using a fork of NPM then there are at least a lot of lessons leaned there. -Mark On Fri, Jan 2, 2015 at 12:26 PM, Richard Jones wrote: > Great to hear the IEUG and the OTP team are giving this some attention - > please post a summary of what you consider the important points are, once > the dust settles :) > > Two main concerns from me: the nature of the repo/api/tools, and appup > related stuff. > > > My instinct would be to first copy or design a well behaved HTTP api for a > package repo, along with defining the relevant package metadata format (an > extension of .app files, perhaps). > With any luck, this will mostly be just a bunch of files on disk in a > special directory structure. > Then expose an HTTP search interface that indexes the metadata. > Make sure it's easy to clone/mirror and run your own private package > repository or a copy of the main one (eg: rsync, then trigger a reindex) > Then build tools that use the aforementioned HTTP API. > Pushing a new version of a package I author should make it publicly > visible without undue delay. > > Although I'm reasonably content with things like rebar that depend on > stuff on github, copying the packages to a central repo is still > preferable, since it can easily work offline and deal with private repos > and mirrors. > > Globally installed packages for erlang projects doesn't make sense to me, > so a traditional package manager seems redundant - hopefully whatever tools > emerge will be concerned with installing packages to a per-project deps > directory. Like how rebar get-deps and npm install do. A robust API, well > defined repo structure, and well maintained bunch of erlang code that uses > it, would allow integration with rebar, rebar3, erlang.mk, hex, future > tools, etc. > > > I tend to deploy releases by upgrading running systems (irccloud stuff), > so I'm concerned with making sure my deps are well behaved in the .app and > .appup department. The main reason i have to fork and patch erlang projects > is to fix app and appup related mistakes or omissions, to make release > upgrades work; at the moment, that means i change the git url to my fork - > I'm not sure if there's a better way. > > When I make a new release, and I'm feeling particularly masochistic, I try > upgrading the versions of various third-party deps. Unfortunately, not many > projects have valid .appups between versions. I end up forking and writing > appups, or just not upgrading things. My fantasy package manager would have > a command like: > > $ epkgm "upgrade any of my deps as long as they have an appup file from > my current version, so release upgrades will work fine" > > ...preferably with a flag for "don't do major version bumps that break the > api". But unless the package repo comes with a giant integration-testing CI > system, I don't think we can enforce or guarantee something like semver. > I'd need to give this more thought, but for now I'm happy to continue > trusting package authors to version things appropriately (and I'd read some > changelogs/readmes..). I'd be content to review which new version of any > dep to upgrade to once I run that command. > > If I then go and fork a dep I'm using to add an appup file, it would be > nice if it was easy to find by other people using that dep. This is kind of > related to my ongoing struggle with figuring out which github fork is the > "best" for any given erlang project. Perhaps a new package repo for erlang > could address this be collecting some package usage statistics. > > > I'm aware this appup stuff is an unnecessary burden for people not > concerned with release-upgrades. I certainly don't care about it when > trying something new, or building something that could tolerate being > restarted for upgrades. I don't want to feel pressured into writing and > testing appups before publishing a useful new library. Per-package metadata > is one way, but perhaps some sort of release-upgrade-friendly channel vs. > main, like how apt does stable/unstable/testing - if you have to support > hot-upgrading releases, stick to the appropriate channel. > > > RJ > > > > > > > > > > > On 31 December 2014 at 14:39, Palmer > wrote: > >> Joe Armstrong gmail.com> writes: >> > >> > Interesting - I read the papers and was impressed - I downloaded NiX >> > on my MacBook and the first command in the tutorial I was following >> > failed. >> > >> > Can anybody point me to a good tutorial other than the manual page >> > and various academic papers? >> >> Hi Joe, >> >> The OS X version of Nix has been under development recently. You should >> try >> using the testing branch. >> >> Setup instructions here: >> >> https://nixos.org/wiki/Nix_on_OS_X >> >> -Palmer >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Fri Jan 2 20:34:42 2015 From: imantc@REDACTED (Imants Cekusins) Date: Fri, 2 Jan 2015 20:34:42 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> Message-ID: Is there really a need to try to compete with OS-specific or OS-agnostic PM's? For Erlang PM (which is being discussed here), why not limit the scope to erlang-only packages & deps? E.g., Java PM's (Ivy, Maven, Ant) focus on Java code, don't they? One of the key benefits of building / designating an "official" Erlang PM would be ease of use of | conversion from a variety of existing Erlang-specific PM's/build tools. In other words, "the Erlang PM" would install 90% of Erlang packages in the wild, or ideally from a central repository. ? From nyarly@REDACTED Fri Jan 2 20:53:39 2015 From: nyarly@REDACTED (Judson Lester) Date: Fri, 02 Jan 2015 19:53:39 +0000 Subject: [erlang-questions] Erlang package manager References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> Message-ID: I'm really concerned to see NPM and Maven presented as anything like options regarding a package manager. On the one hand, there's a significant difference between a build tool (Maven, Ant) and a package manager. On the other, NPM relies on a feature of Node (using a tree to look up dependencies at runtime, rather than a path) that isn't present in Erlang (and, in my opinion, shouldn't be.) NPM is also weak in terms of deployment (shrinkwrap is a big hassle) - and still very young (witness the change in behavior regarding semver.) If Erlang were to emulate another packaging system, I'd recommend looking at rubygems and bundler in the Ruby community. It's biggest failing is that the two projects were started separately, and that there's a reliance on outside projects to deal with changes in VM for development. The strongest reason I can see to build an Erlang centric PM is to support Erlang's existing deployment behaviors i.e. releases. On Fri Jan 02 2015 at 11:34:53 AM Imants Cekusins wrote: > Is there really a need to try to compete with OS-specific or OS-agnostic > PM's? > > For Erlang PM (which is being discussed here), why not limit the scope > to erlang-only packages & deps? > > E.g., Java PM's (Ivy, Maven, Ant) focus on Java code, don't they? > > One of the key benefits of building / designating an "official" Erlang > PM would be ease of use of | conversion from a variety of existing > Erlang-specific PM's/build tools. > > In other words, "the Erlang PM" would install 90% of Erlang packages > in the wild, or ideally from a central repository. > > ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Fri Jan 2 21:02:56 2015 From: t@REDACTED (Tristan Sloughter) Date: Fri, 02 Jan 2015 14:02:56 -0600 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> Message-ID: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> > The strongest reason I can see to build an Erlang centric PM is to > support Erlang's existing deployment behaviors i.e. releases. An Erlang centric PM should not do anything regarding releases. The PM should focus on development and not for deployment. Releases should be used to build target systems and deployed as such, maybe the libs that are used to build the release/target system were fetched with the PM at some point but the PM needs no knowledge of releases. As for npm, gems, bundler, maven I know little about any of them. Except that one is worth millions of dollars or something... And relies on CouchDB and has hilarious outcomes when someone breaks semver rules. Anyway... I don't think there is anything people are proposing we take directly from those, instead to learn from prior art -- I'd suggest more opam/cabal/cpan than those as useful prior art though. Cargo, http://crates.io, is an example of the condensed knowledge taken from those projects into a new package manager. But I do like Maven's(and thus Leiningens) dependency handling, highest spot on the dep tree wins. And we are using that for rebar3. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From nx@REDACTED Fri Jan 2 21:10:35 2015 From: nx@REDACTED (nx) Date: Fri, 2 Jan 2015 15:10:35 -0500 Subject: [erlang-questions] Erlang package manager In-Reply-To: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Gah! I wanted to call the Erlang packages Crates. And have the package index be called Logistics. Thought those names went well with Rebar. Oh well. Guess we'll have to call Erlang packages Lollipops or something and have the package index be called Candyland. That's the only viable naming alternative at this point. On Fri, Jan 2, 2015 at 3:02 PM, Tristan Sloughter wrote: > The strongest reason I can see to build an Erlang centric PM is to support > Erlang's existing deployment behaviors i.e. releases. > > > An Erlang centric PM should not do anything regarding releases. > > The PM should focus on development and not for deployment. Releases should > be used to build target systems and deployed as such, maybe the libs that > are used to build the release/target system were fetched with the PM at some > point but the PM needs no knowledge of releases. > > As for npm, gems, bundler, maven I know little about any of them. Except > that one is worth millions of dollars or something... And relies on CouchDB > and has hilarious outcomes when someone breaks semver rules. Anyway... I > don't think there is anything people are proposing we take directly from > those, instead to learn from prior art -- I'd suggest more opam/cabal/cpan > than those as useful prior art though. > > Cargo, http://crates.io, is an example of the condensed knowledge taken from > those projects into a new package manager. > > But I do like Maven's(and thus Leiningens) dependency handling, highest spot > on the dep tree wins. And we are using that for rebar3. > > Tristan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From tuncer.ayaz@REDACTED Fri Jan 2 21:49:46 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 3 Jan 2015 03:19:46 +0630 Subject: [erlang-questions] travis-ci erlang support broken? Message-ID: Does anybody know if the following travis-ci Erlang distro breakage is due to some kind of recent compartmentalization changes, and if so, what changes are required to continue using travis-ci for Erlang? Can we use only short names for nodes now? curl https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt | tail -n 45 From mark.nijhof@REDACTED Fri Jan 2 21:54:24 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Fri, 2 Jan 2015 21:54:24 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Hehe I knew there would be some protest when I mentioned NPM, but look at their success, probably a big reason for the adaption of Node, it makes getting started easy. Erlang.mk does a lot towards this as well already, discovery is an issue though. On Fri, Jan 2, 2015 at 9:10 PM, nx wrote: > Gah! I wanted to call the Erlang packages Crates. And have the package > index be called Logistics. Thought those names went well with Rebar. > > Oh well. Guess we'll have to call Erlang packages Lollipops or > something and have the package index be called Candyland. That's the > only viable naming alternative at this point. > > On Fri, Jan 2, 2015 at 3:02 PM, Tristan Sloughter wrote: > > The strongest reason I can see to build an Erlang centric PM is to > support > > Erlang's existing deployment behaviors i.e. releases. > > > > > > An Erlang centric PM should not do anything regarding releases. > > > > The PM should focus on development and not for deployment. Releases > should > > be used to build target systems and deployed as such, maybe the libs that > > are used to build the release/target system were fetched with the PM at > some > > point but the PM needs no knowledge of releases. > > > > As for npm, gems, bundler, maven I know little about any of them. Except > > that one is worth millions of dollars or something... And relies on > CouchDB > > and has hilarious outcomes when someone breaks semver rules. Anyway... I > > don't think there is anything people are proposing we take directly from > > those, instead to learn from prior art -- I'd suggest more > opam/cabal/cpan > > than those as useful prior art though. > > > > Cargo, http://crates.io, is an example of the condensed knowledge taken > from > > those projects into a new package manager. > > > > But I do like Maven's(and thus Leiningens) dependency handling, highest > spot > > on the dep tree wins. And we are using that for rebar3. > > > > Tristan > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Fri Jan 2 22:20:13 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 2 Jan 2015 22:20:13 +0100 Subject: [erlang-questions] travis-ci erlang support broken? In-Reply-To: References: Message-ID: On Fri, Jan 2, 2015 at 9:49 PM, Tuncer Ayaz wrote: > Does anybody know if the following travis-ci Erlang distro breakage is > due to some kind of recent compartmentalization changes, and if so, > what changes are required to continue using travis-ci for Erlang? > > Can we use only short names for nodes now? > > curl https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt > | tail -n 45 The relevant bit: {error_logger,{{2015,1,1},{13,48,39}}, "Can't set long node name!\nPlease check your configuration\n",[]} Also, I should have tried -sname before asking the list: https://github.com/rebar/rebar/pull/421 From imantc@REDACTED Sat Jan 3 10:47:47 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 3 Jan 2015 10:47:47 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Wouldn't it be handy if the new PM used Erlang syntax to specify configuration? I.e., config could be an .hrl or .erl file. There would be defined types for config items. Maybe use records. Then Dialyzer could be used to validate config. The .app spec file could specify the config file(s) used for this .app ? From n.oxyde@REDACTED Sat Jan 3 12:53:15 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 3 Jan 2015 12:53:15 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Le 3 janv. 2015 ? 10:47, Imants Cekusins a ?crit : > Wouldn't it be handy if the new PM used Erlang syntax to specify configuration? > > I.e., config could be an .hrl or .erl file. > > There would be defined types for config items. Maybe use records. > > Then Dialyzer could be used to validate config. > The .app spec file could specify the config file(s) used for this .app > > ? Turing-complete package descriptions sound like asking for problems. From imantc@REDACTED Sat Jan 3 13:27:48 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 3 Jan 2015 13:27:48 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Well a parse transform could sanitize the config files so only acceptable statements are used. Maybe use a different extension for the config files. The key benefits are familiar syntax & syntax checks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Jan 3 15:54:14 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sat, 3 Jan 2015 15:54:14 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Hi everyone! I would like to address some of the questions and comments in this discussion. I will be brief and get into more details once I get back to Stockholm. 1. The Package Manager client should (will/must) be written completely in Erlang. There are many reasons for this. For instance, it is easier to maintain an application written in a language we have some familiarity with. It is a nicer fit. We can have the client the client in the bootstrap and let it upgrade itself once downloaded. I don't think OTP can really be satisfied with anything else than a PM written it in Erlang. =) 2. Scope it down. For starters the Package Manager should only handle Erlang applications. Things like Erl_Interface and JInterface are the odd one outs in the lib directory today and they don't really fit here. At least not how I envision it. I'm sure we could jury-rig something but we should not start there. This also mean that we should *not* handle OS dev-libs in our package-manager. We can have a specialized description field for such things, telling the developer he needs to install them prior to installing an Erlang application but we should not handle it beyond that. 3. Protocols first. I saw someone mentioning this already and it made me so happy. Protocols first is perhaps a misnomer but it is important anyway. We have several languages that runs ontop of beam or Erlang. Elixir has its own package manger Hex (client) and Hexweb (server) and I think it is a good thing if we use compatible protocols. Http and restful protocols is all the rage and I don't any reason to sidestep that. My vision here is a common protocol bus and the client can query any server for Erlang applications, i.e. Hex can query an Erlang Application provider for updates and install them. I think this only affect us (Erlang) by pressuring us to formalize our rest protocols / api .. but that is a good thing. I hesitate to discuss the naming of this thing in fear of something exploding. But, I will throw my hat into the race .. Gear (client) and Gearbox (server). Gear - Erlang Application Repositories. With commands like "gear up .." for upgrading packages and "gear down" for downgrading them. Great discussion, have to run, Bj?rn-Egil - did not review my comments before submitting. 2015-01-03 13:27 GMT+01:00 Imants Cekusins : > Well a parse transform could sanitize the config files so only acceptable > statements are used. > > Maybe use a different extension for the config files. > > The key benefits are familiar syntax & syntax checks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sat Jan 3 16:26:07 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 3 Jan 2015 16:26:07 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Isomorphic Javascript is all the rage too, doesn't make it good. Nevermind that most restful protocols aren't restful. A set of static files CPAN-style sounds way easier to maintain than a complex HTTP-based protocol. Le 3 janv. 2015 ? 15:54, Bj?rn-Egil Dahlberg a ?crit : > Http and restful protocols is all the rage and I don't any reason to sidestep that. From essen@REDACTED Sat Jan 3 16:46:54 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 03 Jan 2015 16:46:54 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: <54A80EEE.9040801@ninenines.eu> On 01/03/2015 04:26 PM, Anthony Ramine wrote: > A set of static files CPAN-style sounds way easier to maintain than a complex HTTP-based protocol. A thousand times this. -- Lo?c Hoguin http://ninenines.eu From tuncer.ayaz@REDACTED Sat Jan 3 18:02:19 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 3 Jan 2015 18:02:19 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A80EEE.9040801@ninenines.eu> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A80EEE.9040801@ninenines.eu> Message-ID: On Sat, Jan 3, 2015 at 4:46 PM, Loic Hoguin wrote: > On 01/03/2015 04:26 PM, Anthony Ramine wrote: > > > > A set of static files CPAN-style sounds way easier to maintain > > than a complex HTTP-based protocol. > > > A thousand times this. Yes, and it's also a prerequisite for being easy to mirror properly. From tuncer.ayaz@REDACTED Sat Jan 3 18:16:51 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 3 Jan 2015 18:16:51 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: On Sat, Jan 3, 2015 at 3:54 PM, Bjorn-Egil Dahlberg wrote: > Hi everyone! > > I would like to address some of the questions and comments in this > discussion. I will be brief and get into more details once I get > back to Stockholm. > > 1. The Package Manager client should (will/must) be written > completely in Erlang. There are many reasons for this. For instance, > it is easier to maintain an application written in a language we > have some familiarity with. It is a nicer fit. We can have the > client the client in the bootstrap and let it upgrade itself once > downloaded. I don't think OTP can really be satisfied with anything > else than a PM written it in Erlang. =) +1 > 2. Scope it down. For starters the Package Manager should only > handle Erlang applications. Things like Erl_Interface and JInterface > are the odd one outs in the lib directory today and they don't > really fit here. At least not how I envision it. I'm sure we could > jury-rig something but we should not start there. This also mean > that we should *not* handle OS dev-libs in our package-manager. We > can have a specialized description field for such things, telling > the developer he needs to install them prior to installing an Erlang > application but we should not handle it beyond that. I mostly agree, but I also want to stress that we have to figure out and implement extensions to .ez and .app (embedded in .ez) infrastructure, and maybe prepend or append (probably easier) metadata in archives. As we use a VM anyway, I don't see a good reason not to support multi-arch fat archives which contain .beam, .so, .dll, for ease of deployment. Also, please consider my remarks in: http://erlang.org/pipermail/erlang-questions/2014-December/082132.html If something's not clear in that post, please ask for clarification. I've tried to list things which - these days - are forgotten or mis-implemented in most non-CPAN/non-CTAN managers. > 3. Protocols first. I saw someone mentioning this already and it > made me so happy. Protocols first is perhaps a misnomer but it is > important anyway. We have several languages that runs ontop of beam > or Erlang. Elixir has its own package manger Hex (client) and Hexweb > (server) and I think it is a good thing if we use compatible > protocols. Http and restful protocols is all the rage and I don't > any reason to sidestep that. My vision here is a common protocol bus > and the client can query any server for Erlang applications, i.e. > Hex can query an Erlang Application provider for updates and install > them. I think this only affect us (Erlang) by pressuring us to > formalize our rest protocols / api .. but that is a good thing. As long as we're not held back by limitations of existing protocols, and are free to improve/extend, sure, it's a good approach. > I hesitate to discuss the naming of this thing in fear of something > exploding. But, I will throw my hat into the race .. Gear (client) > and Gearbox (server). Gear - Erlang Application Repositories. With > commands like "gear up .." for upgrading packages and "gear down" > for downgrading them. Why not integrate it as subcommands in 'erl', as we already have a distinctive 3-letter CLI slot occupied in $PATH? Seriously. From mark.nijhof@REDACTED Sat Jan 3 18:55:02 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sat, 3 Jan 2015 18:55:02 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: +1 No seperate tool to install -1 No seperate bug fixes, and a larger attack surface in production as you would always deploy the PM as well. Why not integrate it as subcommands in 'erl', as we already have a > distinctive 3-letter CLI slot occupied in $PATH? Seriously. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Sat Jan 3 18:58:17 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sat, 3 Jan 2015 18:58:17 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Actually how would you manage packages for older apps running on anything then latest? On Saturday, January 3, 2015, Mark Nijhof wrote: > > > +1 > No seperate tool to install > > -1 > No seperate bug fixes, and a larger attack surface in production as you > would always deploy the PM as well. > > Why not integrate it as subcommands in 'erl', as we already have a >> distinctive 3-letter CLI slot occupied in $PATH? Seriously. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Jan 3 19:10:02 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sat, 3 Jan 2015 19:10:02 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: 2015-01-03 18:16 GMT+01:00 Tuncer Ayaz : > > Why not integrate it as subcommands in 'erl', as we already have a > distinctive 3-letter CLI slot occupied in $PATH? Seriously. > I don't want to pollute erl more than necessary. It's not pretty as it is now. Also, older erl won't understand PM commands. A separate gear escript will be is just fine, clean and separate. If we want could introduce something like "erl --gear " which would do the same thing as the escript but I'm not a big fan of this idea. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Jan 3 19:12:09 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 03 Jan 2015 19:12:09 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: <54A830F9.6020700@ninenines.eu> On 01/03/2015 03:54 PM, Bj?rn-Egil Dahlberg wrote: > 1. The Package Manager client should (will/must) be written completely > in Erlang. There are many reasons for this. For instance, it is easier > to maintain an application written in a language we have some > familiarity with. It is a nicer fit. We can have the client the client > in the bootstrap and let it upgrade itself once downloaded. I don't > think OTP can really be satisfied with anything else than a PM written > it in Erlang. =) I'm mostly fine with this, but: please make sure the package manager client is very much like erlc and does not attempt to start a huge VM to do 2 HTTP queries. Escripts, erl -s/-eval and the likes are not good for short-lived command line programs. I'm concerned with "let it upgrade itself once downloaded". It sounds like the package manager has to handle not only OTP/library applications, but also executables. Scope needs to be clarified. > 2. Scope it down. For starters the Package Manager should only handle > Erlang applications. Things like Erl_Interface and JInterface are the > odd one outs in the lib directory today and they don't really fit here. > At least not how I envision it. I'm sure we could jury-rig something but > we should not start there. This also mean that we should *not* handle OS > dev-libs in our package-manager. We can have a specialized description > field for such things, telling the developer he needs to install them > prior to installing an Erlang application but we should not handle it > beyond that. Packages in the index should only be Erlang applications, yes. But the package manager client cares little what the packages contain. All it needs at most is a list of files. It doesn't need to know what the files are. Don't prevent people to make private repositories that work for them if they want to. > I hesitate to discuss the naming of this thing in fear of something > exploding. But, I will throw my hat into the race .. Gear (client) and > Gearbox (server). Gear - Erlang Application Repositories. With commands > like "gear up .." for upgrading packages and "gear down" for downgrading > them. Good job mentioning explosions and Gearbox in the same context. Namespace for things that try to say "packages" or "manager" or "repositories" is fairly crowded, so my vote goes to "tomato". But if you want to keep the name on topic then I vote for "perl", Packages for ERL. -- Lo?c Hoguin http://ninenines.eu From imantc@REDACTED Sat Jan 3 19:20:23 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 3 Jan 2015 19:20:23 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A830F9.6020700@ninenines.eu> References: <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> Message-ID: "erlp" seems to be free ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Jan 3 19:40:17 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sat, 3 Jan 2015 19:40:17 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A830F9.6020700@ninenines.eu> References: <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> Message-ID: 2015-01-03 19:12 GMT+01:00 Lo?c Hoguin : > On 01/03/2015 03:54 PM, Bj?rn-Egil Dahlberg wrote: > >> 1. The Package Manager client should (will/must) be written completely >> in Erlang. There are many reasons for this. For instance, it is easier >> to maintain an application written in a language we have some >> familiarity with. It is a nicer fit. We can have the client the client >> in the bootstrap and let it upgrade itself once downloaded. I don't >> think OTP can really be satisfied with anything else than a PM written >> it in Erlang. =) >> > > I'm mostly fine with this, but: please make sure the package manager > client is very much like erlc and does not attempt to start a huge VM to do > 2 HTTP queries. Escripts, erl -s/-eval and the likes are not good for > short-lived command line programs. > Well, I know this is a pet peeve for you =) I don't think we should let the startup time limit our design choice here. I believe the startup time is tightly coupled to loading all the modules and then letting the beam-loader transform it. This could probably be improved but I don't know how easily. Ideally we would like heat up a running beam with all the modules loaded and then dump the memory to disk .. appended to some binary, e.g. beam. That should shave off some startup time. This is however not in the scope of the package manager. > > I'm concerned with "let it upgrade itself once downloaded". It sounds like > the package manager has to handle not only OTP/library applications, but > also executables. Scope needs to be clarified. I'm getting ahead om myself here. Think of it as having a precompiled package manager client application in the bootstrap, like the erlang compiler. It is an older version, if you want to have a newer version it can upgrade itself just like any other application. It is nothing special about it. > > 2. Scope it down. For starters the Package Manager should only handle >> Erlang applications. Things like Erl_Interface and JInterface are the >> odd one outs in the lib directory today and they don't really fit here. >> At least not how I envision it. I'm sure we could jury-rig something but >> we should not start there. This also mean that we should *not* handle OS >> dev-libs in our package-manager. We can have a specialized description >> field for such things, telling the developer he needs to install them >> prior to installing an Erlang application but we should not handle it >> beyond that. >> > > Packages in the index should only be Erlang applications, yes. But the > package manager client cares little what the packages contain. All it needs > at most is a list of files. It doesn't need to know what the files are. > Don't prevent people to make private repositories that work for them if > they want to. True, but let's limit this to Erlang applications for starters. Make sure we can extend stuff and then take it from there. > I hesitate to discuss the naming of this thing in fear of something >> exploding. But, I will throw my hat into the race .. Gear (client) and >> Gearbox (server). Gear - Erlang Application Repositories. With commands >> like "gear up .." for upgrading packages and "gear down" for downgrading >> them. >> > > Good job mentioning explosions and Gearbox in the same context. > > Namespace for things that try to say "packages" or "manager" or > "repositories" is fairly crowded, so my vote goes to "tomato". But if you > want to keep the name on topic then I vote for "perl", Packages for ERL. *boom* // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Jan 3 19:47:47 2015 From: t@REDACTED (Tristan Sloughter) Date: Sat, 03 Jan 2015 12:47:47 -0600 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> Message-ID: <1420310867.699093.209176441.70BD440B@webmail.messagingengine.com> > Well, I know this is a pet peeve for you =) Yea, I hope this project does not care about any of that. I don't think the VM start time matters even in a build tool and that is run much more often than a package manager! I'd be surprised if anyone even notices... Focusing on that 200ms would be a far greater waste of time than any time lost on start time. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Jan 3 19:51:43 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 03 Jan 2015 19:51:43 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> Message-ID: <54A83A3F.7020407@ninenines.eu> On 01/03/2015 07:40 PM, Bj?rn-Egil Dahlberg wrote: > Well, I know this is a pet peeve for you =) > I don't think we should let the startup time limit our design choice > here. I believe the startup time is tightly coupled to loading all the > modules and then letting the beam-loader transform it. This could > probably be improved but I don't know how easily. Ideally we would like > heat up a running beam with all the modules loaded and then dump the > memory to disk .. appended to some binary, e.g. beam. That should shave > off some startup time. > > This is however not in the scope of the package manager. I'm just saying build it as an ELF (or equivalent) instead of a plain escript here. There's nothing special to do, it's what you guys have done with pretty much everything in the bin/ folder. I'm not sure why that should change. :-) > True, but let's limit this to Erlang applications for starters. Make > sure we can extend stuff and then take it from there. All I'm saying is don't work more to prevent people to do things that might work for them. No need to put extra checks or things like this. The thinking should be Erlang specific but the implementation should not enforce that (if possible). -- Lo?c Hoguin http://ninenines.eu From essen@REDACTED Sat Jan 3 20:00:07 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 03 Jan 2015 20:00:07 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <1420310867.699093.209176441.70BD440B@webmail.messagingengine.com> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> <1420310867.699093.209176441.70BD440B@webmail.messagingengine.com> Message-ID: <54A83C37.3000105@ninenines.eu> On 01/03/2015 07:47 PM, Tristan Sloughter wrote: > Well, I know this is a pet peeve for you =) > > Yea, I hope this project does not care about any of that. I don't think > the VM start time matters even in a build tool and that is run much more > often than a package manager! I'd be surprised if anyone even notices... > Focusing on that 200ms would be a far greater waste of time than any > time lost on start time. If it was extra work, then yes. But here it's just a matter of choosing between 2 different ways of running programs, and they both *already exist*. There's no extra work involved. I just want to make sure the right one is chosen. -- Lo?c Hoguin http://ninenines.eu From the.silly.sad@REDACTED Sat Jan 3 12:55:22 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Sat, 03 Jan 2015 12:55:22 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: <54A7D8AA.70805@gmail.com> On 01/03/2015 12:53 PM, Anthony Ramine wrote: > Le 3 janv. 2015 ? 10:47, Imants Cekusins a ?crit : > >> Wouldn't it be handy if the new PM used Erlang syntax to specify configuration? >> >> I.e., config could be an .hrl or .erl file. >> >> There would be defined types for config items. Maybe use records. >> >> Then Dialyzer could be used to validate config. >> The .app spec file could specify the config file(s) used for this .app >> >> ? > > Turing-complete package descriptions sound like asking for problems. it was only a question of SYNTAX, as far as i understand it. and, yes re-use of already existing syntax is nice. From the.silly.sad@REDACTED Sat Jan 3 16:48:46 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Sat, 03 Jan 2015 16:48:46 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A80EEE.9040801@ninenines.eu> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A80EEE.9040801@ninenines.eu> Message-ID: <54A80F5E.8030702@gmail.com> On 01/03/2015 04:46 PM, Lo?c Hoguin wrote: > On 01/03/2015 04:26 PM, Anthony Ramine wrote: >> A set of static files CPAN-style sounds way easier to maintain than a >> complex HTTP-based protocol. > > A thousand times this. how is one opposite to another? former is a fileformat latter is a network protocol. From hugo@REDACTED Sun Jan 4 05:01:15 2015 From: hugo@REDACTED (Hugo Mills) Date: Sun, 4 Jan 2015 04:01:15 +0000 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A80F5E.8030702@gmail.com> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A80EEE.9040801@ninenines.eu> <54A80F5E.8030702@gmail.com> Message-ID: <20150104040115.GD32182@carfax.org.uk> On Sat, Jan 03, 2015 at 04:48:46PM +0100, the.silly.sad wrote: > On 01/03/2015 04:46 PM, Lo?c Hoguin wrote: > >On 01/03/2015 04:26 PM, Anthony Ramine wrote: > >>A set of static files CPAN-style sounds way easier to maintain than a > >>complex HTTP-based protocol. > > > >A thousand times this. > > how is one opposite to another? > former is a fileformat latter is a network protocol. If the network protocol implies a stateful interaction, or some direct manipulation of an underlying non-file-based data model, then it's probably going to be unusable in practice. Anthony is basically saying that it needs to be a true REST model, in the sense of REST described (and implied) by Fielding. If (and only if?) that's the case, then the serverside of the protocol can be implemented by a set of static files. If you can't do it that way for any given proposal, you don't have a RESTful system, and you should probably abandon the proposal. Hugo. -- Hugo Mills | If it's December 1941 in Casablanca, what time is it hugo@REDACTED carfax.org.uk | in New York? http://carfax.org.uk/ | PGP: 65E74AC0 | Rick Blaine, Casablanca -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From essen@REDACTED Sun Jan 4 10:32:34 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sun, 04 Jan 2015 10:32:34 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A80F5E.8030702@gmail.com> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A80EEE.9040801@ninenines.eu> <54A80F5E.8030702@gmail.com> Message-ID: <54A908B2.5030908@ninenines.eu> On 01/03/2015 04:48 PM, the.silly.sad wrote: > On 01/03/2015 04:46 PM, Lo?c Hoguin wrote: >> On 01/03/2015 04:26 PM, Anthony Ramine wrote: >>> A set of static files CPAN-style sounds way easier to maintain than a >>> complex HTTP-based protocol. >> >> A thousand times this. > > how is one opposite to another? > former is a fileformat latter is a network protocol. Think about public and private mirrors. A set of files is easy to copy and you can put them in a folder already served by the currently running Web server. There's no requirement on what software is used, it could be any HTTP server. That also opens the possibility of having the files locally and not needing a server at all. A protocol? You have to setup the software, update it, and it's not clear how mirrors would work but anything that isn't plain files will require more work than copying files. Also note it's much easier to find people willing to serve files than people willing to install unknown software. On the note of mirrors: considering Erlang is all about reliability, I think it would be an offense if it did not come with at least 3 separate mirrors (maybe 1 America 1 Europe 1 Asia) from scratch, that the manager knows about and can switch to another automatically if one gets down. The list of mirrors of course being ultimately configurable. -- Lo?c Hoguin http://ninenines.eu From tuncer.ayaz@REDACTED Sun Jan 4 11:28:48 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 11:28:48 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: On Sat, Jan 3, 2015 at 6:55 PM, Mark Nijhof wrote: > > -1 > No seperate bug fixes, and a larger attack surface in production as > you would always deploy the PM as well. Given the quick releases for erlang/otp.git, this wouldn't be a problem. From tuncer.ayaz@REDACTED Sun Jan 4 11:30:40 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 11:30:40 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: On Sat, Jan 3, 2015 at 6:58 PM, Mark Nijhof wrote: > Actually how would you manage packages for older apps running on > anything then latest? I don't understand the problem you're describing, and what it has to do with how the PM is distributed. > On Saturday, January 3, 2015, Mark Nijhof wrote: > > > > > > > > +1 > > No seperate tool to install > > > > -1 > > No seperate bug fixes, and a larger attack surface in production > > as you would always deploy the PM as well. From tuncer.ayaz@REDACTED Sun Jan 4 11:39:13 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 11:39:13 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: On Sat, Jan 3, 2015 at 7:10 PM, Bjorn-Egil Dahlberg wrote: > > > 2015-01-03 18:16 GMT+01:00 Tuncer Ayaz : > > > > > > Why not integrate it as subcommands in 'erl', as we already have a > > distinctive 3-letter CLI slot occupied in $PATH? Seriously. > > > I don't want to pollute erl more than necessary. It's not pretty as > it is now. To avoid namespace pollution, we could use a short command prefix, but I don't really mind a separate tool in $PATH, as long as it's not called erlang-package-manager, so I'll stop arguing for erl subcommands but still address your concerns below. > Also, older erl won't understand PM commands. So? Older erl also might not understand a new option that only exists in a newer version. > A separate gear escript will be is just fine, clean and separate. If > we want could introduce something like "erl --gear > " which would do the same thing as the escript but I'm not > a big fan of this idea. This idea does indeed sound bad, and my suggestion was more like "erl pin", "erl source", etc. From vladdu55@REDACTED Sun Jan 4 11:54:05 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 4 Jan 2015 11:54:05 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Hi, On Sun, Jan 4, 2015 at 11:30 AM, Tuncer Ayaz wrote: > On Sat, Jan 3, 2015 at 6:58 PM, Mark Nijhof wrote: > > Actually how would you manage packages for older apps running on > > anything then latest? > > I don't understand the problem you're describing, and what it has to > do with how the PM is distributed. > I'm not sure if this is what Mark means, but if the PM is not a separate executable/script/escript from erl, then if I use Erlang 17 or earlier, it won't be easily available. A separate script can be made available independently of the OTP releases, for the benefit of legacy applications. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun Jan 4 12:01:27 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 12:01:27 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: On Sat, Jan 3, 2015 at 6:16 PM, Tuncer Ayaz wrote: > On Sat, Jan 3, 2015 at 3:54 PM, Bjorn-Egil Dahlberg wrote: > > Hi everyone! [...] > > 3. Protocols first. I saw someone mentioning this already and it > > made me so happy. Protocols first is perhaps a misnomer but it is > > important anyway. We have several languages that runs ontop of > > beam or Erlang. Elixir has its own package manger Hex (client) and > > Hexweb (server) and I think it is a good thing if we use > > compatible protocols. Http and restful protocols is all the rage > > and I don't any reason to sidestep that. My vision here is a > > common protocol bus and the client can query any server for Erlang > > applications, i.e. Hex can query an Erlang Application provider > > for updates and install them. I think this only affect us (Erlang) > > by pressuring us to formalize our rest protocols / api .. but that > > is a good thing. > > As long as we're not held back by limitations of existing protocols, > and are free to improve/extend, sure, it's a good approach. Thinking about, it appears what you're actually suggesting is a smart http protocol which would disallow easy mirroring. If that's the case, then it's a bad idea. I've explained this in my initial response to Bruce, but I'll try to summarize the benefits of a simple files+dirs structure served via http/ftp/_whatever_: * Like mirrors for CTAN, anyone who wishes to host a mirror can do so easily alongside CTAN, *nix distros, etc., without the need for a special CGI script or extra http daemon. This greatly increases the reliability and availability by avoiding the need for a centrally managed and to-be-accessed smart host that has to be paid for and administered. No matter how professional and CDN'ed something is, it will eventually go down for a period. Using a conventionally mirror'able structure avoids "is foo.erlang.org down?" questions and broken builds, as users will by convention use mirrors instead of a central location. For this to be a non-issue, we'd have to have van Jacobson's content-centric networking (or one of the derivatives) in addition to more reliable global network links. * It's easy to set up public or private mirrors. * In case of problems, it gives you the option of restoring or checking the integrity of files by making use several independent, trusted mirrors that hadn't been updated to the problematic state yet. That said, it's perfectly fine to have one or more registries where stuff is published to and mirrored from. From mark.nijhof@REDACTED Sun Jan 4 12:05:54 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sun, 4 Jan 2015 12:05:54 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: References: <54980224.8070106@ninenines.eu> <1419280721.3759353.205867005.30015C38@webmail.messagingengine.com> <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> Message-ID: Exactly what I mean, also if the PM would be used for building releases as well then a version conflict would be really painful. I.e. Making a release that embeds OTP 15 from a PM that is embedded in OTP 17 sounds like a challange. On Sunday, January 4, 2015, Vlad Dumitrescu wrote: > Hi, > > On Sun, Jan 4, 2015 at 11:30 AM, Tuncer Ayaz > wrote: > >> On Sat, Jan 3, 2015 at 6:58 PM, Mark Nijhof wrote: >> > Actually how would you manage packages for older apps running on >> > anything then latest? >> >> I don't understand the problem you're describing, and what it has to >> do with how the PM is distributed. >> > > I'm not sure if this is what Mark means, but if the PM is not a separate > executable/script/escript from erl, then if I use Erlang 17 or earlier, it > won't be easily available. A separate script can be made available > independently of the OTP releases, for the benefit of legacy applications. > > best regards, > Vlad > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun Jan 4 12:09:40 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 12:09:40 +0100 Subject: [erlang-questions] Erlang package manager In-Reply-To: <54A83C37.3000105@ninenines.eu> References: <1420228976.383269.208922205.38DDC0CC@webmail.messagingengine.com> <54A830F9.6020700@ninenines.eu> <1420310867.699093.209176441.70BD440B@webmail.messagingengine.com> <54A83C37.3000105@ninenines.eu> Message-ID: On Sat, Jan 3, 2015 at 8:00 PM, Loic Hoguin wrote: > On 01/03/2015 07:47 PM, Tristan Sloughter wrote: > > > > Well, I know this is a pet peeve for you =) > > > > Yea, I hope this project does not care about any of that. I don't > > think the VM start time matters even in a build tool and that is > > run much more often than a package manager! I'd be surprised if > > anyone even notices... Focusing on that 200ms would be a far > > greater waste of time than any time lost on start time. > > > If it was extra work, then yes. But here it's just a matter of > choosing between 2 different ways of running programs, and they both > *already exist*. There's no extra work involved. I just want to make > sure the right one is chosen. Just to be clear, running compile:file via erlc isn't magically quicker to start just because erlc is a native executable. What makes it quicker is that the VM is started differently compared to invoking "erl -eval". erlc also spawns the VM, just differently. With that being said, I agree with Loic that for interactive tools it makes a difference, and we should aim to get escript startup time back to R13 levels. If that's hard to achieve, Loic's suggestion of packaging epam/gears/erlpm like erlc may be the right option. From tuncer.ayaz@REDACTED Sun Jan 4 12:23:36 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 12:23:36 +0100 Subject: [erlang-questions] Erlang package manager Message-ID: On Sun, Jan 4, 2015 at 12:05 PM, Mark Nijhof wrote: > Exactly what I mean, also if the PM would be used for building > releases as well then a version conflict would be really painful. > I.e. Making a release that embeds OTP 15 from a PM that is embedded > in OTP 17 sounds like a challange. Ah, I see. The PM will use one VM for getting itself off the ground, and assuming for a moment it should be concerned with embedding erts, which I'm not sure is its job, it already has to have a way of being told what VM to find how. Therefore, I don't see a big problem. Personally, I don't think it's the PM's job to embed erts and required packages. The PM should be concerned with packaging .app, .beam, .so/.dll, .whatever in a .ez archive, and reltool (or alternatives) will bundle a release. But, I must say I'm also thinking that a more coherent packaging of .ez archives as releases with the relup/appup files, and promoting that to be the conventional deployment mechanism (optionally with erts), has a certain appeal, but something tells me its out of the scope and natural responsibility of a PM. > On Sunday, January 4, 2015, Vlad Dumitrescu wrote: > > > > Hi, > > > > On Sun, Jan 4, 2015 at 11:30 AM, Tuncer Ayaz wrote: > > > > > > On Sat, Jan 3, 2015 at 6:58 PM, Mark Nijhof wrote: > > > > Actually how would you manage packages for older apps running on > > > > anything then latest? > > > > > > I don't understand the problem you're describing, and what it has to > > > do with how the PM is distributed. > > > > > > I'm not sure if this is what Mark means, but if the PM is not a > > separate executable/script/escript from erl, then if I use Erlang > > 17 or earlier, it won't be easily available. A separate script can > > be made available independently of the OTP releases, for the > > benefit of legacy applications. From tuncer.ayaz@REDACTED Sun Jan 4 12:38:01 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 12:38:01 +0100 Subject: [erlang-questions] travis-ci erlang support broken? In-Reply-To: References: Message-ID: On Fri, Jan 2, 2015 at 10:20 PM, Tuncer Ayaz wrote: > On Fri, Jan 2, 2015 at 9:49 PM, Tuncer Ayaz wrote: > > Does anybody know if the following travis-ci Erlang distro > > breakage is due to some kind of recent compartmentalization > > changes, and if so, what changes are required to continue using > > travis-ci for Erlang? > > > > Can we use only short names for nodes now? > > > > curl > > https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt > > | tail -n 45 > > The relevant bit: > {error_logger,{{2015,1,1},{13,48,39}}, > "Can't set long node name!\nPlease check your configuration\n",[]} > > Also, I should have tried -sname before asking the list: > https://github.com/rebar/rebar/pull/421 After switching -name to -sname, the test runs into: {error_logger,{{2015,1,2},{21,22,11}}, \"Protocol: ~tp: the name ct_rt3@REDACTED seems to be in use by another Erlang node\", [\"inet_tcp\"]} Clearly, it didn't fix the problem, and since the tests ran fine previously, I'll have to assume something's changed in travis-ci's Erlang support. They're out-of-office for the moment, so we'll hopefully find out what changed once they're back. From mark.nijhof@REDACTED Sun Jan 4 13:42:37 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sun, 4 Jan 2015 13:42:37 +0100 Subject: [erlang-questions] travis-ci erlang support broken? In-Reply-To: References: Message-ID: Did you try pinging them on Twitter? They always seems to be "on call" there? On Sun, Jan 4, 2015 at 12:38 PM, Tuncer Ayaz wrote: > On Fri, Jan 2, 2015 at 10:20 PM, Tuncer Ayaz wrote: > > On Fri, Jan 2, 2015 at 9:49 PM, Tuncer Ayaz wrote: > > > Does anybody know if the following travis-ci Erlang distro > > > breakage is due to some kind of recent compartmentalization > > > changes, and if so, what changes are required to continue using > > > travis-ci for Erlang? > > > > > > Can we use only short names for nodes now? > > > > > > curl > > > https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt > > > | tail -n 45 > > > > The relevant bit: > > {error_logger,{{2015,1,1},{13,48,39}}, > > "Can't set long node name!\nPlease check your configuration\n",[]} > > > > Also, I should have tried -sname before asking the list: > > https://github.com/rebar/rebar/pull/421 > > After switching -name to -sname, the test runs into: > {error_logger,{{2015,1,2},{21,22,11}}, > \"Protocol: ~tp: the name ct_rt3@REDACTED seems to be in use by > another Erlang node\", [\"inet_tcp\"]} > > Clearly, it didn't fix the problem, and since the tests ran fine > previously, I'll have to assume something's changed in travis-ci's > Erlang support. They're out-of-office for the moment, so we'll > hopefully find out what changed once they're back. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun Jan 4 14:27:46 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 4 Jan 2015 14:27:46 +0100 Subject: [erlang-questions] travis-ci erlang support broken? In-Reply-To: References: Message-ID: On Sun, Jan 4, 2015 at 1:42 PM, Mark Nijhof wrote: > Did you try pinging them on Twitter? They always seems to be "on > call" there? No Twitter here, and I respect their off-time, even though it's quite unfortunate that it broke right before their vacation start. > On Sun, Jan 4, 2015 at 12:38 PM, Tuncer Ayaz wrote: > > > > On Fri, Jan 2, 2015 at 10:20 PM, Tuncer Ayaz wrote: > > > On Fri, Jan 2, 2015 at 9:49 PM, Tuncer Ayaz wrote: > > > > Does anybody know if the following travis-ci Erlang distro > > > > breakage is due to some kind of recent compartmentalization > > > > changes, and if so, what changes are required to continue using > > > > travis-ci for Erlang? > > > > > > > > Can we use only short names for nodes now? > > > > > > > > curl > > > > https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt > > > > | tail -n 45 > > > > > > The relevant bit: > > > {error_logger,{{2015,1,1},{13,48,39}}, > > > "Can't set long node name!\nPlease check your configuration\n",[]} > > > > > > Also, I should have tried -sname before asking the list: > > > https://github.com/rebar/rebar/pull/421 > > > > After switching -name to -sname, the test runs into: > > {error_logger,{{2015,1,2},{21,22,11}}, > > \"Protocol: ~tp: the name ct_rt3@REDACTED seems to be in use by > > another Erlang node\", [\"inet_tcp\"]} > > > > Clearly, it didn't fix the problem, and since the tests ran fine > > previously, I'll have to assume something's changed in travis-ci's > > Erlang support. They're out-of-office for the moment, so we'll > > hopefully find out what changed once they're back. From p.wycisk@REDACTED Sun Jan 4 16:11:41 2015 From: p.wycisk@REDACTED (=?UTF-8?Q?Przemys=C5=82aw_Wycisk?=) Date: Sun, 4 Jan 2015 16:11:41 +0100 Subject: [erlang-questions] running rebar release in background Message-ID: Hi, i have an issue with running erlang application in background on Centos 6.6 (with ./bin/myapp start). The wild fact is that it starts and works fine when i try to run it with "./bin/myapp foreground". I tried also ./bin/myapp console and output was like this: {error_logger,{{2015,1,4},{16,2,59}},crash_report,[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.1208.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,jiffy},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,330}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[kernel_sup,<0.1184.0>]},{messages,[]},{links,[<0.1185.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,124}],[]]} {error_logger,{{2015,1,4},{16,2,59}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,jiffy}},{offender,[{pid,undefined},{name,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]} {error_logger,{{2015,1,4},{16,3,0}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.1183.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[<0.1182.0>]},{messages,[{'EXIT',<0.1184.0>,normal}]},{links,[<0.1182.0>,<0.1181.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]} {error_logger,{{2015,1,4},{16,3,0}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}) On mac the application is working fine with the same configuration i have no clue why is this happening, and i haven't found any question like that or solution in the internet, can anyone help? I'm also curious that is there any way to start an rebar release in foreground and then detach it from user/console, that solution could be a good workaround in this problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Sun Jan 4 19:01:34 2015 From: askjuise@REDACTED (Alexander Petrovsky) Date: Mon, 5 Jan 2015 03:01:34 +0900 Subject: [erlang-questions] running rebar release in background In-Reply-To: References: Message-ID: Hi! Looks like you have a problem with jiffy, take a look at glibc version on build server and on worker were you run your app. ???????????, 4 ?????? 2015 ?. ???????????? Przemys?aw Wycisk ???????: > Hi, i have an issue with running erlang application in background on > Centos 6.6 (with ./bin/myapp start). The wild fact is that it starts and > works fine when i try to run it with "./bin/myapp foreground". > I tried also ./bin/myapp console and output was like this: > > > {error_logger,{{2015,1,4},{16,2,59}},crash_report,[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.1208.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,jiffy},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,330}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[kernel_sup,<0.1184.0>]},{messages,[]},{links,[<0.1185.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,124}],[]]} > > > {error_logger,{{2015,1,4},{16,2,59}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,jiffy}},{offender,[{pid,undefined},{name,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]} > > > {error_logger,{{2015,1,4},{16,3,0}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.1183.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[<0.1182.0>]},{messages,[{'EXIT',<0.1184.0>,normal}]},{links,[<0.1182.0>,<0.1181.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]} > > > {error_logger,{{2015,1,4},{16,3,0}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}},{type,permanent}]} > > {"Kernel pid > terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}"} > > > Crash dump was written to: erl_crash.dump > > Kernel pid terminated (application_controller) > ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}) > > > On mac the application is working fine with the same configuration i have > no clue why is this happening, and i haven't found any question like that > or solution in the internet, can anyone help? > > I'm also curious that is there any way to start an rebar release in > foreground and then detach it from user/console, that solution could be a > good workaround in this problem. > -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lostcolony@REDACTED Mon Jan 5 01:41:31 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Sun, 4 Jan 2015 19:41:31 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine Message-ID: https://github.com/lostcolony/damocles I asked a while back on this mailing list if anyone had any useful libraries or similar for testing distribution scenarios. I only got back a few responses (maybe co-op riak_test? Maybe make use of the underlying Linux traffic control and network emulation apps?), and my own searches, while finding a few libraries, didn't find anything I could easily co-op for my purposes. To that end, I went ahead and spent part of my break on this, and it just got sufficiently feature complete to throw out there. I haven't had a chance to really start using it heavily, and I've only been testing it on my dev box, but a basic run through of the functionality as I typed up the readme worked (so any issues being pointed out would be appreciated). Essentially, it allows you to create and manipulate local interfaces on a Linux machine to emulate packet delay and loss (using the underlying traffic control and network emulation mechanisms), with a number of convenience methods to (hopefully) easily describe fairly intricate distribution scenarios. Things like "create these 5 interfaces, (now from my test code, launch a copy of my app on each one, or even a different app on one of them, to see what happens when that resource is flaky); now make it so 1 and 2 can't talk to 3 and 4, and vice versa, but everyone can still talk to 5, but replies have a 50% chance of being dropped from 5 when responding to 1 and 2, and there's a 300ms delay between 3 and 4; (now, let's run more of our test code to assert that trying to write to any node still succeeds); okay, now let's restore the network back to normal (and have our test code make sure the write was retained)", or whatever, can be set up in a straightforward, automated manner as part of a common test run, and not be reliant on certain VMs being up, nor the tests being run on a specific network. The tradeoff, obviously, being that you can't really load test things with it. Still, it fits my basic needs, and I figured it might be of use to others. I'll be adding some simple examples when I next get free time (I ran out of it from the holiday break without getting to them; dunno when I will), and will try and get to any bugs or simple suggestions in a timely manner, but hopefully it's fairly straightforward and useful as is. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Mon Jan 5 09:51:11 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 05 Jan 2015 10:51:11 +0200 Subject: [erlang-questions] Order of evaluation of guards Message-ID: <54AA507F.5040509@gmail.com> Hello, First of all - Happy New Year! Suppose we have a function that accepts 2 integers and we want to react only when both integers are greater than 0 and one of them is less than 100: I tried: (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 -> cool; (_, _) -> not_cool end. #Fun (dilbert@REDACTED)2> F(1,2). cool (dilbert@REDACTED)3> F(1,200). cool (dilbert@REDACTED)4> F(0,200). not_cool (dilbert@REDACTED)5> F(0,50). cool The last is not correct. What is the order of evaluation of the guards? Sadly parentheses are not allowed in guards. Best regards, Martin From sergej.jurecko@REDACTED Mon Jan 5 09:53:56 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 5 Jan 2015 09:53:56 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <54AA507F.5040509@gmail.com> References: <54AA507F.5040509@gmail.com> Message-ID: You can use paranthesis with andalso and orelse Sergej On Jan 5, 2015 9:51 AM, "Martin Koroudjiev" wrote: > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to react > only when both integers are greater than 0 and one of them is less than > 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 > -> cool; (_, _) -> not_cool end. > #Fun > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. > What is the order of evaluation of the guards? Sadly parentheses are not > allowed in guards. > > Best regards, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Jan 5 09:57:49 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 5 Jan 2015 08:57:49 +0000 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <54AA507F.5040509@gmail.com> References: <54AA507F.5040509@gmail.com> Message-ID: The last is correct according to your guards: M < 100. Perhaps you could try: ... when N > 0, M > 0, N < 100; N > 0, M > 0, M < 100 -> ... Happy New Year! Ivan -- festina lente > On 5 Jan 2015, at 08:51, Martin Koroudjiev wrote: > > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to react > only when both integers are greater than 0 and one of them is less than 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 > -> cool; (_, _) -> not_cool end. > #Fun > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. > What is the order of evaluation of the guards? Sadly parentheses are not > allowed in guards. > > Best regards, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Mon Jan 5 09:58:48 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 05 Jan 2015 10:58:48 +0200 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: <54AA5248.6010308@gmail.com> Cool. Thanks. This works as expected: G = fun(N, M) when (N > 0 andalso M > 0) andalso (N < 100 orelse M < 100) -> cool; (_, _) -> not_cool end. On 1/5/2015 10:53 AM, Sergej Jure?ko wrote: > > You can use paranthesis with andalso and orelse > > Sergej > > On Jan 5, 2015 9:51 AM, "Martin Koroudjiev" > wrote: > > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to > react > only when both integers are greater than 0 and one of them is less > than 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M > < 100 > -> cool; (_, _) -> not_cool end. > #Fun> > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. > What is the order of evaluation of the guards? Sadly parentheses > are not > allowed in guards. > > Best regards, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Mon Jan 5 10:04:07 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 05 Jan 2015 11:04:07 +0200 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: <54AA5387.7090009@gmail.com> Hi, thanks. So the "and" (,) is evaluated first and since (,) is not short circuit all will be replaced with true/false. And then the "or" will be evaluated. On 1/5/2015 10:57 AM, Ivan Uemlianin wrote: > The last is correct according to your guards: M < 100. Perhaps you could try: > > ... when N > 0, M > 0, N < 100; > N > 0, M > 0, M < 100 -> ... > > Happy New Year! > > Ivan > > -- > festina lente > > >> On 5 Jan 2015, at 08:51, Martin Koroudjiev wrote: >> >> Hello, >> >> First of all - Happy New Year! >> >> Suppose we have a function that accepts 2 integers and we want to react >> only when both integers are greater than 0 and one of them is less than 100: >> >> I tried: >> (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 >> -> cool; (_, _) -> not_cool end. >> #Fun >> (dilbert@REDACTED)2> F(1,2). >> cool >> (dilbert@REDACTED)3> F(1,200). >> cool >> (dilbert@REDACTED)4> F(0,200). >> not_cool >> (dilbert@REDACTED)5> F(0,50). >> cool >> >> The last is not correct. >> What is the order of evaluation of the guards? Sadly parentheses are not >> allowed in guards. >> >> Best regards, >> Martin >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From z@REDACTED Mon Jan 5 10:04:52 2015 From: z@REDACTED (Danil Zagoskin) Date: Mon, 5 Jan 2015 13:04:52 +0400 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <54AA507F.5040509@gmail.com> References: <54AA507F.5040509@gmail.com> Message-ID: Alternatively, you may split complex logic to several clauses: f(N, M) when N > 100, M > 100 -> % Both too large not_cool; f(N, M) when N < 100, M < 100 -> % Both too small not_cool; f(N, M) when N > 0, M > 0 -> % Both are greater than 0, after previous clauses exactly one of them greater than 100 cool; f(_, _) -> not_cool. On Mon, Jan 5, 2015 at 11:51 AM, Martin Koroudjiev wrote: > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to react > only when both integers are greater than 0 and one of them is less than > 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 > -> cool; (_, _) -> not_cool end. > #Fun > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. > What is the order of evaluation of the guards? Sadly parentheses are not > allowed in guards. > > Best regards, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Mon Jan 5 10:08:01 2015 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 5 Jan 2015 09:08:01 +0000 Subject: [erlang-questions] Why is erlc so quick to start? Message-ID: Lo?c has mentioned that any Erlang package manager must be as quick to start as 'erlc'. So, I figured that I'd take a quick look at how 'erlc' is started. The code's in erts/etc/common/erlc.c, and it appears to do this: erlc +sbtu +A0 -noinput -mode minimal -boot start_clean -s erl_compile compile_cmdline For comparison, "erl -noinput -s init stop" takes, according to /usr/bin/time, on average, 1.15 secs elapsed to exit, versus "erlc +sbtu +A0 -noinput -mode minimal -boot start_clean -s init stop", which takes 1.10 secs. This is with a Core i7-3930K @ 3.20GHz, running from an SSD. That's not much of a saving, but I guess it could make enough difference. For another data point, I used "-s erlang halt" instead of "-s init stop", and saw much bigger savings. The average time here was 0.10 secs. From a brief look, I don't know which of these is more relevant. It's difficult to get a direct comparison with running the "erlc" binary, because it doesn't spawn the VM if there's nothing to compile, and if there is something to compile, it compiles it. So I wouldn't be comparing like with like. From mrtndimitrov@REDACTED Mon Jan 5 10:08:27 2015 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 05 Jan 2015 11:08:27 +0200 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: <54AA548B.8090206@gmail.com> Yes, this looks much better. I just wanted to figure out how guards are evaluated. On 1/5/2015 11:04 AM, Danil Zagoskin wrote: > Alternatively, you may split complex logic to several clauses: > f(N, M) when N > 100, M > 100 -> % Both too large > not_cool; > f(N, M) when N < 100, M < 100 -> % Both too small > not_cool; > f(N, M) when N > 0, M > 0 -> % Both are greater than 0, after previous > clauses exactly one of them greater than 100 > cool; > f(_, _) -> > not_cool. > > On Mon, Jan 5, 2015 at 11:51 AM, Martin Koroudjiev > > wrote: > > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to > react > only when both integers are greater than 0 and one of them is less > than 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M > < 100 > -> cool; (_, _) -> not_cool end. > #Fun > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. > What is the order of evaluation of the guards? Sadly parentheses > are not > allowed in guards. > > Best regards, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon Jan 5 10:14:12 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 5 Jan 2015 10:14:12 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: Hi! Maybe you can compare "erlc myfile" with "erl -s compile file myfile"? Or start some other code that does something relevant, with and without the magic arguments. I believe that a big part of the savings is due to the differences in code loading, which "init:stop" or "erlang:halt" might not trigger. regards, /Vlad On Mon, Jan 5, 2015 at 10:08 AM, Roger Lipscombe wrote: > Lo?c has mentioned that any Erlang package manager must be as quick to > start as 'erlc'. So, I figured that I'd take a quick look at how > 'erlc' is started. > > The code's in erts/etc/common/erlc.c, and it appears to do this: > > erlc +sbtu +A0 -noinput -mode minimal -boot start_clean -s > erl_compile compile_cmdline > > For comparison, "erl -noinput -s init stop" takes, according to > /usr/bin/time, on average, 1.15 secs elapsed to exit, versus "erlc > +sbtu +A0 -noinput -mode minimal -boot start_clean -s init stop", > which takes 1.10 secs. This is with a Core i7-3930K @ 3.20GHz, running > from an SSD. > > That's not much of a saving, but I guess it could make enough difference. > > For another data point, I used "-s erlang halt" instead of "-s init > stop", and saw much bigger savings. The average time here was 0.10 > secs. From a brief look, I don't know which of these is more relevant. > > It's difficult to get a direct comparison with running the "erlc" > binary, because it doesn't spawn the VM if there's nothing to compile, > and if there is something to compile, it compiles it. So I wouldn't be > comparing like with like. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Jan 5 10:18:30 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 5 Jan 2015 09:18:30 +0000 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <54AA5387.7090009@gmail.com> References: <54AA507F.5040509@gmail.com> <54AA5387.7090009@gmail.com> Message-ID: <2D253B91-D602-4557-9C34-53C84633D6A9@llaisdy.com> The main thing is ; is applied first, so your guard was like: (( N > 0, M > 0, N < 100) ; (M < 100)) Then the whole single expression is evaluated. Ivan -- festina lente > On 5 Jan 2015, at 09:04, Martin Koroudjiev wrote: > > Hi, thanks. So the "and" (,) is evaluated first and since (,) is not > short circuit all will be replaced with true/false. And then the "or" > will be evaluated. > > >> On 1/5/2015 10:57 AM, Ivan Uemlianin wrote: >> The last is correct according to your guards: M < 100. Perhaps you could try: >> >> ... when N > 0, M > 0, N < 100; >> N > 0, M > 0, M < 100 -> ... >> >> Happy New Year! >> >> Ivan >> >> -- >> festina lente >> >> >>> On 5 Jan 2015, at 08:51, Martin Koroudjiev wrote: >>> >>> Hello, >>> >>> First of all - Happy New Year! >>> >>> Suppose we have a function that accepts 2 integers and we want to react >>> only when both integers are greater than 0 and one of them is less than 100: >>> >>> I tried: >>> (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 >>> -> cool; (_, _) -> not_cool end. >>> #Fun >>> (dilbert@REDACTED)2> F(1,2). >>> cool >>> (dilbert@REDACTED)3> F(1,200). >>> cool >>> (dilbert@REDACTED)4> F(0,200). >>> not_cool >>> (dilbert@REDACTED)5> F(0,50). >>> cool >>> >>> The last is not correct. >>> What is the order of evaluation of the guards? Sadly parentheses are not >>> allowed in guards. >>> >>> Best regards, >>> Martin >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Mon Jan 5 10:56:07 2015 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 5 Jan 2015 10:56:07 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: Making an Erlang application start quickly is a engineering problem. It's the "then make it fast" bit of "first make it work then make it fast" It's relatively easy to make a specific Erlang application start quickly. In what follows I've outlined three techniques to do this: The first step is to identify exactly which modules are actually called. Try this: -module(dummy). -compile(export_all). start() -> erlang:display(length(code:all_loaded())), init:stop(). Compile this and run with > erl -s dummy start It prints out 75 So to do *anything* the system loads 75 modules. To make things faster you can 1) Hand tune your application to use less than 75 modules In my book (shameless plug) Appendix C - I've cut this down to four modules - BUT It's NOT the OTP system. So the standard libraries don't work - the book version loaded a simple program in 0.02 seconds (vs 1.13 seconds on the same machine for erl) 2) Take all the beam files for the 75 modules you need, remove debugging symbols and put them in *one* file - and load from this. I'm not saying you *should* do this - but you *could* do it if necessary. The downside is you can't use the standard libraries - the upside is a deeper understanding of the system. The "default assumption" when we built Erlang was that the system once started never stops - so trimming the odd second of the startup time was never considered. Amortise the startup time over a few years and the odd second doesn't matter. A third method of ensuring fast startups is to keep a resident Erlang node always up and running and using to_erl to connect to it - or keeping a resident erlang node up and using telnet (or some other protocol) to talk to it - then you'll be running in a few ms. Happy New Year /Joe On Mon, Jan 5, 2015 at 10:08 AM, Roger Lipscombe wrote: > Lo?c has mentioned that any Erlang package manager must be as quick to > start as 'erlc'. So, I figured that I'd take a quick look at how > 'erlc' is started. > > The code's in erts/etc/common/erlc.c, and it appears to do this: > > erlc +sbtu +A0 -noinput -mode minimal -boot start_clean -s > erl_compile compile_cmdline > > For comparison, "erl -noinput -s init stop" takes, according to > /usr/bin/time, on average, 1.15 secs elapsed to exit, versus "erlc > +sbtu +A0 -noinput -mode minimal -boot start_clean -s init stop", > which takes 1.10 secs. This is with a Core i7-3930K @ 3.20GHz, running > from an SSD. > > That's not much of a saving, but I guess it could make enough difference. > > For another data point, I used "-s erlang halt" instead of "-s init > stop", and saw much bigger savings. The average time here was 0.10 > secs. From a brief look, I don't know which of these is more relevant. > > It's difficult to get a direct comparison with running the "erlc" > binary, because it doesn't spawn the VM if there's nothing to compile, > and if there is something to compile, it compiles it. So I wouldn't be > comparing like with like. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roger@REDACTED Mon Jan 5 11:05:11 2015 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 5 Jan 2015 10:05:11 +0000 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: On 5 January 2015 at 09:14, Vlad Dumitrescu wrote: > Hi! > > Maybe you can compare "erlc myfile" with "erl -s compile file myfile"? Or > start some other code that does something relevant, with and without the > magic arguments. Given foo.erl: -module(foo). -export([compile/0]). compile() -> compile:file("foo.erl"), erlang:halt(). "erlc foo.erl" takes 0.18 secs. "erl -noinput -s foo compile" takes 0.23 secs. "erl +sbtu +A0 -noinput -mode minimal -boot start_clean -s foo compile" takes 0.17 secs. "+sbtu" doesn't seem to make any difference; "+A0" seems to responsible for ~0.04 secs; "-mode minimal" for ~0.03 secs; "-boot start_clean", again, seems to make no difference. With an escript: #!/usr/bin/env escript %%! +sbtu +A0 -mode minimal -boot start_clean main([]) -> compile:file("foo.erl"). It takes ~0.17 seconds; without the %%! line, it takes ~0.23 seconds. These are comparable to the above results. These results aren't exactly scientifically rigorous. PC spec is as given in my first email; Erlang is R16B03-1; OS is Linux Mint 17; methodology was simply to run "/usr/bin/time whatever" at least 5 times and report the number that looked most like the mode. From essen@REDACTED Mon Jan 5 11:33:21 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 05 Jan 2015 11:33:21 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: <54AA6871.1020603@ninenines.eu> On 01/05/2015 11:05 AM, Roger Lipscombe wrote: > On 5 January 2015 at 09:14, Vlad Dumitrescu wrote: >> Hi! >> >> Maybe you can compare "erlc myfile" with "erl -s compile file myfile"? Or >> start some other code that does something relevant, with and without the >> magic arguments. > > Given foo.erl: > > -module(foo). > -export([compile/0]). > > compile() -> > compile:file("foo.erl"), > erlang:halt(). > > "erlc foo.erl" takes 0.18 secs. > "erl -noinput -s foo compile" takes 0.23 secs. > "erl +sbtu +A0 -noinput -mode minimal -boot start_clean -s foo > compile" takes 0.17 secs. > > "+sbtu" doesn't seem to make any difference; "+A0" seems to > responsible for ~0.04 secs; "-mode minimal" for ~0.03 secs; "-boot > start_clean", again, seems to make no difference. We discussed it a bit on IRC after my post here and came to the same conclusions. The biggest gain from "erl" to "erlc" comes from -noinput. After that, the small gains you mention are what do the rest. We then went on to guess that what makes some escripts slow would be either what "escript" is doing (the compile/interpreting/dealing with packaged escripts; this won't show up in a 2 lines escript :-) or what the escript itself is doing. None of us being really interested in escripts the discussion ended there. "erlc" could most likely be much faster because it currently uses the start_clean boot file. It would be interesting to experiment with a minimal boot file. I like the current findings though, I can use them in other areas (edoc, triq, erlydtl, ...). -- Lo?c Hoguin http://ninenines.eu From essen@REDACTED Mon Jan 5 11:44:19 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 05 Jan 2015 11:44:19 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: <54AA6B03.2020804@ninenines.eu> On 01/05/2015 10:56 AM, Joe Armstrong wrote: > To make things faster you can > > 1) Hand tune your application to use less than 75 modules > In my book (shameless plug) Appendix C - I've cut this down to four > modules - BUT It's NOT the OTP system. So the standard libraries > don't work - the book version loaded a simple program in 0.02 seconds > (vs 1.13 seconds on the same machine for erl) Yes that's what I want to experiment with for erlc. I do not think it needs most of the OTP system to compile a file. I hope it yields interesting results. > The "default assumption" when we built Erlang was that the system once > started never stops - so trimming the odd second of the startup time > was never considered. Amortise the startup time over a few years and > the odd second doesn't matter. That's perfectly understandable for a long running system. Not so much for a compiler. It's actually quite funny how Erlang compiles many projects faster than it boots. :-) -- Lo?c Hoguin http://ninenines.eu From essen@REDACTED Mon Jan 5 11:50:12 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 05 Jan 2015 11:50:12 +0100 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: Message-ID: <54AA6C64.2060006@ninenines.eu> This looks really good! On 01/05/2015 01:41 AM, Christopher Phillips wrote: > https://github.com/lostcolony/damocles > > I asked a while back on this mailing list if anyone had any useful > libraries or similar for testing distribution scenarios. I only got back > a few responses (maybe co-op riak_test? Maybe make use of the underlying > Linux traffic control and network emulation apps?), and my own searches, > while finding a few libraries, didn't find anything I could easily co-op > for my purposes. > > To that end, I went ahead and spent part of my break on this, and it > just got sufficiently feature complete to throw out there. I haven't had > a chance to really start using it heavily, and I've only been testing it > on my dev box, but a basic run through of the functionality as I typed > up the readme worked (so any issues being pointed out would be > appreciated). Essentially, it allows you to create and manipulate local > interfaces on a Linux machine to emulate packet delay and loss (using > the underlying traffic control and network emulation mechanisms), with a > number of convenience methods to (hopefully) easily describe fairly > intricate distribution scenarios. > > Things like "create these 5 interfaces, (now from my test code, launch a > copy of my app on each one, or even a different app on one of them, to > see what happens when that resource is flaky); now make it so 1 and 2 > can't talk to 3 and 4, and vice versa, but everyone can still talk to 5, > but replies have a 50% chance of being dropped from 5 when responding to > 1 and 2, and there's a 300ms delay between 3 and 4; (now, let's run more > of our test code to assert that trying to write to any node still > succeeds); okay, now let's restore the network back to normal (and have > our test code make sure the write was retained)", or whatever, can be > set up in a straightforward, automated manner as part of a common test > run, and not be reliant on certain VMs being up, nor the tests being run > on a specific network. The tradeoff, obviously, being that you can't > really load test things with it. Still, it fits my basic needs, and I > figured it might be of use to others. > > I'll be adding some simple examples when I next get free time (I ran out > of it from the holiday break without getting to them; dunno when I > will), and will try and get to any bugs or simple suggestions in a > timely manner, but hopefully it's fairly straightforward and useful as is. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From jesper.louis.andersen@REDACTED Mon Jan 5 12:36:02 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 05 Jan 2015 11:36:02 +0000 Subject: [erlang-questions] Order of evaluation of guards References: <54AA507F.5040509@gmail.com> <54AA5387.7090009@gmail.com> <2D253B91-D602-4557-9C34-53C84633D6A9@llaisdy.com> Message-ID: Hi, Guard order don't matter! You are limited to a small set of expressions and BIFs in guards. All of these are guaranteed to be total in the sense that either producing a result or terminating. By not defining a rule for guard order, the compiler is free to reorder guard expressions as it sees fit. It can also cache results of guard expressions, since they won't change during the pattern match evaluation. On Mon Jan 05 2015 at 10:18:37 AM Ivan Uemlianin wrote: > The main thing is ; is applied first, so your guard was like: > > (( N > 0, M > 0, N < 100) ; (M < 100)) > > Then the whole single expression is evaluated. > > Ivan > > -- > festina lente > > > > On 5 Jan 2015, at 09:04, Martin Koroudjiev > wrote: > > > > Hi, thanks. So the "and" (,) is evaluated first and since (,) is not > > short circuit all will be replaced with true/false. And then the "or" > > will be evaluated. > > > > > >> On 1/5/2015 10:57 AM, Ivan Uemlianin wrote: > >> The last is correct according to your guards: M < 100. Perhaps you > could try: > >> > >> ... when N > 0, M > 0, N < 100; > >> N > 0, M > 0, M < 100 -> ... > >> > >> Happy New Year! > >> > >> Ivan > >> > >> -- > >> festina lente > >> > >> > >>> On 5 Jan 2015, at 08:51, Martin Koroudjiev > wrote: > >>> > >>> Hello, > >>> > >>> First of all - Happy New Year! > >>> > >>> Suppose we have a function that accepts 2 integers and we want to react > >>> only when both integers are greater than 0 and one of them is less > than 100: > >>> > >>> I tried: > >>> (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < > 100 > >>> -> cool; (_, _) -> not_cool end. > >>> #Fun > >>> (dilbert@REDACTED)2> F(1,2). > >>> cool > >>> (dilbert@REDACTED)3> F(1,200). > >>> cool > >>> (dilbert@REDACTED)4> F(0,200). > >>> not_cool > >>> (dilbert@REDACTED)5> F(0,50). > >>> cool > >>> > >>> The last is not correct. > >>> What is the order of evaluation of the guards? Sadly parentheses are > not > >>> allowed in guards. > >>> > >>> Best regards, > >>> Martin > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jan 5 12:52:57 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 5 Jan 2015 12:52:57 +0100 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: Message-ID: This looks like a great tool and something that could easily be added to unit tests. Anyone with ipfw skills to add bsd/osx support? Sergej On Jan 5, 2015 1:41 AM, "Christopher Phillips" wrote: > https://github.com/lostcolony/damocles > > I asked a while back on this mailing list if anyone had any useful > libraries or similar for testing distribution scenarios. I only got back a > few responses (maybe co-op riak_test? Maybe make use of the underlying > Linux traffic control and network emulation apps?), and my own searches, > while finding a few libraries, didn't find anything I could easily co-op > for my purposes. > > To that end, I went ahead and spent part of my break on this, and it just > got sufficiently feature complete to throw out there. I haven't had a > chance to really start using it heavily, and I've only been testing it on > my dev box, but a basic run through of the functionality as I typed up the > readme worked (so any issues being pointed out would be appreciated). > Essentially, it allows you to create and manipulate local interfaces on a > Linux machine to emulate packet delay and loss (using the underlying > traffic control and network emulation mechanisms), with a number of > convenience methods to (hopefully) easily describe fairly intricate > distribution scenarios. > > Things like "create these 5 interfaces, (now from my test code, launch a > copy of my app on each one, or even a different app on one of them, to see > what happens when that resource is flaky); now make it so 1 and 2 can't > talk to 3 and 4, and vice versa, but everyone can still talk to 5, but > replies have a 50% chance of being dropped from 5 when responding to 1 and > 2, and there's a 300ms delay between 3 and 4; (now, let's run more of our > test code to assert that trying to write to any node still succeeds); okay, > now let's restore the network back to normal (and have our test code make > sure the write was retained)", or whatever, can be set up in a > straightforward, automated manner as part of a common test run, and not be > reliant on certain VMs being up, nor the tests being run on a specific > network. The tradeoff, obviously, being that you can't really load test > things with it. Still, it fits my basic needs, and I figured it might be of > use to others. > > I'll be adding some simple examples when I next get free time (I ran out > of it from the holiday break without getting to them; dunno when I will), > and will try and get to any bugs or simple suggestions in a timely manner, > but hopefully it's fairly straightforward and useful as is. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Mon Jan 5 14:09:46 2015 From: zkessin@REDACTED (Zachary Kessin) Date: Mon, 05 Jan 2015 15:09:46 +0200 Subject: [erlang-questions] Mostly Erlang on Packages In-Reply-To: <54A45B0C.1080408@gmail.com> References: <54A45B0C.1080408@gmail.com> Message-ID: <54AA8D1A.7060105@gmail.com> Here is the link for the live podcast tomorrow https://plus.google.com/events/cj31fp9dm6hnr1m2diu8qaeen88 It will be at 18:00 gmt, and will be released afterwards as the normal audio --Zach On 12/31/2014 10:22 PM, Zachary Kessin wrote: > Hi Everyone > > With all the conversation here on Packages and how we should manage > them I thought that it might make sense to have a podcast on the > topic. However for this I wanted to do something a bit different. > Instead of recording it with the Panel and then releasing after > editing. I had thought that we will try something new and do it live. > We will do a google hangout on Air at 18:00GMT on January 6 (Convert > that into your local timezone) and anyone who wants to can listen in > and ask questions via the chat. > > --Zach Kessin > Host Mostly Erlang Podcast > @zkessin From lostcolony@REDACTED Mon Jan 5 15:16:42 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Mon, 5 Jan 2015 09:16:42 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: Message-ID: For OSX ipfw was deprecated in Lion and removed in Yosemite. I've done a bit of looking at the replacement, pf, and it looks like dropping packets based on percentage is doable, as is bandwidth throttling (something I'd like to add, in general), but I don't see any way to induce a delay, beyond an implicit one based on tos prioritization. If someone knows how and can point me in the right direction I'd appreciate it. On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko wrote: > This looks like a great tool and something that could easily be added to > unit tests. > Anyone with ipfw skills to add bsd/osx support? > > Sergej > On Jan 5, 2015 1:41 AM, "Christopher Phillips" > wrote: > >> https://github.com/lostcolony/damocles >> >> I asked a while back on this mailing list if anyone had any useful >> libraries or similar for testing distribution scenarios. I only got back a >> few responses (maybe co-op riak_test? Maybe make use of the underlying >> Linux traffic control and network emulation apps?), and my own searches, >> while finding a few libraries, didn't find anything I could easily co-op >> for my purposes. >> >> To that end, I went ahead and spent part of my break on this, and it just >> got sufficiently feature complete to throw out there. I haven't had a >> chance to really start using it heavily, and I've only been testing it on >> my dev box, but a basic run through of the functionality as I typed up the >> readme worked (so any issues being pointed out would be appreciated). >> Essentially, it allows you to create and manipulate local interfaces on a >> Linux machine to emulate packet delay and loss (using the underlying >> traffic control and network emulation mechanisms), with a number of >> convenience methods to (hopefully) easily describe fairly intricate >> distribution scenarios. >> >> Things like "create these 5 interfaces, (now from my test code, launch a >> copy of my app on each one, or even a different app on one of them, to see >> what happens when that resource is flaky); now make it so 1 and 2 can't >> talk to 3 and 4, and vice versa, but everyone can still talk to 5, but >> replies have a 50% chance of being dropped from 5 when responding to 1 and >> 2, and there's a 300ms delay between 3 and 4; (now, let's run more of our >> test code to assert that trying to write to any node still succeeds); okay, >> now let's restore the network back to normal (and have our test code make >> sure the write was retained)", or whatever, can be set up in a >> straightforward, automated manner as part of a common test run, and not be >> reliant on certain VMs being up, nor the tests being run on a specific >> network. The tradeoff, obviously, being that you can't really load test >> things with it. Still, it fits my basic needs, and I figured it might be of >> use to others. >> >> I'll be adding some simple examples when I next get free time (I ran out >> of it from the holiday break without getting to them; dunno when I will), >> and will try and get to any bugs or simple suggestions in a timely manner, >> but hopefully it's fairly straightforward and useful as is. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jan 5 15:39:05 2015 From: rvirding@REDACTED (Robert Virding) Date: Mon, 5 Jan 2015 15:39:05 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> <54AA5387.7090009@gmail.com> <2D253B91-D602-4557-9C34-53C84633D6A9@llaisdy.com> Message-ID: That the ',' is not short circuit is an implementation detail, and the wrong choice at that. Originally guards were evaluated from left-to-right and as soon as a test failed the guard failed, the ',' was short circuiting. This was also consistent with ';' alternate guards which was also left-to-right and short-circuiting, as soon as one of the guard sequences succeeded the whole guard succeeded. I don't know when ',' became strict but it was never intended to be that. It should behave the same as 'andalso' and not as 'and'. Note that there is one very significant difference between ';' and or/orelse and that is how they behave with errors. ';' will just fail that one guard sequence and attempt the next one while or/orelse will fail the guard sequence they are in. So if they are used as an alternative to ';' you will get different behaviour. That errors in guards cause the guard to fail and not generate an exception was an intentional design decision, it saved a lot of explicit type tests and they were implicit in the operations. For example you could do tuple_size(T) without first having to check if T is a tuple. Robert On 5 January 2015 at 12:36, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi, > > Guard order don't matter! You are limited to a small set of expressions > and BIFs in guards. All of these are guaranteed to be total in the sense > that either producing a result or terminating. By not defining a rule for > guard order, the compiler is free to reorder guard expressions as it sees > fit. It can also cache results of guard expressions, since they won't > change during the pattern match evaluation. > > > On Mon Jan 05 2015 at 10:18:37 AM Ivan Uemlianin wrote: > >> The main thing is ; is applied first, so your guard was like: >> >> (( N > 0, M > 0, N < 100) ; (M < 100)) >> >> Then the whole single expression is evaluated. >> >> Ivan >> >> -- >> festina lente >> >> >> > On 5 Jan 2015, at 09:04, Martin Koroudjiev >> wrote: >> > >> > Hi, thanks. So the "and" (,) is evaluated first and since (,) is not >> > short circuit all will be replaced with true/false. And then the "or" >> > will be evaluated. >> > >> > >> >> On 1/5/2015 10:57 AM, Ivan Uemlianin wrote: >> >> The last is correct according to your guards: M < 100. Perhaps you >> could try: >> >> >> >> ... when N > 0, M > 0, N < 100; >> >> N > 0, M > 0, M < 100 -> ... >> >> >> >> Happy New Year! >> >> >> >> Ivan >> >> >> >> -- >> >> festina lente >> >> >> >> >> >>> On 5 Jan 2015, at 08:51, Martin Koroudjiev >> wrote: >> >>> >> >>> Hello, >> >>> >> >>> First of all - Happy New Year! >> >>> >> >>> Suppose we have a function that accepts 2 integers and we want to >> react >> >>> only when both integers are greater than 0 and one of them is less >> than 100: >> >>> >> >>> I tried: >> >>> (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < >> 100 >> >>> -> cool; (_, _) -> not_cool end. >> >>> #Fun >> >>> (dilbert@REDACTED)2> F(1,2). >> >>> cool >> >>> (dilbert@REDACTED)3> F(1,200). >> >>> cool >> >>> (dilbert@REDACTED)4> F(0,200). >> >>> not_cool >> >>> (dilbert@REDACTED)5> F(0,50). >> >>> cool >> >>> >> >>> The last is not correct. >> >>> What is the order of evaluation of the guards? Sadly parentheses are >> not >> >>> allowed in guards. >> >>> >> >>> Best regards, >> >>> Martin >> >>> >> >>> _______________________________________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED >> >>> http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Mon Jan 5 16:00:08 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 5 Jan 2015 16:00:08 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> <54AA5387.7090009@gmail.com> <2D253B91-D602-4557-9C34-53C84633D6A9@llaisdy.com> Message-ID: Le 5 janv. 2015 ? 15:39, Robert Virding a ?crit : > That the ',' is not short circuit is an implementation detail, and the wrong choice at that. Originally guards were evaluated from left-to-right and as soon as a test failed the guard failed, the ',' was short circuiting. This was also consistent with ';' alternate guards which was also left-to-right and short-circuiting, as soon as one of the guard sequences succeeded the whole guard succeeded. > > I don't know when ',' became strict but it was never intended to be that. It should behave the same as 'andalso' and not as 'and'. ',', 'and' and 'andalso' all compile to short-circuiting code, but the 'andalso' code is more convoluted than it should: https://gist.github.com/nox/655450660d148095071c From t@REDACTED Mon Jan 5 16:08:39 2015 From: t@REDACTED (Tristan Sloughter) Date: Mon, 05 Jan 2015 09:08:39 -0600 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <54AA6B03.2020804@ninenines.eu> References: <54AA6B03.2020804@ninenines.eu> Message-ID: <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> I added all those arguments to rebar3's escript and seems to be a consistent speedup as expected, dropping from 0.6s to 0.3s for a run that doesn't do anything but print the help. Can't say that my eyes see much of a difference though ;). But no reason not to keep it in. From p.wycisk@REDACTED Mon Jan 5 11:55:22 2015 From: p.wycisk@REDACTED (=?UTF-8?Q?Przemys=C5=82aw_Wycisk?=) Date: Mon, 5 Jan 2015 11:55:22 +0100 Subject: [erlang-questions] running rebar release in background In-Reply-To: References: Message-ID: thanks a lot !! I compiled jiffy on the same machine and it works ;) On Sun, Jan 4, 2015 at 7:01 PM, Alexander Petrovsky wrote: > Hi! > > Looks like you have a problem with jiffy, take a look at glibc version on > build server and on worker were you run your app. > > ???????????, 4 ?????? 2015 ?. ???????????? Przemys?aw Wycisk ???????: > > Hi, i have an issue with running erlang application in background on >> Centos 6.6 (with ./bin/myapp start). The wild fact is that it starts and >> works fine when i try to run it with "./bin/myapp foreground". >> I tried also ./bin/myapp console and output was like this: >> >> >> {error_logger,{{2015,1,4},{16,2,59}},crash_report,[[{initial_call,{supervisor,kernel,['Argument__1']}},{pid,<0.1208.0>},{registered_name,[]},{error_info,{exit,{on_load_function_failed,jiffy},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,330}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[kernel_sup,<0.1184.0>]},{messages,[]},{links,[<0.1185.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,124}],[]]} >> >> >> {error_logger,{{2015,1,4},{16,2,59}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{on_load_function_failed,jiffy}},{offender,[{pid,undefined},{name,kernel_safe_sup},{mfargs,{supervisor,start_link,[{local,kernel_safe_sup},kernel,safe]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]} >> >> >> {error_logger,{{2015,1,4},{16,3,0}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.1183.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}},{ancestors,[<0.1182.0>]},{messages,[{'EXIT',<0.1184.0>,normal}]},{links,[<0.1182.0>,<0.1181.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]} >> >> >> {error_logger,{{2015,1,4},{16,3,0}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}},{type,permanent}]} >> >> {"Kernel pid >> terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}"} >> >> >> Crash dump was written to: erl_crash.dump >> >> Kernel pid terminated (application_controller) >> ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,jiffy}}},{kernel,start,[normal,[]]}}}) >> >> >> On mac the application is working fine with the same configuration i have >> no clue why is this happening, and i haven't found any question like that >> or solution in the internet, can anyone help? >> >> I'm also curious that is there any way to start an rebar release in >> foreground and then detach it from user/console, that solution could be a >> good workaround in this problem. >> > > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Mon Jan 5 18:00:01 2015 From: sean@REDACTED (Functional Jobs) Date: Mon, 5 Jan 2015 12:00:01 -0500 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <54aac3137093c@functionaljobs.com> Here are some functional programming job opportunities that were posted recently: Senior Software Engineer at McGraw-Hill Education http://functionaljobs.com/jobs/8775-senior-software-engineer-at-mcgraw-hill-education Cheers, Sean Murphy FunctionalJobs.com From wallentin.dahlberg@REDACTED Mon Jan 5 18:12:54 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Mon, 5 Jan 2015 18:12:54 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> References: <54AA6B03.2020804@ninenines.eu> <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> Message-ID: As I mentioned in a previous thread, the start time of escript and erl is to an high degree influenced by the all the modules that needs to be loaded. This is also why +A0 has an impact. I have to caution you to use +A0 though. Short-circuiting file operation by using no async-threads may be hazardous to your system. In most cases it is just fine to use it but this is very unpredictable. The schedulers can become very unhappy. We can change the default arguments to escript to mitigate this, for instance: https://github.com/psyeugenic/otp/compare/erlang:master...egil/default-escript-arguments As for something more rigorous: Dizzy had a proposal of creating escripts with a beam-binary with erlang-archives appended to it. I don't know how far he got but with this. Something like that might go a long way to speed things up. // Bj?rn-Egil 2015-01-05 16:08 GMT+01:00 Tristan Sloughter : > I added all those arguments to rebar3's escript and seems to be a > consistent speedup as expected, dropping from 0.6s to 0.3s for a run > that doesn't do anything but print the help. > > Can't say that my eyes see much of a difference though ;). But no reason > not to keep it in. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xiut.xu@REDACTED Mon Jan 5 17:28:02 2015 From: xiut.xu@REDACTED (xu xiut) Date: Mon, 5 Jan 2015 11:28:02 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? Message-ID: I am looking at using Erlang for an exchange. If I'm lucky, my transaction volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec At this rate, maybe Erlang wouldn't be necessary, however, would it be the case that Erlang provides better semantics surrounding failure compared to other languages and their ecosystems, and so, even then it would be a great tool choice? Is Erlang ideal for scaling out such an exchange? I wonder what is the minimum number of trades a second that would be difficult for erlang to process. I realize it's difficult to answer given the trade execution path isn't described, but it's the first question that comes to mind when I started thinking about the language of choice. Any thoughts or suggestions are welcomed, and thank you for letting me share this here! -------------- next part -------------- An HTML attachment was scrubbed... URL: From jared.kofron@REDACTED Mon Jan 5 21:23:37 2015 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 05 Jan 2015 20:23:37 +0000 Subject: [erlang-questions] Is Erlang ideal for a global exchange? References: Message-ID: It's a hard question to answer, but I can give you some proxy data. Until recently I was maintaining an Erlang application that was processing data requests in a laboratory - get the value of this sensor, start that process, etc. Communication took place over HTTP and was mostly between the Erlang app and a CouchDB node which did not live on the same physical hardware. When I "stress tested" the app with several hundred concurrent processes that were requesting data (and each transaction took several ms or more to process), the latency was flat compared to a single requestor running in as tight a loop as possible. Granted, there was some resource competition in the app so some of those requests were running serially - BUT, processing many hundreds of requests per second was nowhere near enough load to cause any issues at all. Not sure if that's any real help, but my two cents. On Mon Jan 05 2015 at 10:14:38 AM xu xiut wrote: > I am looking at using Erlang for an exchange. If I'm lucky, my transaction > volume will be 100 - 150k day or about 4000 transactions/hour, about > 1.5/sec At this rate, maybe Erlang wouldn't be necessary, however, would it > be the case that Erlang provides better semantics surrounding failure > compared to other languages and their ecosystems, and so, even then it > would be a great tool choice? > > Is Erlang ideal for scaling out such an exchange? I wonder what is the > minimum number of trades a second that would be difficult for erlang to > process. I realize it's difficult to answer given the trade execution path > isn't described, but it's the first question that comes to mind when I > started thinking about the language of choice. > > Any thoughts or suggestions are welcomed, and thank you for letting me > share this here! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jan 5 21:34:47 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Mon, 5 Jan 2015 21:34:47 +0100 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: Message-ID: <690FEA96-04F2-4063-B14D-EF9E1376A2C3@gmail.com> Erlang is most likely ideal for this use case. But if your first worry is scalability, then you are looking at it from the wrong direction. You need to start with your database. That is where the scalability issue is. Sergej On 05 Jan 2015, at 17:28, xu xiut wrote: > I am looking at using Erlang for an exchange. If I'm lucky, my transaction volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec At this rate, maybe Erlang wouldn't be necessary, however, would it be the case that Erlang provides better semantics surrounding failure compared to other languages and their ecosystems, and so, even then it would be a great tool choice? > > Is Erlang ideal for scaling out such an exchange? I wonder what is the minimum number of trades a second that would be difficult for erlang to process. I realize it's difficult to answer given the trade execution path isn't described, but it's the first question that comes to mind when I started thinking about the language of choice. > > Any thoughts or suggestions are welcomed, and thank you for letting me share this here! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon Jan 5 22:14:39 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 6 Jan 2015 01:14:39 +0400 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <690FEA96-04F2-4063-B14D-EF9E1376A2C3@gmail.com> References: <690FEA96-04F2-4063-B14D-EF9E1376A2C3@gmail.com> Message-ID: Stock exchange is not about scalability =) It is about performance. If you don't need to think about sub-microsecond precision and more than 1 million events per second, then erlang is ok. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Mon Jan 5 22:42:25 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 5 Jan 2015 22:42:25 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 10:08 AM, Roger Lipscombe wrote: > For another data point, I used "-s erlang halt" instead of "-s init > stop", and saw much bigger savings. The average time here was 0.10 > secs. From a brief look, I don't know which of these is more > relevant. That's most likely due to Raimo's flushing fixes in R15B01. Before that, the VM would terminate too soon. From g@REDACTED Mon Jan 5 22:49:18 2015 From: g@REDACTED (Garrett Smith) Date: Mon, 5 Jan 2015 15:49:18 -0600 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: Message-ID: On Mon, Jan 5, 2015 at 10:28 AM, xu xiut wrote: > I am looking at using Erlang for an exchange. If I'm lucky, my transaction > volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec > At this rate, maybe Erlang wouldn't be necessary, however, would it be the > case that Erlang provides better semantics surrounding failure compared to > other languages and their ecosystems, and so, even then it would be a great > tool choice? > > Is Erlang ideal for scaling out such an exchange? I wonder what is the > minimum number of trades a second that would be difficult for erlang to > process. I realize it's difficult to answer given the trade execution path > isn't described, but it's the first question that comes to mind when I > started thinking about the language of choice. > > Any thoughts or suggestions are welcomed, and thank you for letting me share > this here! >From my point of view, Erlang is *the* platform to use for iterative and experimental development of back-end/server-side applications. If you know *precisely* what you need to build and you're fairly certain that its scope won't change over time, other languages could give you some advantages, especially in terms of latency and throughout - and even concurrency, provided again you know *exactly* what you're building ahead of time and how they'll perform in production, over time. It's hard to know all this. Maybe if this is the fifth exchange you've built of this exact type. The reason Erlang is great when you don't have god knowledge about what you're building ahead of time is that it's "process oriented". You write code that runs sequentially in separate fully isolated processes (one can't corrupt another). This makes your application a system of small, independent threads of logic. Each process can fail (and likely will given enough time) and your system of a whole will keep running. The result is that you can move fast with experiments, new code, changes, etc. and not fubar your entire app. Ask around and you'll hear from others that this is a material savings in time. You go faster. You tend to write code that evolves more gracefully with more isolated effects. You also tend not to run into the tangled, impossible-to-truly-fix mess that is multi-threaded code at high levels of concurrency. If you've ever worked with (or written) code that just stops working at some level of scale because of threading issues and concluded, "OMG, this code will never actually work" you will be happy with Erlang. It's very hard to get into that situation with Erlang. You just don't have the length of rope needed. Apart from my touchy-feely reply here, I know a number of trading companies that are using Erlang to build exchanges and are very happy. Erlang is more politically risky than technically risky. It you have some autonomy in making that decision, I'd recommend writing a small kernel of an app in Erlang that does something you think is interesting (related to your problem) and get a feel for things. If you need any specific help about where to start, approach, etc. this is the right place to ask. The more specific you are in your problem definition, the easier it will be to help :) Garrett From xiut.xu@REDACTED Mon Jan 5 21:49:43 2015 From: xiut.xu@REDACTED (xu xiut) Date: Mon, 5 Jan 2015 15:49:43 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <690FEA96-04F2-4063-B14D-EF9E1376A2C3@gmail.com> References: <690FEA96-04F2-4063-B14D-EF9E1376A2C3@gmail.com> Message-ID: Jared, that's assuring. Sergej, I am looking into this as well: http://www.postgresql.org/message-id/flat/CANVqfJFqAuBkP5dwG7q-uwnZFd_CMOy4Si5dD8aCCtGQqDw=KA@REDACTED#CANVqfJFqAuBkP5dwG7q-uwnZFd_CMOy4Si5dD8aCCtGQqDw=KA@REDACTED On Mon, Jan 5, 2015 at 3:34 PM, Sergej Jurecko wrote: > Erlang is most likely ideal for this use case. But if your first worry is > scalability, then you are looking at it from the wrong direction. You need > to start with your database. That is where the scalability issue is. > > > > Sergej > > On 05 Jan 2015, at 17:28, xu xiut wrote: > > > I am looking at using Erlang for an exchange. If I'm lucky, my > transaction volume will be 100 - 150k day or about 4000 transactions/hour, > about 1.5/sec At this rate, maybe Erlang wouldn't be necessary, however, > would it be the case that Erlang provides better semantics surrounding > failure compared to other languages and their ecosystems, and so, even then > it would be a great tool choice? > > > > Is Erlang ideal for scaling out such an exchange? I wonder what is the > minimum number of trades a second that would be difficult for erlang to > process. I realize it's difficult to answer given the trade execution path > isn't described, but it's the first question that comes to mind when I > started thinking about the language of choice. > > > > Any thoughts or suggestions are welcomed, and thank you for letting me > share this here! > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Mon Jan 5 23:07:29 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 5 Jan 2015 23:07:29 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> References: <54AA6B03.2020804@ninenines.eu> <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> Message-ID: On Mon, Jan 5, 2015 at 4:08 PM, Tristan Sloughter wrote: > I added all those arguments to rebar3's escript and seems to be a > consistent speedup as expected, dropping from 0.6s to 0.3s for a run > that doesn't do anything but print the help. Are you certain that nothing will break or no plugin will be constrained by that? If not, we should consider it for rebar 2.x as well. > Can't say that my eyes see much of a difference though ;). But no > reason not to keep it in. Well, it depends on what response times your other interactions with the machine in front of you have. Icon animations and other solutions were implemented to signal that it's pointless to click again and one has to wait for the action to be completed. On the command line it _definitely_ has to be under .5 or .4 seconds to not disrupt, so going from .6 to .3 is very noticeable difference. From t@REDACTED Mon Jan 5 23:15:30 2015 From: t@REDACTED (Tristan Sloughter) Date: Mon, 05 Jan 2015 16:15:30 -0600 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> Message-ID: <1420496130.1428538.209942793.035FBD35@webmail.messagingengine.com> I'm not sure what ramifications it'll have. 0 async threads may be bad to do for a tool that does so much file processing. -- Tristan Sloughter t@REDACTED On Mon, Jan 5, 2015, at 04:07 PM, Tuncer Ayaz wrote: > On Mon, Jan 5, 2015 at 4:08 PM, Tristan Sloughter wrote: > > I added all those arguments to rebar3's escript and seems to be a > > consistent speedup as expected, dropping from 0.6s to 0.3s for a run > > that doesn't do anything but print the help. > > Are you certain that nothing will break or no plugin will be > constrained by that? If not, we should consider it for rebar 2.x as > well. > > > Can't say that my eyes see much of a difference though ;). But no > > reason not to keep it in. > > Well, it depends on what response times your other interactions with > the machine in front of you have. Icon animations and other solutions > were implemented to signal that it's pointless to click again and one > has to wait for the action to be completed. > > On the command line it _definitely_ has to be under .5 or .4 seconds > to not disrupt, so going from .6 to .3 is very noticeable difference. From tuncer.ayaz@REDACTED Mon Jan 5 23:19:14 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 5 Jan 2015 23:19:14 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <54AA6B03.2020804@ninenines.eu> References: <54AA6B03.2020804@ninenines.eu> Message-ID: On Mon, Jan 5, 2015 at 11:44 AM, Lo?c Hoguin wrote: > It's actually quite funny how Erlang compiles many projects > faster than it boots. :-) A lot of "compilation" is also going on in the beam loader, so... From tuncer.ayaz@REDACTED Mon Jan 5 23:21:36 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 5 Jan 2015 23:21:36 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> Message-ID: On Mon, Jan 5, 2015 at 6:12 PM, Bjorn-Egil Dahlberg wrote: > As I mentioned in a previous thread, the start time of escript and > erl is to an high degree influenced by the all the modules that > needs to be loaded. More modules, bigger modules (line info). > This is also why +A0 has an impact. Just to be clear, you aren't saying +A1 implies loading extra modules, are you? > I have to caution you to use +A0 though. Short-circuiting file > operation by using no async-threads may be hazardous to your system. > In most cases it is just fine to use it but this is very > unpredictable. The schedulers can become very unhappy. > > We can change the default arguments to escript to mitigate this, for > instance: > erlang:master...egil/default-escript-arguments Isn't +sbtu already the default? > As for something more rigorous: Dizzy had a proposal of creating > escripts with a beam-binary with erlang-archives appended to it. I > don't know how far he got but with this. Something like that might > go a long way to speed things up. Never heard of that. Care to elaborate? The escript archives we create currently are already a header followed by a zip archive, but IIRC concatenating beam files also works, though I never tried it and just recall the escript loader bits related to it. From wallentin.dahlberg@REDACTED Mon Jan 5 23:41:20 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Mon, 5 Jan 2015 23:41:20 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <1420470519.1315535.209746369.0DAC0238@webmail.messagingengine.com> Message-ID: 2015-01-05 23:21 GMT+01:00 Tuncer Ayaz : > On Mon, Jan 5, 2015 at 6:12 PM, Bjorn-Egil Dahlberg wrote: > > As I mentioned in a previous thread, the start time of escript and > > erl is to an high degree influenced by the all the modules that > > needs to be loaded. > > More modules, bigger modules (line info). > > > This is also why +A0 has an impact. > > Just to be clear, you aren't saying +A1 implies loading extra modules, > are you? > No extra modules. > > > I have to caution you to use +A0 though. Short-circuiting file > > operation by using no async-threads may be hazardous to your system. > > In most cases it is just fine to use it but this is very > > unpredictable. The schedulers can become very unhappy. > > > > We can change the default arguments to escript to mitigate this, for > > instance: > > erlang:master...egil/default-escript-arguments > > Isn't +sbtu already the default? > Yes. > > > As for something more rigorous: Dizzy had a proposal of creating > > escripts with a beam-binary with erlang-archives appended to it. I > > don't know how far he got but with this. Something like that might > > go a long way to speed things up. > > Never heard of that. Care to elaborate? > > The escript archives we create currently are already a header followed > by a zip archive, but IIRC concatenating beam files also works, though > I never tried it and just recall the escript loader bits related to > it. > I talked to him recently. I think he dropped the project. If he sees this I hope he can elaborate on it. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Mon Jan 5 23:43:58 2015 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 5 Jan 2015 23:43:58 +0100 Subject: [erlang-questions] Maps in typespecs (and dialyzer) Message-ID: Hello everyone, The typespecs documentation says dialyzer does not yet use map pairs information (which is fine). The issue is that it does not describe what are the semantic implications of map pairs in types. To be clear, the following type: #{foo => any()} should: 1) match maps containing only the key foo 2) match maps containing at least the key foo Additionally, what would #{_ => _} mean in any of the schemas above? *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jan 6 00:06:53 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 06 Jan 2015 00:06:53 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> Message-ID: <54AB190D.8070402@ninenines.eu> On 01/05/2015 11:19 PM, Tuncer Ayaz wrote: > On Mon, Jan 5, 2015 at 11:44 AM, Lo?c Hoguin wrote: >> It's actually quite funny how Erlang compiles many projects >> faster than it boots. :-) > > A lot of "compilation" is also going on in the beam loader, so... Yes that also adds to the load time, but it probably wouldn't be too noticeable if erlc was loading only the modules it requires instead of everything. I don't think it uses the network code, dets, process groups and so on, and it may also not need the whole OTP application framework. I will play with generating a smaller boot file when I get the time and see what happens. If it can be made closer to 0s this would allow me to just use %.erl: %.beam rules, something that's been on my wishlist from the start. Exciting thread, loving everyone involved! -- Lo?c Hoguin http://ninenines.eu From michael.santos@REDACTED Tue Jan 6 02:11:06 2015 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 5 Jan 2015 20:11:06 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: Message-ID: <20150106011106.GA31238@brk> On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips wrote: > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've done a > bit of looking at the replacement, pf, and it looks like dropping packets > based on percentage is doable, as is bandwidth throttling (something I'd > like to add, in general), but I don't see any way to induce a delay, beyond > an implicit one based on tos prioritization. If someone knows how and can > point me in the right direction I'd appreciate it. The portable way is to use a tuntap device. Then you can arbitrarily drop packets, throttle bandwidth, introduce latency, whatever, from your code. Sort of like quickcheck for networks :) > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko > wrote: > > > This looks like a great tool and something that could easily be added to > > unit tests. > > Anyone with ipfw skills to add bsd/osx support? > > > > Sergej > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" > > wrote: > > > >> https://github.com/lostcolony/damocles > >> > >> I asked a while back on this mailing list if anyone had any useful > >> libraries or similar for testing distribution scenarios. I only got back a > >> few responses (maybe co-op riak_test? Maybe make use of the underlying > >> Linux traffic control and network emulation apps?), and my own searches, > >> while finding a few libraries, didn't find anything I could easily co-op > >> for my purposes. > >> > >> To that end, I went ahead and spent part of my break on this, and it just > >> got sufficiently feature complete to throw out there. I haven't had a > >> chance to really start using it heavily, and I've only been testing it on > >> my dev box, but a basic run through of the functionality as I typed up the > >> readme worked (so any issues being pointed out would be appreciated). > >> Essentially, it allows you to create and manipulate local interfaces on a > >> Linux machine to emulate packet delay and loss (using the underlying > >> traffic control and network emulation mechanisms), with a number of > >> convenience methods to (hopefully) easily describe fairly intricate > >> distribution scenarios. > >> > >> Things like "create these 5 interfaces, (now from my test code, launch a > >> copy of my app on each one, or even a different app on one of them, to see > >> what happens when that resource is flaky); now make it so 1 and 2 can't > >> talk to 3 and 4, and vice versa, but everyone can still talk to 5, but > >> replies have a 50% chance of being dropped from 5 when responding to 1 and > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run more of our > >> test code to assert that trying to write to any node still succeeds); okay, > >> now let's restore the network back to normal (and have our test code make > >> sure the write was retained)", or whatever, can be set up in a > >> straightforward, automated manner as part of a common test run, and not be > >> reliant on certain VMs being up, nor the tests being run on a specific > >> network. The tradeoff, obviously, being that you can't really load test > >> things with it. Still, it fits my basic needs, and I figured it might be of > >> use to others. > >> > >> I'll be adding some simple examples when I next get free time (I ran out > >> of it from the holiday break without getting to them; dunno when I will), > >> and will try and get to any bugs or simple suggestions in a timely manner, > >> but hopefully it's fairly straightforward and useful as is. > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lostcolony@REDACTED Tue Jan 6 02:55:06 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Mon, 5 Jan 2015 20:55:06 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: <20150106011106.GA31238@brk> References: <20150106011106.GA31238@brk> Message-ID: Hmm, I've had so little experience with tuntap interfaces I didn't think of them. That sounds like the way to go to get things working on OSX, though at a much larger amount of effort (both the learning curve on my part, and re-implementing the degradation behaviors I get for free from the kernel). Thanks for suggesting it; once I get things to where it's usable on Linux in all the ways I want, some research and a rewrite may be in order to get it fully portable. On Mon, Jan 5, 2015 at 8:11 PM, Michael Santos wrote: > On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips wrote: > > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've done a > > bit of looking at the replacement, pf, and it looks like dropping packets > > based on percentage is doable, as is bandwidth throttling (something I'd > > like to add, in general), but I don't see any way to induce a delay, > beyond > > an implicit one based on tos prioritization. If someone knows how and can > > point me in the right direction I'd appreciate it. > > The portable way is to use a tuntap device. Then you can arbitrarily > drop packets, throttle bandwidth, introduce latency, whatever, from your > code. Sort of like quickcheck for networks :) > > > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko > > > wrote: > > > > > This looks like a great tool and something that could easily be added > to > > > unit tests. > > > Anyone with ipfw skills to add bsd/osx support? > > > > > > Sergej > > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" > > > wrote: > > > > > >> https://github.com/lostcolony/damocles > > >> > > >> I asked a while back on this mailing list if anyone had any useful > > >> libraries or similar for testing distribution scenarios. I only got > back a > > >> few responses (maybe co-op riak_test? Maybe make use of the underlying > > >> Linux traffic control and network emulation apps?), and my own > searches, > > >> while finding a few libraries, didn't find anything I could easily > co-op > > >> for my purposes. > > >> > > >> To that end, I went ahead and spent part of my break on this, and it > just > > >> got sufficiently feature complete to throw out there. I haven't had a > > >> chance to really start using it heavily, and I've only been testing > it on > > >> my dev box, but a basic run through of the functionality as I typed > up the > > >> readme worked (so any issues being pointed out would be appreciated). > > >> Essentially, it allows you to create and manipulate local interfaces > on a > > >> Linux machine to emulate packet delay and loss (using the underlying > > >> traffic control and network emulation mechanisms), with a number of > > >> convenience methods to (hopefully) easily describe fairly intricate > > >> distribution scenarios. > > >> > > >> Things like "create these 5 interfaces, (now from my test code, > launch a > > >> copy of my app on each one, or even a different app on one of them, > to see > > >> what happens when that resource is flaky); now make it so 1 and 2 > can't > > >> talk to 3 and 4, and vice versa, but everyone can still talk to 5, but > > >> replies have a 50% chance of being dropped from 5 when responding to > 1 and > > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run more of > our > > >> test code to assert that trying to write to any node still succeeds); > okay, > > >> now let's restore the network back to normal (and have our test code > make > > >> sure the write was retained)", or whatever, can be set up in a > > >> straightforward, automated manner as part of a common test run, and > not be > > >> reliant on certain VMs being up, nor the tests being run on a specific > > >> network. The tradeoff, obviously, being that you can't really load > test > > >> things with it. Still, it fits my basic needs, and I figured it might > be of > > >> use to others. > > >> > > >> I'll be adding some simple examples when I next get free time (I ran > out > > >> of it from the holiday break without getting to them; dunno when I > will), > > >> and will try and get to any bugs or simple suggestions in a timely > manner, > > >> but hopefully it's fairly straightforward and useful as is. > > >> > > >> _______________________________________________ > > >> erlang-questions mailing list > > >> erlang-questions@REDACTED > > >> http://erlang.org/mailman/listinfo/erlang-questions > > >> > > >> > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jan 6 05:20:26 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 6 Jan 2015 17:20:26 +1300 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <54AA507F.5040509@gmail.com> References: <54AA507F.5040509@gmail.com> Message-ID: On 5/01/2015, at 9:51 pm, Martin Koroudjiev wrote: > Hello, > > First of all - Happy New Year! > > Suppose we have a function that accepts 2 integers and we want to react > only when both integers are greater than 0 and one of them is less than 100: > > I tried: > (dilbert@REDACTED)1> F = fun(N, M) when N > 0, M > 0, N < 100; M < 100 > -> cool; (_, _) -> not_cool end. > #Fun > (dilbert@REDACTED)2> F(1,2). > cool > (dilbert@REDACTED)3> F(1,200). > cool > (dilbert@REDACTED)4> F(0,200). > not_cool > (dilbert@REDACTED)5> F(0,50). > cool > > The last is not correct. What do you mean, ?not correct?? You have defined: F(N, M) = if (N > 0 and N < 100 and M > 0) or M < 100 then cool else not_cool In the call f(0, 50), 50 < 100, so the answer ?cool? is the only possible answer. > What is the order of evaluation of the guards? Sadly parentheses are not > allowed in guards. It?s not clear to me what you mean by ?order of evaluation? either. The outcome here has nothing to do with the order in which things are evaluated. I _think_ you are talking about precedence, which is clearly spelled out in the reference manual. ?;? (OR) has, as is traditional, wider scope than ?,? (AND). I have always found it inexplicable that Erlang does not allow the guard combination operators ?,? and ?;? to be nested, with proper use of parentheses when appropriate. When ?andalso? and ?orelse? were allowed in guards, the restriction became impossible to justify any longer. If you want an ?OR? to govern an ?AND? you will have to use ?andalso? instead of ?,? and ?orelse? instead of ?;?. From boozelclark@REDACTED Tue Jan 6 06:44:16 2015 From: boozelclark@REDACTED (Chris Clark) Date: Tue, 6 Jan 2015 07:44:16 +0200 Subject: [erlang-questions] Stopping a supervisor running under a supervisor In-Reply-To: References: Message-ID: Hi Antonio Thanks for the assistance. From what I have read a supervisor should stop if it is sent a shutdown signal using exit(Pid, shutdown). but it will only accept that signal from its parent process in an OTP application. I tried creating a function in the parent supervisor(my_supersup) stop_child(Pid) that then sends the exit(pid, shutdown) signal and then call that from the shell but that doesn't seem to stop it either. Thanks, On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias wrote: > Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message and > use that to stop the process? > > ADias > > On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark > wrote: > >> Hi >> >> I am fairly new to erlang and have a question regarding properly stopping >> a supervisor that resides directly under another supervisor. >> >> My application consist of a top level supervisor (my_supersup) which is a >> simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >> supervises a gen_server, serv and another supervisor that is also a >> simple_one_for_one supervisor of some workers >> >> my_supersup >> (simple_one_for_one, transient) >> | >> |_______ ... >> | >> my_sup (one_for_all, transient) >> |________________ >> | | >> | serv >> worker_sup >> (simple_one_for_one, transient) >> |___________ ... >> | | >> w1 w2 >> >> How do shutdown an instance of my_sup gracefully without killing it. At >> present I can do it with supervisor:terminate_child and >> supervisor:delete_child but I would like them to shutdown rather than be >> killed if possible? >> >> Thanks in advance. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Jan 6 10:44:01 2015 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 6 Jan 2015 10:44:01 +0100 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: Message-ID: > On 05 Jan 2015, at 17:28, xu xiut wrote: > > I am looking at using Erlang for an exchange. If I'm lucky, my transaction volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec At this rate, maybe Erlang wouldn't be necessary It usually isn?t the ?fair-weather? performance requirements that test the metal of a system, but the exception cases. Presumably, the transaction volume isn?t ideally spread across the day. What kinds of bursts can be envisioned? Also, whether 1.5 transactions/sec is easy or hard depends in part on how much persisting needs to be done, and how many external systems need to be involved in the transaction. Even so, this is the stuff Erlang was made for - *especially* if transactions involve communication with external systems. With Erlang, you can easily break down the design into multiple state machines, which execute in their own thread and maintain their own state. This makes it easier to understand the design, and to test and debug it. After all, when weighing different requirements, correctness and stability usually rank higher than speed. And in ?interesting? systems, the main thing affecting performance tends to be complexity: if the design is opaque, it will be hard to reason about performance. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jan 6 11:13:41 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 6 Jan 2015 11:13:41 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <54AB190D.8070402@ninenines.eu> References: <54AA6B03.2020804@ninenines.eu> <54AB190D.8070402@ninenines.eu> Message-ID: On Tue, Jan 6, 2015 at 12:06 AM, Loic Hoguin wrote: > On 01/05/2015 11:19 PM, Tuncer Ayaz wrote: > > > > On Mon, Jan 5, 2015 at 11:44 AM, Loic Hoguin wrote: > > > > > > It's actually quite funny how Erlang compiles many projects > > > faster than it boots. :-) > > > > > > A lot of "compilation" is also going on in the beam loader, so... > > > Yes that also adds to the load time, but it probably wouldn't be too > noticeable if erlc was loading only the modules it requires instead > of everything. I don't think it uses the network code, dets, process It can use the network code if code is not loaded from disk. Erlang's distributed nature is the reason for some indirection you tried to circumvent in, for example, file operations :). > groups and so on, and it may also not need the whole OTP application > framework. > > I will play with generating a smaller boot file when I get the time > and see what happens. If it can be made closer to 0s this would > allow me to just use %.erl: %.beam rules, something that's been on > my wishlist from the start. Maybe the resulting boot file can be used for escript as well. Also, the speedups we've discussed also apply to older versions and are not fixing the overhead introduced between R13 and now. So, if you're serious about this, and want to invest time, I'd suggest to profile R13 against R14 and R15 with your trace/profile tool of choice and see where time in (mostly) C and maybe Erlang is spent since >=R14. From ulf@REDACTED Tue Jan 6 11:25:20 2015 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 6 Jan 2015 11:25:20 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: > On 06 Jan 2015, at 05:20, Richard A. O'Keefe wrote: > > If you want an ?OR? to govern an ?AND? you will have to use ?andalso? instead > of ?,? and ?orelse? instead of ?;?. ?or suffer some repetition together with some careful layout. When digging for an example, I noticed that I have been ambivalent on how to express these kinds of guards. In locks_agent.erl, for example, I found the following: lock(Agent, [_|_] = Obj, Mode, [_|_] = Where, R) when is_pid(Agent) andalso (Mode == read orelse Mode == write) andalso (R == all orelse R == any orelse R == majority orelse R == majority_alive orelse R == all_alive) -> (yuck!) and later: change_flag(Agent, Option, Bool) when is_boolean(Bool), Option == abort_on_deadlock; is_boolean(Bool), Option == await_nodes; is_boolean(Bool), Option == notify -> I actually prefer the repetition, but the first example isn?t helped by that. It really could use some new syntax: #agent(A) when is_pid(A) -> A. #obj([_|_] = O) -> O. #mode(M) when M==read; M==write -> M. #where([_|_] = W) -> W. #req(R) when R==all; R==any; R==majority; ? -> R. lock(#agent(Agent), #obj(Obj), #mode(Mode), #where(Where), #req(R)) -> ? (EEP 29, or at least my sloppy understanding of it.) BR, Ulf Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From imantc@REDACTED Tue Jan 6 11:33:10 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 6 Jan 2015 11:33:10 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: Are guards - the right place for complicated logic? Why not move such logic inside a function and use "if"? From essen@REDACTED Tue Jan 6 11:43:30 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 06 Jan 2015 11:43:30 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <54AB190D.8070402@ninenines.eu> Message-ID: <54ABBC52.90408@ninenines.eu> On 01/06/2015 11:13 AM, Tuncer Ayaz wrote: > On Tue, Jan 6, 2015 at 12:06 AM, Loic Hoguin wrote: >> Yes that also adds to the load time, but it probably wouldn't be too >> noticeable if erlc was loading only the modules it requires instead >> of everything. I don't think it uses the network code, dets, process > > It can use the network code if code is not loaded from disk. Erlang's > distributed nature is the reason for some indirection you tried to > circumvent in, for example, file operations :). But erlc only compiles *files* doesn't it? It doesn't start distribution or anything so there should be no indirection? >> groups and so on, and it may also not need the whole OTP application >> framework. >> >> I will play with generating a smaller boot file when I get the time >> and see what happens. If it can be made closer to 0s this would >> allow me to just use %.erl: %.beam rules, something that's been on >> my wishlist from the start. > > Maybe the resulting boot file can be used for escript as well. Well "escript" wouldn't be part of the boot file if erlc doesn't need it, so my guess is no. :-) The hard part is figuring out what it actually needs. > Also, the speedups we've discussed also apply to older versions and > are not fixing the overhead introduced between R13 and now. So, if > you're serious about this, and want to invest time, I'd suggest to > profile R13 against R14 and R15 with your trace/profile tool of choice > and see where time in (mostly) C and maybe Erlang is spent since >=R14. I wish I could invest time now but this and the boot file experiment will probably have to wait for a while. -- Lo?c Hoguin http://ninenines.eu From ulf@REDACTED Tue Jan 6 14:00:54 2015 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 6 Jan 2015 14:00:54 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <54AA507F.5040509@gmail.com> Message-ID: > On 06 Jan 2015, at 11:33, Imants Cekusins wrote: > > Are guards - the right place for complicated logic? > > Why not move such logic inside a function and use "if?? In that particular example, it?s an API function, and the point of putting the checks in a guard sequence is that the function should fail if called with invalid arguments. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From zkessin@REDACTED Tue Jan 6 15:13:37 2015 From: zkessin@REDACTED (Zachary Kessin) Date: Tue, 06 Jan 2015 16:13:37 +0200 Subject: [erlang-questions] Just a reminder, live Podcast today! Message-ID: <54ABED91.8070403@gmail.com> Here is the link for the live podcast tomorrow https://plus.google.com/events/cj31fp9dm6hnr1m2diu8qaeen88 It will be at 18:00 gmt, and will be released afterwards as the normal audio --Zach From sean@REDACTED Tue Jan 6 15:20:30 2015 From: sean@REDACTED (Sean Cribbs) Date: Tue, 6 Jan 2015 08:20:30 -0600 Subject: [erlang-questions] Stopping a supervisor running under a supervisor In-Reply-To: References: Message-ID: Did you try supervisor:terminate_child/2? On Mon, Jan 5, 2015 at 11:44 PM, Chris Clark wrote: > Hi Antonio > > Thanks for the assistance. From what I have read a supervisor should stop > if it is sent a shutdown signal using exit(Pid, shutdown). but it will > only accept that signal from its parent process in an OTP application. I > tried creating a function in the parent supervisor(my_supersup) > stop_child(Pid) that then sends the exit(pid, shutdown) signal and then > call that from the shell but that doesn't seem to stop it either. > > Thanks, > > On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias > wrote: > >> Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message >> and use that to stop the process? >> >> ADias >> >> On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark >> wrote: >> >>> Hi >>> >>> I am fairly new to erlang and have a question regarding properly >>> stopping a supervisor that resides directly under another supervisor. >>> >>> My application consist of a top level supervisor (my_supersup) which is >>> a simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >>> supervises a gen_server, serv and another supervisor that is also a >>> simple_one_for_one supervisor of some workers >>> >>> my_supersup >>> (simple_one_for_one, transient) >>> | >>> |_______ ... >>> | >>> my_sup (one_for_all, transient) >>> |________________ >>> | | >>> | serv >>> worker_sup >>> (simple_one_for_one, transient) >>> |___________ ... >>> | | >>> w1 w2 >>> >>> How do shutdown an instance of my_sup gracefully without killing it. At >>> present I can do it with supervisor:terminate_child and >>> supervisor:delete_child but I would like them to shutdown rather than be >>> killed if possible? >>> >>> Thanks in advance. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Sean Cribbs Sr. Software Engineer Basho Technologies, Inc. http://basho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From boozelclark@REDACTED Tue Jan 6 16:31:31 2015 From: boozelclark@REDACTED (Chris Clark) Date: Tue, 6 Jan 2015 17:31:31 +0200 Subject: [erlang-questions] Stopping a supervisor running under a supervisor In-Reply-To: References: Message-ID: Hi Sean supervisor:terminate_child/2 does work however it causes errors in my logs so i am looking for a way to gracefully terminate the children. Thanks On Tue, Jan 6, 2015 at 4:20 PM, Sean Cribbs wrote: > Did you try supervisor:terminate_child/2? > > On Mon, Jan 5, 2015 at 11:44 PM, Chris Clark > wrote: > >> Hi Antonio >> >> Thanks for the assistance. From what I have read a supervisor should >> stop if it is sent a shutdown signal using exit(Pid, shutdown). but it >> will only accept that signal from its parent process in an OTP application. >> I tried creating a function in the parent supervisor(my_supersup) >> stop_child(Pid) that then sends the exit(pid, shutdown) signal and then >> call that from the shell but that doesn't seem to stop it either. >> >> Thanks, >> >> On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias >> wrote: >> >>> Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message >>> and use that to stop the process? >>> >>> ADias >>> >>> On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark >>> wrote: >>> >>>> Hi >>>> >>>> I am fairly new to erlang and have a question regarding properly >>>> stopping a supervisor that resides directly under another supervisor. >>>> >>>> My application consist of a top level supervisor (my_supersup) which is >>>> a simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >>>> supervises a gen_server, serv and another supervisor that is also a >>>> simple_one_for_one supervisor of some workers >>>> >>>> my_supersup >>>> (simple_one_for_one, transient) >>>> | >>>> |_______ ... >>>> | >>>> my_sup (one_for_all, transient) >>>> |________________ >>>> | | >>>> | serv >>>> worker_sup >>>> (simple_one_for_one, transient) >>>> |___________ ... >>>> | | >>>> w1 w2 >>>> >>>> How do shutdown an instance of my_sup gracefully without killing it. At >>>> present I can do it with supervisor:terminate_child and >>>> supervisor:delete_child but I would like them to shutdown rather than be >>>> killed if possible? >>>> >>>> Thanks in advance. >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Sean Cribbs > Sr. Software Engineer > Basho Technologies, Inc. > http://basho.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.ralev@REDACTED Tue Jan 6 17:30:22 2015 From: vladimir.ralev@REDACTED (Vladimir Ralev) Date: Tue, 6 Jan 2015 18:30:22 +0200 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: Message-ID: IMHO, think about what's your worst-case transaction. If are doing complex transactions (like a bitcoin contract transaction) or other distributed transactions with rules, then it will be a bit more difficult to accomodate that in Erlang processes in a scalable way. If the transactions are isolated from one another and short then it's ok. Think also what would you want to happen when a machine crashes in the middle of a transaction in any state, Erlang's way is to retry or forget, but if you want to continue running no matter what then you must reach outside Erlang (socket replication, process state replication, things like this). On Mon, Jan 5, 2015 at 6:28 PM, xu xiut wrote: > I am looking at using Erlang for an exchange. If I'm lucky, my transaction > volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec > At this rate, maybe Erlang wouldn't be necessary, however, would it be the > case that Erlang provides better semantics surrounding failure compared to > other languages and their ecosystems, and so, even then it would be a great > tool choice? > > Is Erlang ideal for scaling out such an exchange? I wonder what is the > minimum number of trades a second that would be difficult for erlang to > process. I realize it's difficult to answer given the trade execution path > isn't described, but it's the first question that comes to mind when I > started thinking about the language of choice. > > Any thoughts or suggestions are welcomed, and thank you for letting me share > this here! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From thomasl_erlang@REDACTED Tue Jan 6 19:25:39 2015 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 6 Jan 2015 18:25:39 +0000 (UTC) Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: Message-ID: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> Guards are a mess IMO, for reasons including, but not restricted to, the ones you mention. Your example suggests that member/2 could be a useful new guard operation though. Best,Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Jan 6 20:02:44 2015 From: rvirding@REDACTED (Robert Virding) Date: Tue, 6 Jan 2015 20:02:44 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> References: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> Message-ID: Guards were added as an extension to pattern matching, a way of expressing things which could not comfortably done by extending the pattern syntax, for example saying that something had to be an atom or greater then 5. The original guard syntax was different from standard expressions in many ways and much more restricted than today so the difference was clearer. There was less chance of confusion, expressions were expressions and guards were guards. It was making the guard syntax completely the same as expressions and adding things like logical expressions which made the difference less clear and things became more confusing. And calling them guard *expressions* instead of guard *tests* does not help. I think it is very important to stress that the guard and the body of a clause are very different beasts and obey different rules. Adding a proper 'cond' might help. All in IMAO. Robert On 6 January 2015 at 19:25, Thomas Lindgren wrote: > Guards are a mess IMO, for reasons including, but not restricted to, the > ones you mention. > > Your example suggests that member/2 could be a useful new guard operation > though. > > Best, > Thomas > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From flw@REDACTED Tue Jan 6 19:41:58 2015 From: flw@REDACTED (Florian Waas) Date: Tue, 6 Jan 2015 18:41:58 +0000 Subject: [erlang-questions] Differences in code_change callback between gen_server and gen_fsm Message-ID: In gen_server the callback for code_change is spec?d as Module:code_change(OldVsn, State, Extra) -> {ok, NewState} | {error, Reason} in gen_fsm as Module:code_change(OldVsn, StateName, StateData, Extra) -> {ok, NextStateName, NewStateData} i.e., upgrades on FSM?s had better succeed? The underlying implementation in sys.erl is the same for both and allows for an error case. Am I missing some subtlety here? Thanks, -fl. From n.oxyde@REDACTED Wed Jan 7 00:44:35 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 7 Jan 2015 00:44:35 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> Message-ID: Le 6 janv. 2015 ? 20:02, Robert Virding a ?crit : > I think it is very important to stress that the guard and the body of a clause are very different beasts and obey different rules. Adding a proper 'cond' might help. All in IMAO. 'Cond' would help, indeed [1]. Deprecating guard sequences in 'if' expressions too [2]. :) > On 6 January 2015 at 19:25, Thomas Lindgren wrote: > Guards are a mess IMO, for reasons including, but not restricted to, the ones you mention. The award goes to erlang:float/1, messiest guard ever [3]. I would like to see it deprecated (as other old type guards without the 'is' prefix) and the function to be renamed to integer_to_float/1. [1] https://github.com/erlang/otp/pull/464 [2] https://github.com/rebar/rebar/commit/2c4d7d1d9bdc2a11d3f485f844500bf4c2aa77a2 [3] http://www.erlang.org/doc/man/erlang.html#float-1 From ct.radu.001@REDACTED Wed Jan 7 09:57:59 2015 From: ct.radu.001@REDACTED (CT Radu) Date: Wed, 7 Jan 2015 10:57:59 +0200 Subject: [erlang-questions] edoc and erlang code quoting Message-ID: Hi all, Does anyone know how to properly embed erlang code examples inside e @doc tag ? For example, I have something like: %% @doc Mongo Query. %% Answer example: %% %% [ %% {<<"client">>, <<"ClientOne">>}, %% {<<"client-dir">>, <<"/dir/path">>} %% ] %% %% @end I run edoc using : erl -pa ebin/ -noshell\ -eval 'edoc:application(app_name, ".", [{source_path, ["/full/source/path"]}]), init:stop().' And I get this: 2991- fatal: {invalid_name,"<\"clie"} /full/source/path/module.erl, function get_clients_dirs/0: at line 156: error in XML parser: {fatal,{{invalid_name,"<\"clie"}, {file,file_name_unknown}, {line,159}, {col,11}}}. edoc: skipping source file '/full/source/path/module.erl': {'EXIT',error}. edoc: error in doclet 'edoc_doclet': {'EXIT',error}. {"init terminating in do_boot",error} If you have any ideas on how I can generate this properly, I'll be gratefull. Costin-Tiberiu Radu, -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Wed Jan 7 10:07:23 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 7 Jan 2015 10:07:23 +0100 Subject: [erlang-questions] edoc and erlang code quoting In-Reply-To: References: Message-ID: Hi! Since the doc is HTML, I think you need use '<' instead of '<' in the code snippet. regards, Vlad On Wed, Jan 7, 2015 at 9:57 AM, CT Radu wrote: > Hi all, > > Does anyone know how to properly embed erlang code examples inside e @doc > tag ? > > For example, I have something like: > > %% @doc Mongo Query. > %% Answer example: > %% > %% [ > %% {<<"client">>, <<"ClientOne">>}, > %% {<<"client-dir">>, <<"/dir/path">>} > %% ] > %% > %% @end > > I run edoc using : > erl -pa ebin/ -noshell\ > -eval 'edoc:application(app_name, ".", [{source_path, > ["/full/source/path"]}]), init:stop().' > > And I get this: > 2991- fatal: {invalid_name,"<\"clie"} > /full/source/path/module.erl, function get_clients_dirs/0: at line 156: > error in XML parser: {fatal,{{invalid_name,"<\"clie"}, > {file,file_name_unknown}, > {line,159}, > {col,11}}}. > edoc: skipping source file '/full/source/path/module.erl': {'EXIT',error}. > edoc: error in doclet 'edoc_doclet': {'EXIT',error}. > {"init terminating in do_boot",error} > > If you have any ideas on how I can generate this properly, I'll be > gratefull. > Costin-Tiberiu Radu, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Jan 7 10:07:58 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 7 Jan 2015 10:07:58 +0100 Subject: [erlang-questions] edoc and erlang code quoting In-Reply-To: References: Message-ID: Use ```....''' wiki notation. See http://www.erlang.org/doc/apps/edoc/chapter.html#Verbatim_quoting You can use ..., but then you have to HTML-escape all < and & characters in your example code manually. For example: ...{ <<"client">>,<<"ClientOne">>}, ... /Richard On Wed, Jan 7, 2015 at 9:57 AM, CT Radu wrote: > Hi all, > > Does anyone know how to properly embed erlang code examples inside e @doc > tag ? > > For example, I have something like: > > %% @doc Mongo Query. > %% Answer example: > %% > %% [ > %% {<<"client">>, <<"ClientOne">>}, > %% {<<"client-dir">>, <<"/dir/path">>} > %% ] > %% > %% @end > > I run edoc using : > erl -pa ebin/ -noshell\ > -eval 'edoc:application(app_name, ".", [{source_path, > ["/full/source/path"]}]), init:stop().' > > And I get this: > 2991- fatal: {invalid_name,"<\"clie"} > /full/source/path/module.erl, function get_clients_dirs/0: at line 156: > error in XML parser: {fatal,{{invalid_name,"<\"clie"}, > {file,file_name_unknown}, > {line,159}, > {col,11}}}. > edoc: skipping source file '/full/source/path/module.erl': {'EXIT',error}. > edoc: error in doclet 'edoc_doclet': {'EXIT',error}. > {"init terminating in do_boot",error} > > If you have any ideas on how I can generate this properly, I'll be > gratefull. > Costin-Tiberiu Radu, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Jan 7 11:45:51 2015 From: tony@REDACTED (Tony Rogvall) Date: Wed, 7 Jan 2015 11:45:51 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: References: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> Message-ID: <87750908-C635-4B81-B5B1-E6F1FABFA3DA@rogvall.se> > On 7 jan 2015, at 00:44, Anthony Ramine wrote: > > Le 6 janv. 2015 ? 20:02, Robert Virding a ?crit : > >> I think it is very important to stress that the guard and the body of a clause are very different beasts and obey different rules. Adding a proper 'cond' might help. All in IMAO. > > 'Cond' would help, indeed [1]. Deprecating guard sequences in 'if' expressions too [2]. :) > >> On 6 January 2015 at 19:25, Thomas Lindgren wrote: >> Guards are a mess IMO, for reasons including, but not restricted to, the ones you mention. > > The award goes to erlang:float/1, messiest guard ever [3]. I would like to see it deprecated (as other old type guards without the 'is' prefix) and the function to be renamed to integer_to_float/1. I guess number_to_float/1 is more compatible with the old erlang:float/1 function ? > > [1] https://github.com/erlang/otp/pull/464 > [2] https://github.com/rebar/rebar/commit/2c4d7d1d9bdc2a11d3f485f844500bf4c2aa77a2 > [3] http://www.erlang.org/doc/man/erlang.html#float-1 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From n.oxyde@REDACTED Wed Jan 7 12:08:22 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 7 Jan 2015 12:08:22 +0100 Subject: [erlang-questions] Order of evaluation of guards In-Reply-To: <87750908-C635-4B81-B5B1-E6F1FABFA3DA@rogvall.se> References: <1164564425.1263364.1420568739341.JavaMail.yahoo@jws10707.mail.gq1.yahoo.com> <87750908-C635-4B81-B5B1-E6F1FABFA3DA@rogvall.se> Message-ID: <45706B54-C0B2-492D-A3C8-15D35467DB68@gmail.com> Le 7 janv. 2015 ? 11:45, Tony Rogvall a ?crit : > I guess number_to_float/1 is more compatible with the old erlang:float/1 function ? You guessed that right, I forgot float/1 can take floats. From ct.radu.001@REDACTED Wed Jan 7 13:45:54 2015 From: ct.radu.001@REDACTED (CT Radu) Date: Wed, 7 Jan 2015 14:45:54 +0200 Subject: [erlang-questions] edoc and erlang code quoting In-Reply-To: References: Message-ID: Thank you. That was the missing piece. Tiberiu 2015-01-07 11:07 GMT+02:00 Richard Carlsson : > Use ```....''' wiki notation. See > http://www.erlang.org/doc/apps/edoc/chapter.html#Verbatim_quoting > > You can use ..., but then you have to HTML-escape all < and & > characters in your example code manually. For example: ...{ > <<"client">>,<<"ClientOne">>}, ... > > > /Richard > > On Wed, Jan 7, 2015 at 9:57 AM, CT Radu wrote: > >> Hi all, >> >> Does anyone know how to properly embed erlang code examples inside e @doc >> tag ? >> >> For example, I have something like: >> >> %% @doc Mongo Query. >> %% Answer example: >> %% >> %% [ >> %% {<<"client">>, <<"ClientOne">>}, >> %% {<<"client-dir">>, <<"/dir/path">>} >> %% ] >> %% >> %% @end >> >> I run edoc using : >> erl -pa ebin/ -noshell\ >> -eval 'edoc:application(app_name, ".", [{source_path, >> ["/full/source/path"]}]), init:stop().' >> >> And I get this: >> 2991- fatal: {invalid_name,"<\"clie"} >> /full/source/path/module.erl, function get_clients_dirs/0: at line 156: >> error in XML parser: {fatal,{{invalid_name,"<\"clie"}, >> {file,file_name_unknown}, >> {line,159}, >> {col,11}}}. >> edoc: skipping source file '/full/source/path/module.erl': {'EXIT',error}. >> edoc: error in doclet 'edoc_doclet': {'EXIT',error}. >> {"init terminating in do_boot",error} >> >> If you have any ideas on how I can generate this properly, I'll be >> gratefull. >> Costin-Tiberiu Radu, >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Jan 7 16:20:36 2015 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 7 Jan 2015 10:20:36 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: <20150106011106.GA31238@brk> Message-ID: <20150107152036.GA27201@brk> On Mon, Jan 05, 2015 at 08:55:06PM -0500, Christopher Phillips wrote: > Hmm, I've had so little experience with tuntap interfaces I didn't think > of them. That sounds like the way to go to get things working on OSX, > though at a much larger amount of effort (both the learning curve on my > part, and re-implementing the degradation behaviors I get for free from the > kernel). Thanks for suggesting it; once I get things to where it's usable > on Linux in all the ways I want, some research and a rewrite may be in > order to get it fully portable. Sure, that is definitely reasonable. Working with tuntap devices isn't too difficult. For comparison, here is a very simple example that forwards ethernet frames over erlang distribution and randomly drops or delays a percentage of the frames: https://gist.github.com/msantos/f2823fcba40975003dc3 Set up 2 distributed nodes: erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH Then create the tunnel: % Drop 10% of the frames, randomly delay 40% up to 1 second wastrel:start('n@REDACTED', "10.1.1.1", "10.1.1.2", 0.1, 0.4). Try to login using ssh (or whatever)! ssh 10.1.1.2 Meh, I've had to use worse ... Notes: * relies on this erlang tuntap driver https://github.com/msantos/tunctl * beam needs to be running as root or have CAP_NET_ADMIN set: sudo setcap cap_net_admin=ep /usr/local/lib/erlang/erts-*/bin/beam* > On Mon, Jan 5, 2015 at 8:11 PM, Michael Santos > wrote: > > > On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips wrote: > > > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've done a > > > bit of looking at the replacement, pf, and it looks like dropping packets > > > based on percentage is doable, as is bandwidth throttling (something I'd > > > like to add, in general), but I don't see any way to induce a delay, > > beyond > > > an implicit one based on tos prioritization. If someone knows how and can > > > point me in the right direction I'd appreciate it. > > > > The portable way is to use a tuntap device. Then you can arbitrarily > > drop packets, throttle bandwidth, introduce latency, whatever, from your > > code. Sort of like quickcheck for networks :) > > > > > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko > > > > > wrote: > > > > > > > This looks like a great tool and something that could easily be added > > to > > > > unit tests. > > > > Anyone with ipfw skills to add bsd/osx support? > > > > > > > > Sergej > > > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" > > > > wrote: > > > > > > > >> https://github.com/lostcolony/damocles > > > >> > > > >> I asked a while back on this mailing list if anyone had any useful > > > >> libraries or similar for testing distribution scenarios. I only got > > back a > > > >> few responses (maybe co-op riak_test? Maybe make use of the underlying > > > >> Linux traffic control and network emulation apps?), and my own > > searches, > > > >> while finding a few libraries, didn't find anything I could easily > > co-op > > > >> for my purposes. > > > >> > > > >> To that end, I went ahead and spent part of my break on this, and it > > just > > > >> got sufficiently feature complete to throw out there. I haven't had a > > > >> chance to really start using it heavily, and I've only been testing > > it on > > > >> my dev box, but a basic run through of the functionality as I typed > > up the > > > >> readme worked (so any issues being pointed out would be appreciated). > > > >> Essentially, it allows you to create and manipulate local interfaces > > on a > > > >> Linux machine to emulate packet delay and loss (using the underlying > > > >> traffic control and network emulation mechanisms), with a number of > > > >> convenience methods to (hopefully) easily describe fairly intricate > > > >> distribution scenarios. > > > >> > > > >> Things like "create these 5 interfaces, (now from my test code, > > launch a > > > >> copy of my app on each one, or even a different app on one of them, > > to see > > > >> what happens when that resource is flaky); now make it so 1 and 2 > > can't > > > >> talk to 3 and 4, and vice versa, but everyone can still talk to 5, but > > > >> replies have a 50% chance of being dropped from 5 when responding to > > 1 and > > > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run more of > > our > > > >> test code to assert that trying to write to any node still succeeds); > > okay, > > > >> now let's restore the network back to normal (and have our test code > > make > > > >> sure the write was retained)", or whatever, can be set up in a > > > >> straightforward, automated manner as part of a common test run, and > > not be > > > >> reliant on certain VMs being up, nor the tests being run on a specific > > > >> network. The tradeoff, obviously, being that you can't really load > > test > > > >> things with it. Still, it fits my basic needs, and I figured it might > > be of > > > >> use to others. > > > >> > > > >> I'll be adding some simple examples when I next get free time (I ran > > out > > > >> of it from the holiday break without getting to them; dunno when I > > will), > > > >> and will try and get to any bugs or simple suggestions in a timely > > manner, > > > >> but hopefully it's fairly straightforward and useful as is. > > > >> > > > >> _______________________________________________ > > > >> erlang-questions mailing list > > > >> erlang-questions@REDACTED > > > >> http://erlang.org/mailman/listinfo/erlang-questions > > > >> > > > >> > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > From roberto@REDACTED Wed Jan 7 16:23:57 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 7 Jan 2015 16:23:57 +0100 Subject: [erlang-questions] {error,emfile} Message-ID: Dear all, I've built a benchmarking tool and I experience an error when connecting more than 1024 clients to a SSL server. The call to ssl:connect/3 returns a {error,emfile} instead of a Socket. According to inet POSIX Error Codes [1], emfile corresponds to too many open files. I'm running from a UBUNTU 14.04 LTS box. However, I've set my /etc/security/limits.conf with: * soft nofile 999999 * hard nofile 999999 and rebooted the machine. I can see it applied: $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 241197 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 999999 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 241197 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited I've also set /etc/sysctl.conf with: # number of file descriptors fs.file-max=2001000 # extend the ports range net.ipv4.ip_local_port_range=1024 65535 # increase the max number of receive and send buffer size net.core.rmem_max=33554432 net.core.wmem_max=33554432 # increase TCP auto-tuning buffer limits settings net.ipv4.tcp_rmem=4096 16384 33554432 net.ipv4.tcp_wmem=4096 16384 33554432 net.ipv4.tcp_mem=786432 1048576 26777216 # increase the number of memory map areas for the server process vm.max_map_count=131060 # swap only to avoid an out of memory condition vm.swappiness=0 # try to always keep 64MB of RAM free vm.min_free_kbytes=65536 # prevent flooding detection when receiving large number of SYN packets # use with care as they can detec DoS attacks # needed with artificial loads while running the benchmark net.ipv4.tcp_max_tw_buckets=360000 net.core.somaxconn=4096 net.ipv4.tcp_max_syn_backlog=16000 net.ipv4.tcp_syncookies=0 net.ipv4.tcp_tw_recycle=1 # insure keepalive is enabled - use only if needed net.ipv4.tcp_keepalive_time=900 net.ipv4.tcp_keepalive_intvl=180 net.ipv4.tcp_keepalive_probes=5 And applied them with sudo sysctl -p /etc/sysctl.conf. Finally, in my Erlang release I've set into vm.args: +K true +P 2000000 +Q 1000000 +A 10 When I run my release and attach to the node, I can see: 1> erlang:system_info(check_io). [{name,erts_poll}, {primary,epoll}, {fallback,poll}, {kernel_poll,epoll}, {memory_size,113328}, {total_poll_set_size,1243}, {fallback_poll_set_size,0}, {lazy_updates,true}, {pending_updates,0}, {batch_updates,false}, {concurrent_updates,true}, {max_fds,999999}, {active_fds,1}] After all this I cannot connect more than 1024 outgoing HTTPS clients. Any ideas? Thank you, r. [1] http://erlang.org/doc/man/inet.html#id132028 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Wed Jan 7 16:26:42 2015 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Wed, 7 Jan 2015 16:26:42 +0100 Subject: [erlang-questions] All possible internal states of Erlang/OTP random module are practically computable In-Reply-To: References: <20141223004648.GA34572@k2r.org> Message-ID: <54AD5032.2070909@erix.ericsson.se> On 12/23/2014 03:56 PM, Jesper Louis Andersen wrote: > 3. ... The 'strong_rand_bytes/1' function can > return 'low_entropy' which is outright wrong and preposterous on modern > machines. It never will, if the underlying random primitive is correctly > implemented. The whole idea of "running out of entropy" is false. > 'low_entropy' from strong_rand_bytes/1 is a direct mapping to RAND_bytes returning error, which OpenSSL docs say can happen "... if the PRNG has not been seeded with enough randomness to ensure an unpredictable byte sequence." And the Linux man page for /dev/random and /dev/urandom says things like: "When the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered.". So, it seems to me that "running out of entropy" is at least a valid concept. And doing "cat /dev/random" on my Linux machine sure do block after a few hundred bytes of output. /Sverker From lostcolony@REDACTED Wed Jan 7 17:00:42 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Wed, 7 Jan 2015 11:00:42 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: <20150107152036.GA27201@brk> References: <20150106011106.GA31238@brk> <20150107152036.GA27201@brk> Message-ID: Thanks for that. Yeah, I had a look at your tuntap library, and it looks like both a portable way to reimplement the existing functionality, but also some ways to add some features more easily that I intended anyway, and some I had mentally dismissed as too unpleasant. I just need to find time to play with it and confirm/evolve my mental model. I'm thinking I'm going to try dropping one more feature on the existing code base, then start experimenting on a branch. There's some complexity in managing multiple destinations from one source, as well as the actual implementation of drop/delay (I need to make it so queued packets don't chain their delay. I.e., if I changed the code in that example to have a solid 1 second delay, for every frame coming in, and I got 5 frames in rapid succession, I'd have one received at the destination after one second, the next after two, the next after three, etc. I need to track when I received them and apply rules appropriately, across multiple nodes) On Wed, Jan 7, 2015 at 10:20 AM, Michael Santos wrote: > On Mon, Jan 05, 2015 at 08:55:06PM -0500, Christopher Phillips wrote: > > Hmm, I've had so little experience with tuntap interfaces I didn't > think > > of them. That sounds like the way to go to get things working on OSX, > > though at a much larger amount of effort (both the learning curve on my > > part, and re-implementing the degradation behaviors I get for free from > the > > kernel). Thanks for suggesting it; once I get things to where it's usable > > on Linux in all the ways I want, some research and a rewrite may be in > > order to get it fully portable. > > Sure, that is definitely reasonable. > > Working with tuntap devices isn't too difficult. For comparison, here is > a very simple example that forwards ethernet frames over erlang > distribution and randomly drops or delays a percentage of the frames: > > https://gist.github.com/msantos/f2823fcba40975003dc3 > > Set up 2 distributed nodes: > > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > > Then create the tunnel: > > % Drop 10% of the frames, randomly delay 40% up to 1 second > wastrel:start('n@REDACTED', "10.1.1.1", "10.1.1.2", 0.1, 0.4). > > Try to login using ssh (or whatever)! > > ssh 10.1.1.2 > > Meh, I've had to use worse ... > > Notes: > * relies on this erlang tuntap driver > > https://github.com/msantos/tunctl > > * beam needs to be running as root or have CAP_NET_ADMIN set: > > sudo setcap cap_net_admin=ep /usr/local/lib/erlang/erts-*/bin/beam* > > > On Mon, Jan 5, 2015 at 8:11 PM, Michael Santos > > > wrote: > > > > > On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips wrote: > > > > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've > done a > > > > bit of looking at the replacement, pf, and it looks like dropping > packets > > > > based on percentage is doable, as is bandwidth throttling (something > I'd > > > > like to add, in general), but I don't see any way to induce a delay, > > > beyond > > > > an implicit one based on tos prioritization. If someone knows how > and can > > > > point me in the right direction I'd appreciate it. > > > > > > The portable way is to use a tuntap device. Then you can arbitrarily > > > drop packets, throttle bandwidth, introduce latency, whatever, from > your > > > code. Sort of like quickcheck for networks :) > > > > > > > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko < > sergej.jurecko@REDACTED > > > > > > > > wrote: > > > > > > > > > This looks like a great tool and something that could easily be > added > > > to > > > > > unit tests. > > > > > Anyone with ipfw skills to add bsd/osx support? > > > > > > > > > > Sergej > > > > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" < > lostcolony@REDACTED> > > > > > wrote: > > > > > > > > > >> https://github.com/lostcolony/damocles > > > > >> > > > > >> I asked a while back on this mailing list if anyone had any useful > > > > >> libraries or similar for testing distribution scenarios. I only > got > > > back a > > > > >> few responses (maybe co-op riak_test? Maybe make use of the > underlying > > > > >> Linux traffic control and network emulation apps?), and my own > > > searches, > > > > >> while finding a few libraries, didn't find anything I could easily > > > co-op > > > > >> for my purposes. > > > > >> > > > > >> To that end, I went ahead and spent part of my break on this, and > it > > > just > > > > >> got sufficiently feature complete to throw out there. I haven't > had a > > > > >> chance to really start using it heavily, and I've only been > testing > > > it on > > > > >> my dev box, but a basic run through of the functionality as I > typed > > > up the > > > > >> readme worked (so any issues being pointed out would be > appreciated). > > > > >> Essentially, it allows you to create and manipulate local > interfaces > > > on a > > > > >> Linux machine to emulate packet delay and loss (using the > underlying > > > > >> traffic control and network emulation mechanisms), with a number > of > > > > >> convenience methods to (hopefully) easily describe fairly > intricate > > > > >> distribution scenarios. > > > > >> > > > > >> Things like "create these 5 interfaces, (now from my test code, > > > launch a > > > > >> copy of my app on each one, or even a different app on one of > them, > > > to see > > > > >> what happens when that resource is flaky); now make it so 1 and 2 > > > can't > > > > >> talk to 3 and 4, and vice versa, but everyone can still talk to > 5, but > > > > >> replies have a 50% chance of being dropped from 5 when responding > to > > > 1 and > > > > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run > more of > > > our > > > > >> test code to assert that trying to write to any node still > succeeds); > > > okay, > > > > >> now let's restore the network back to normal (and have our test > code > > > make > > > > >> sure the write was retained)", or whatever, can be set up in a > > > > >> straightforward, automated manner as part of a common test run, > and > > > not be > > > > >> reliant on certain VMs being up, nor the tests being run on a > specific > > > > >> network. The tradeoff, obviously, being that you can't really load > > > test > > > > >> things with it. Still, it fits my basic needs, and I figured it > might > > > be of > > > > >> use to others. > > > > >> > > > > >> I'll be adding some simple examples when I next get free time (I > ran > > > out > > > > >> of it from the holiday break without getting to them; dunno when I > > > will), > > > > >> and will try and get to any bugs or simple suggestions in a timely > > > manner, > > > > >> but hopefully it's fairly straightforward and useful as is. > > > > >> > > > > >> _______________________________________________ > > > > >> erlang-questions mailing list > > > > >> erlang-questions@REDACTED > > > > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > >> > > > > >> > > > > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Wed Jan 7 17:20:05 2015 From: jon@REDACTED (Jon Schneider) Date: Wed, 7 Jan 2015 16:20:05 -0000 Subject: [erlang-questions] {error,emfile} In-Reply-To: References: Message-ID: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> Could it be due to FD_SETSIZE ? Even if Erlang itself might be able to handle many more sockets there are still things constrained by this. Jon > Dear all, > I've built a benchmarking tool and I experience an error when connecting > more than 1024 clients to a SSL server. The call to ssl:connect/3 returns > a > {error,emfile} instead of a Socket. > > According to inet POSIX Error Codes [1], emfile corresponds to too many > open files. > I'm running from a UBUNTU 14.04 LTS box. > > However, I've set my /etc/security/limits.conf with: > > * soft nofile 999999 > * hard nofile 999999 > > and rebooted the machine. I can see it applied: > > $ ulimit -a > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 241197 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 999999 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 241197 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > > > I've also set /etc/sysctl.conf with: > > # number of file descriptors > fs.file-max=2001000 > > # extend the ports range > net.ipv4.ip_local_port_range=1024 65535 > > # increase the max number of receive and send buffer size > net.core.rmem_max=33554432 > net.core.wmem_max=33554432 > > # increase TCP auto-tuning buffer limits settings > net.ipv4.tcp_rmem=4096 16384 33554432 > net.ipv4.tcp_wmem=4096 16384 33554432 > net.ipv4.tcp_mem=786432 1048576 26777216 > > # increase the number of memory map areas for the server process > vm.max_map_count=131060 > > # swap only to avoid an out of memory condition > vm.swappiness=0 > > # try to always keep 64MB of RAM free > vm.min_free_kbytes=65536 > > # prevent flooding detection when receiving large number of SYN packets > # use with care as they can detec DoS attacks > # needed with artificial loads while running the benchmark > net.ipv4.tcp_max_tw_buckets=360000 > net.core.somaxconn=4096 > net.ipv4.tcp_max_syn_backlog=16000 > net.ipv4.tcp_syncookies=0 > net.ipv4.tcp_tw_recycle=1 > > # insure keepalive is enabled - use only if needed > net.ipv4.tcp_keepalive_time=900 > net.ipv4.tcp_keepalive_intvl=180 > net.ipv4.tcp_keepalive_probes=5 > > And applied them with sudo sysctl -p /etc/sysctl.conf. > > Finally, in my Erlang release I've set into vm.args: > > +K true > +P 2000000 > +Q 1000000 > +A 10 > > When I run my release and attach to the node, I can see: > > 1> erlang:system_info(check_io). > [{name,erts_poll}, > {primary,epoll}, > {fallback,poll}, > {kernel_poll,epoll}, > {memory_size,113328}, > {total_poll_set_size,1243}, > {fallback_poll_set_size,0}, > {lazy_updates,true}, > {pending_updates,0}, > {batch_updates,false}, > {concurrent_updates,true}, > {max_fds,999999}, > {active_fds,1}] > > After all this I cannot connect more than 1024 outgoing HTTPS clients. > > Any ideas? > > Thank you, > r. > > > > [1] http://erlang.org/doc/man/inet.html#id132028 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From stephane.pamelard@REDACTED Wed Jan 7 17:27:21 2015 From: stephane.pamelard@REDACTED (Pamelard Stephane) Date: Wed, 7 Jan 2015 17:27:21 +0100 Subject: [erlang-questions] {error,emfile} In-Reply-To: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> Message-ID: <54AD5E69.4020106@myriadgroup.com> Hi Roberto, Have you tried to increase ERL_MAX_PORTS env variable value? Its default value is 1024. Hope that could help. BR, St?phane Le 07/01/2015 17:20, Jon Schneider a ?crit : > Could it be due to FD_SETSIZE ? Even if Erlang itself might be able to > handle many more sockets there are still things constrained by this. > > Jon > >> Dear all, >> I've built a benchmarking tool and I experience an error when connecting >> more than 1024 clients to a SSL server. The call to ssl:connect/3 returns >> a >> {error,emfile} instead of a Socket. >> >> According to inet POSIX Error Codes [1], emfile corresponds to too many >> open files. >> I'm running from a UBUNTU 14.04 LTS box. >> >> However, I've set my /etc/security/limits.conf with: >> >> * soft nofile 999999 >> * hard nofile 999999 >> >> and rebooted the machine. I can see it applied: >> >> $ ulimit -a >> core file size (blocks, -c) 0 >> data seg size (kbytes, -d) unlimited >> scheduling priority (-e) 0 >> file size (blocks, -f) unlimited >> pending signals (-i) 241197 >> max locked memory (kbytes, -l) 64 >> max memory size (kbytes, -m) unlimited >> open files (-n) 999999 >> pipe size (512 bytes, -p) 8 >> POSIX message queues (bytes, -q) 819200 >> real-time priority (-r) 0 >> stack size (kbytes, -s) 8192 >> cpu time (seconds, -t) unlimited >> max user processes (-u) 241197 >> virtual memory (kbytes, -v) unlimited >> file locks (-x) unlimited >> >> >> I've also set /etc/sysctl.conf with: >> >> # number of file descriptors >> fs.file-max=2001000 >> >> # extend the ports range >> net.ipv4.ip_local_port_range=1024 65535 >> >> # increase the max number of receive and send buffer size >> net.core.rmem_max=33554432 >> net.core.wmem_max=33554432 >> >> # increase TCP auto-tuning buffer limits settings >> net.ipv4.tcp_rmem=4096 16384 33554432 >> net.ipv4.tcp_wmem=4096 16384 33554432 >> net.ipv4.tcp_mem=786432 1048576 26777216 >> >> # increase the number of memory map areas for the server process >> vm.max_map_count=131060 >> >> # swap only to avoid an out of memory condition >> vm.swappiness=0 >> >> # try to always keep 64MB of RAM free >> vm.min_free_kbytes=65536 >> >> # prevent flooding detection when receiving large number of SYN packets >> # use with care as they can detec DoS attacks >> # needed with artificial loads while running the benchmark >> net.ipv4.tcp_max_tw_buckets=360000 >> net.core.somaxconn=4096 >> net.ipv4.tcp_max_syn_backlog=16000 >> net.ipv4.tcp_syncookies=0 >> net.ipv4.tcp_tw_recycle=1 >> >> # insure keepalive is enabled - use only if needed >> net.ipv4.tcp_keepalive_time=900 >> net.ipv4.tcp_keepalive_intvl=180 >> net.ipv4.tcp_keepalive_probes=5 >> >> And applied them with sudo sysctl -p /etc/sysctl.conf. >> >> Finally, in my Erlang release I've set into vm.args: >> >> +K true >> +P 2000000 >> +Q 1000000 >> +A 10 >> >> When I run my release and attach to the node, I can see: >> >> 1> erlang:system_info(check_io). >> [{name,erts_poll}, >> {primary,epoll}, >> {fallback,poll}, >> {kernel_poll,epoll}, >> {memory_size,113328}, >> {total_poll_set_size,1243}, >> {fallback_poll_set_size,0}, >> {lazy_updates,true}, >> {pending_updates,0}, >> {batch_updates,false}, >> {concurrent_updates,true}, >> {max_fds,999999}, >> {active_fds,1}] >> >> After all this I cannot connect more than 1024 outgoing HTTPS clients. >> >> Any ideas? >> >> Thank you, >> r. >> >> >> >> [1] http://erlang.org/doc/man/inet.html#id132028 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions *** DISCLAIMER *** This message, including attachments, is intended solely for the addressee indicated in this message and is strictly confidential or otherwise privileged. If you are not the intended recipient (or responsible for delivery of the message to such person) : - (1) please immediately (i) notify the sender by reply email and (ii) delete this message and attachments, - (2) any use, copy or dissemination of this transmission is strictly prohibited. If you or your employer does not consent to Internet email messages of this kind, please advise Myriad Group AG by reply e-mail immediately. Opinions, conclusions and other information expressed in this message are not given or endorsed by Myriad Group AG unless otherwise indicated by an authorized representative independent of this message. From egil@REDACTED Wed Jan 7 17:55:10 2015 From: egil@REDACTED (=?windows-1252?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 7 Jan 2015 17:55:10 +0100 Subject: [erlang-questions] Maps in typespecs (and dialyzer) In-Reply-To: References: Message-ID: <54AD64EE.5040202@erlang.org> On 2015-01-05 23:43, Jos? Valim wrote: > Hello everyone, > > The typespecs > documentation > says dialyzer does not yet use map pairs information (which is fine). > The issue is that it does not describe what are the semantic > implications of map pairs in types. To be clear, the following type: > > #{foo => any()} > > > should: > > 1) match maps containing only the key foo > 2) match maps containing at least the key foo We've had some debate on this. Especially regarding mandatory and/or optional fields. From my viewpoint your second point is the correct one. You must specify the fields and types that are needed for your function to succeed. I don't think it should be an error to over specify the fields (more keys than needed). Perhaps you want to construct a -type with a map with several fields but different fields are used in different functions where you specify that you are using this type. > Additionally, what would #{_ => _} mean in any of the schemas above? This is essentially what we are saying today .. though i would refrain from specifying it with underscore, #{ any() => any() } is a bit clearer. Our contract says we accept a map containing any keys with any values, which is not very useful but valid. The jury is still out on empty map though. // Bj?rn-Egil > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Lead Developer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed Jan 7 18:05:48 2015 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 7 Jan 2015 18:05:48 +0100 Subject: [erlang-questions] Maps in typespecs (and dialyzer) In-Reply-To: <54AD64EE.5040202@erlang.org> References: <54AD64EE.5040202@erlang.org> Message-ID: Thanks for the reply. I agree that 2) is what makes most sense with maps usage too. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Jan 7 18:42:07 2015 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 7 Jan 2015 12:42:07 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: References: <20150106011106.GA31238@brk> <20150107152036.GA27201@brk> Message-ID: <20150107174207.GB27201@brk> On Wed, Jan 07, 2015 at 11:00:42AM -0500, Christopher Phillips wrote: > Thanks for that. Yeah, I had a look at your tuntap library, and it looks > like both a portable way to reimplement the existing functionality, but > also some ways to add some features more easily that I intended anyway, and > some I had mentally dismissed as too unpleasant. I just need to find time > to play with it and confirm/evolve my mental model. I'm thinking I'm going > to try dropping one more feature on the existing code base, then start > experimenting on a branch. Yeah, just a suggestion, not sure it will work out for you. By the way, very cool project! > There's some complexity in managing multiple > destinations from one source, as well as the actual implementation of > drop/delay (I need to make it so queued packets don't chain their delay. > I.e., if I changed the code in that example to have a solid 1 second delay, > for every frame coming in, and I got 5 frames in rapid succession, I'd have > one received at the destination after one second, the next after two, the > next after three, etc. I need to track when I received them and apply rules > appropriately, across multiple nodes) One way would be to have a number of buckets for holding frames. Each bucket is an erlang process. Frames are distributed across the processes and if the mailbox queue exceeds some size, new frames are discarded. > On Wed, Jan 7, 2015 at 10:20 AM, Michael Santos > wrote: > > > On Mon, Jan 05, 2015 at 08:55:06PM -0500, Christopher Phillips wrote: > > > Hmm, I've had so little experience with tuntap interfaces I didn't > > think > > > of them. That sounds like the way to go to get things working on OSX, > > > though at a much larger amount of effort (both the learning curve on my > > > part, and re-implementing the degradation behaviors I get for free from > > the > > > kernel). Thanks for suggesting it; once I get things to where it's usable > > > on Linux in all the ways I want, some research and a rewrite may be in > > > order to get it fully portable. > > > > Sure, that is definitely reasonable. > > > > Working with tuntap devices isn't too difficult. For comparison, here is > > a very simple example that forwards ethernet frames over erlang > > distribution and randomly drops or delays a percentage of the frames: > > > > https://gist.github.com/msantos/f2823fcba40975003dc3 > > > > Set up 2 distributed nodes: > > > > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > > > > Then create the tunnel: > > > > % Drop 10% of the frames, randomly delay 40% up to 1 second > > wastrel:start('n@REDACTED', "10.1.1.1", "10.1.1.2", 0.1, 0.4). > > > > Try to login using ssh (or whatever)! > > > > ssh 10.1.1.2 > > > > Meh, I've had to use worse ... > > > > Notes: > > * relies on this erlang tuntap driver > > > > https://github.com/msantos/tunctl > > > > * beam needs to be running as root or have CAP_NET_ADMIN set: > > > > sudo setcap cap_net_admin=ep /usr/local/lib/erlang/erts-*/bin/beam* > > > > > On Mon, Jan 5, 2015 at 8:11 PM, Michael Santos > > > > > wrote: > > > > > > > On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips wrote: > > > > > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've > > done a > > > > > bit of looking at the replacement, pf, and it looks like dropping > > packets > > > > > based on percentage is doable, as is bandwidth throttling (something > > I'd > > > > > like to add, in general), but I don't see any way to induce a delay, > > > > beyond > > > > > an implicit one based on tos prioritization. If someone knows how > > and can > > > > > point me in the right direction I'd appreciate it. > > > > > > > > The portable way is to use a tuntap device. Then you can arbitrarily > > > > drop packets, throttle bandwidth, introduce latency, whatever, from > > your > > > > code. Sort of like quickcheck for networks :) > > > > > > > > > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko < > > sergej.jurecko@REDACTED > > > > > > > > > > wrote: > > > > > > > > > > > This looks like a great tool and something that could easily be > > added > > > > to > > > > > > unit tests. > > > > > > Anyone with ipfw skills to add bsd/osx support? > > > > > > > > > > > > Sergej > > > > > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" < > > lostcolony@REDACTED> > > > > > > wrote: > > > > > > > > > > > >> https://github.com/lostcolony/damocles > > > > > >> > > > > > >> I asked a while back on this mailing list if anyone had any useful > > > > > >> libraries or similar for testing distribution scenarios. I only > > got > > > > back a > > > > > >> few responses (maybe co-op riak_test? Maybe make use of the > > underlying > > > > > >> Linux traffic control and network emulation apps?), and my own > > > > searches, > > > > > >> while finding a few libraries, didn't find anything I could easily > > > > co-op > > > > > >> for my purposes. > > > > > >> > > > > > >> To that end, I went ahead and spent part of my break on this, and > > it > > > > just > > > > > >> got sufficiently feature complete to throw out there. I haven't > > had a > > > > > >> chance to really start using it heavily, and I've only been > > testing > > > > it on > > > > > >> my dev box, but a basic run through of the functionality as I > > typed > > > > up the > > > > > >> readme worked (so any issues being pointed out would be > > appreciated). > > > > > >> Essentially, it allows you to create and manipulate local > > interfaces > > > > on a > > > > > >> Linux machine to emulate packet delay and loss (using the > > underlying > > > > > >> traffic control and network emulation mechanisms), with a number > > of > > > > > >> convenience methods to (hopefully) easily describe fairly > > intricate > > > > > >> distribution scenarios. > > > > > >> > > > > > >> Things like "create these 5 interfaces, (now from my test code, > > > > launch a > > > > > >> copy of my app on each one, or even a different app on one of > > them, > > > > to see > > > > > >> what happens when that resource is flaky); now make it so 1 and 2 > > > > can't > > > > > >> talk to 3 and 4, and vice versa, but everyone can still talk to > > 5, but > > > > > >> replies have a 50% chance of being dropped from 5 when responding > > to > > > > 1 and > > > > > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run > > more of > > > > our > > > > > >> test code to assert that trying to write to any node still > > succeeds); > > > > okay, > > > > > >> now let's restore the network back to normal (and have our test > > code > > > > make > > > > > >> sure the write was retained)", or whatever, can be set up in a > > > > > >> straightforward, automated manner as part of a common test run, > > and > > > > not be > > > > > >> reliant on certain VMs being up, nor the tests being run on a > > specific > > > > > >> network. The tradeoff, obviously, being that you can't really load > > > > test > > > > > >> things with it. Still, it fits my basic needs, and I figured it > > might > > > > be of > > > > > >> use to others. > > > > > >> > > > > > >> I'll be adding some simple examples when I next get free time (I > > ran > > > > out > > > > > >> of it from the holiday break without getting to them; dunno when I > > > > will), > > > > > >> and will try and get to any bugs or simple suggestions in a timely > > > > manner, > > > > > >> but hopefully it's fairly straightforward and useful as is. > > > > > >> > > > > > >> _______________________________________________ > > > > > >> erlang-questions mailing list > > > > > >> erlang-questions@REDACTED > > > > > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > >> > > > > > >> > > > > > > > > > _______________________________________________ > > > > > erlang-questions mailing list > > > > > erlang-questions@REDACTED > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > From lostcolony@REDACTED Wed Jan 7 19:28:45 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Wed, 7 Jan 2015 13:28:45 -0500 Subject: [erlang-questions] [ANN]: Damocles, a library for testing distribution scenarios on a single machine In-Reply-To: <20150107174207.GB27201@brk> References: <20150106011106.GA31238@brk> <20150107152036.GA27201@brk> <20150107174207.GB27201@brk> Message-ID: Yeah, I've got a few ideas on how I could do it. Just gonna take a lot of playing and testing. Thanks again for bringing up tuntap though (and for writing a library to make using them easier ), it's opened up some new possibilities for when I've got time (one more feature against the current code base and it does everything I want it to for my current project, but rewriting to leverage tuns or taps would let me subsume all the related functionalities I've seen/thought of in some manner or other, in a portable manner, which is...really, really cool). On Wed, Jan 7, 2015 at 12:42 PM, Michael Santos wrote: > On Wed, Jan 07, 2015 at 11:00:42AM -0500, Christopher Phillips wrote: > > Thanks for that. Yeah, I had a look at your tuntap library, and it > looks > > like both a portable way to reimplement the existing functionality, but > > also some ways to add some features more easily that I intended anyway, > and > > some I had mentally dismissed as too unpleasant. I just need to find time > > to play with it and confirm/evolve my mental model. I'm thinking I'm > going > > to try dropping one more feature on the existing code base, then start > > experimenting on a branch. > > Yeah, just a suggestion, not sure it will work out for you. By the way, > very cool project! > > > There's some complexity in managing multiple > > destinations from one source, as well as the actual implementation of > > drop/delay (I need to make it so queued packets don't chain their delay. > > I.e., if I changed the code in that example to have a solid 1 second > delay, > > for every frame coming in, and I got 5 frames in rapid succession, I'd > have > > one received at the destination after one second, the next after two, the > > next after three, etc. I need to track when I received them and apply > rules > > appropriately, across multiple nodes) > > One way would be to have a number of buckets for holding frames. Each > bucket is an erlang process. Frames are distributed across the processes > and if the mailbox queue exceeds some size, new frames are discarded. > > > On Wed, Jan 7, 2015 at 10:20 AM, Michael Santos < > michael.santos@REDACTED> > > wrote: > > > > > On Mon, Jan 05, 2015 at 08:55:06PM -0500, Christopher Phillips wrote: > > > > Hmm, I've had so little experience with tuntap interfaces I didn't > > > think > > > > of them. That sounds like the way to go to get things working on OSX, > > > > though at a much larger amount of effort (both the learning curve on > my > > > > part, and re-implementing the degradation behaviors I get for free > from > > > the > > > > kernel). Thanks for suggesting it; once I get things to where it's > usable > > > > on Linux in all the ways I want, some research and a rewrite may be > in > > > > order to get it fully portable. > > > > > > Sure, that is definitely reasonable. > > > > > > Working with tuntap devices isn't too difficult. For comparison, here > is > > > a very simple example that forwards ethernet frames over erlang > > > distribution and randomly drops or delays a percentage of the frames: > > > > > > https://gist.github.com/msantos/f2823fcba40975003dc3 > > > > > > Set up 2 distributed nodes: > > > > > > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > > > erl -name n@REDACTED -setcookie CHOCOLATEBUTTERSCOTCH > > > > > > Then create the tunnel: > > > > > > % Drop 10% of the frames, randomly delay 40% up to 1 second > > > wastrel:start('n@REDACTED', "10.1.1.1", "10.1.1.2", 0.1, 0.4). > > > > > > Try to login using ssh (or whatever)! > > > > > > ssh 10.1.1.2 > > > > > > Meh, I've had to use worse ... > > > > > > Notes: > > > * relies on this erlang tuntap driver > > > > > > https://github.com/msantos/tunctl > > > > > > * beam needs to be running as root or have CAP_NET_ADMIN set: > > > > > > sudo setcap cap_net_admin=ep /usr/local/lib/erlang/erts-*/bin/beam* > > > > > > > On Mon, Jan 5, 2015 at 8:11 PM, Michael Santos < > michael.santos@REDACTED > > > > > > > > wrote: > > > > > > > > > On Mon, Jan 05, 2015 at 09:16:42AM -0500, Christopher Phillips > wrote: > > > > > > For OSX ipfw was deprecated in Lion and removed in Yosemite. I've > > > done a > > > > > > bit of looking at the replacement, pf, and it looks like dropping > > > packets > > > > > > based on percentage is doable, as is bandwidth throttling > (something > > > I'd > > > > > > like to add, in general), but I don't see any way to induce a > delay, > > > > > beyond > > > > > > an implicit one based on tos prioritization. If someone knows how > > > and can > > > > > > point me in the right direction I'd appreciate it. > > > > > > > > > > The portable way is to use a tuntap device. Then you can > arbitrarily > > > > > drop packets, throttle bandwidth, introduce latency, whatever, from > > > your > > > > > code. Sort of like quickcheck for networks :) > > > > > > > > > > > On Mon, Jan 5, 2015 at 6:52 AM, Sergej Jure?ko < > > > sergej.jurecko@REDACTED > > > > > > > > > > > > wrote: > > > > > > > > > > > > > This looks like a great tool and something that could easily be > > > added > > > > > to > > > > > > > unit tests. > > > > > > > Anyone with ipfw skills to add bsd/osx support? > > > > > > > > > > > > > > Sergej > > > > > > > On Jan 5, 2015 1:41 AM, "Christopher Phillips" < > > > lostcolony@REDACTED> > > > > > > > wrote: > > > > > > > > > > > > > >> https://github.com/lostcolony/damocles > > > > > > >> > > > > > > >> I asked a while back on this mailing list if anyone had any > useful > > > > > > >> libraries or similar for testing distribution scenarios. I > only > > > got > > > > > back a > > > > > > >> few responses (maybe co-op riak_test? Maybe make use of the > > > underlying > > > > > > >> Linux traffic control and network emulation apps?), and my own > > > > > searches, > > > > > > >> while finding a few libraries, didn't find anything I could > easily > > > > > co-op > > > > > > >> for my purposes. > > > > > > >> > > > > > > >> To that end, I went ahead and spent part of my break on this, > and > > > it > > > > > just > > > > > > >> got sufficiently feature complete to throw out there. I > haven't > > > had a > > > > > > >> chance to really start using it heavily, and I've only been > > > testing > > > > > it on > > > > > > >> my dev box, but a basic run through of the functionality as I > > > typed > > > > > up the > > > > > > >> readme worked (so any issues being pointed out would be > > > appreciated). > > > > > > >> Essentially, it allows you to create and manipulate local > > > interfaces > > > > > on a > > > > > > >> Linux machine to emulate packet delay and loss (using the > > > underlying > > > > > > >> traffic control and network emulation mechanisms), with a > number > > > of > > > > > > >> convenience methods to (hopefully) easily describe fairly > > > intricate > > > > > > >> distribution scenarios. > > > > > > >> > > > > > > >> Things like "create these 5 interfaces, (now from my test > code, > > > > > launch a > > > > > > >> copy of my app on each one, or even a different app on one of > > > them, > > > > > to see > > > > > > >> what happens when that resource is flaky); now make it so 1 > and 2 > > > > > can't > > > > > > >> talk to 3 and 4, and vice versa, but everyone can still talk > to > > > 5, but > > > > > > >> replies have a 50% chance of being dropped from 5 when > responding > > > to > > > > > 1 and > > > > > > >> 2, and there's a 300ms delay between 3 and 4; (now, let's run > > > more of > > > > > our > > > > > > >> test code to assert that trying to write to any node still > > > succeeds); > > > > > okay, > > > > > > >> now let's restore the network back to normal (and have our > test > > > code > > > > > make > > > > > > >> sure the write was retained)", or whatever, can be set up in a > > > > > > >> straightforward, automated manner as part of a common test > run, > > > and > > > > > not be > > > > > > >> reliant on certain VMs being up, nor the tests being run on a > > > specific > > > > > > >> network. The tradeoff, obviously, being that you can't really > load > > > > > test > > > > > > >> things with it. Still, it fits my basic needs, and I figured > it > > > might > > > > > be of > > > > > > >> use to others. > > > > > > >> > > > > > > >> I'll be adding some simple examples when I next get free time > (I > > > ran > > > > > out > > > > > > >> of it from the holiday break without getting to them; dunno > when I > > > > > will), > > > > > > >> and will try and get to any bugs or simple suggestions in a > timely > > > > > manner, > > > > > > >> but hopefully it's fairly straightforward and useful as is. > > > > > > >> > > > > > > >> _______________________________________________ > > > > > > >> erlang-questions mailing list > > > > > > >> erlang-questions@REDACTED > > > > > > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > >> > > > > > > >> > > > > > > > > > > > _______________________________________________ > > > > > > erlang-questions mailing list > > > > > > erlang-questions@REDACTED > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Thu Jan 8 04:56:01 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Wed, 7 Jan 2015 19:56:01 -0800 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true Message-ID: The more formal definition of problem is Write a function map_search_pred(Map, Pred) that returns the first element {Key,Value} in the map for which Pred(Key, Value) is true. My attempt, looks like map_search_pred(Map, Pred) -> M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, Value) =:= true], case length(M) of 0 -> {}; _ -> lists:nth(1, M) end. When I run this, I get correct answer 1> lib_misc:map_search_pred(#{}, fun(X, Y) -> X =:= Y end). {} 2> lib_misc:map_search_pred(#{1 => 1, 2 => 2}, fun(X, Y) -> X =:= Y end). {1,1} 3> lib_misc:map_search_pred(#{1 => 1, 2 => 3}, fun(X, Y) -> X =:= Y end). {1,1} 4> lib_misc:map_search_pred(#{1 => 2, 2 => 2}, fun(X, Y) -> X =:= Y end). {2,2} 5> *Problem?* The problem is efficiency. It uses list comprehension and runs for all the elements in the list. The better solution would be to return after the first match. As a beginner to language, I do not know a better idiomatic way to solve this, so looking for better ideas and suggestions Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanfiedler@REDACTED Thu Jan 8 06:20:59 2015 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Wed, 7 Jan 2015 21:20:59 -0800 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: The maps module would be the first place to look, but it seems to lack a find(Pred, Map) function. And it doesn't look like proplists would be of any advantage, either. Nor does Jakob Sievers stdlib2 have anything helpful here. The best I can offer is to use [H|_T] = lists:filter(Pred, maps:to_list(Map)), which is shorter, but no more efficient than what you have so far. Hopefully someone more knowledgable can help. n On Wed, Jan 7, 2015 at 7:56 PM, Harit Himanshu wrote: > The more formal definition of problem is > > Write a function map_search_pred(Map, Pred) that returns the first element > {Key,Value} in the map for which Pred(Key, Value) is true. > > My attempt, looks like > > map_search_pred(Map, Pred) -> > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, Value) > =:= true], > case length(M) of > 0 -> {}; > _ -> lists:nth(1, M) > end. > > When I run this, I get correct answer > > 1> lib_misc:map_search_pred(#{}, fun(X, Y) -> X =:= Y end). > {} > 2> lib_misc:map_search_pred(#{1 => 1, 2 => 2}, fun(X, Y) -> X =:= Y end). > {1,1} > 3> lib_misc:map_search_pred(#{1 => 1, 2 => 3}, fun(X, Y) -> X =:= Y end). > {1,1} > 4> lib_misc:map_search_pred(#{1 => 2, 2 => 2}, fun(X, Y) -> X =:= Y end). > {2,2} > 5> > > Problem? > The problem is efficiency. > It uses list comprehension and runs for all the elements in the list. The > better solution would be to return after the first match. > > As a beginner to language, I do not know a better idiomatic way to solve > this, so looking for better ideas and suggestions > > Thank you > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From harit.subscriptions@REDACTED Thu Jan 8 06:32:51 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Wed, 7 Jan 2015 21:32:51 -0800 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: Thanks Nathan On Wed, Jan 7, 2015 at 9:20 PM, Nathan Fiedler wrote: > The maps module would be the first place to look, but it seems to lack > a find(Pred, Map) function. And it doesn't look like proplists would > be of any advantage, either. Nor does Jakob Sievers stdlib2 have > anything helpful here. The best I can offer is to use [H|_T] = > lists:filter(Pred, maps:to_list(Map)), which is shorter, but no more > efficient than what you have so far. > > Hopefully someone more knowledgable can help. > > n > > > On Wed, Jan 7, 2015 at 7:56 PM, Harit Himanshu > wrote: > > The more formal definition of problem is > > > > Write a function map_search_pred(Map, Pred) that returns the first > element > > {Key,Value} in the map for which Pred(Key, Value) is true. > > > > My attempt, looks like > > > > map_search_pred(Map, Pred) -> > > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, > Value) > > =:= true], > > case length(M) of > > 0 -> {}; > > _ -> lists:nth(1, M) > > end. > > > > When I run this, I get correct answer > > > > 1> lib_misc:map_search_pred(#{}, fun(X, Y) -> X =:= Y end). > > {} > > 2> lib_misc:map_search_pred(#{1 => 1, 2 => 2}, fun(X, Y) -> X =:= Y end). > > {1,1} > > 3> lib_misc:map_search_pred(#{1 => 1, 2 => 3}, fun(X, Y) -> X =:= Y end). > > {1,1} > > 4> lib_misc:map_search_pred(#{1 => 2, 2 => 2}, fun(X, Y) -> X =:= Y end). > > {2,2} > > 5> > > > > Problem? > > The problem is efficiency. > > It uses list comprehension and runs for all the elements in the list. The > > better solution would be to return after the first match. > > > > As a beginner to language, I do not know a better idiomatic way to solve > > this, so looking for better ideas and suggestions > > > > Thank you > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Jan 8 08:47:46 2015 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 8 Jan 2015 08:47:46 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: > On 08 Jan 2015, at 04:56, Harit Himanshu wrote: > > My attempt, looks like > > map_search_pred(Map, Pred) -> > M = [{Key, Value} || {Key, Value} <- maps:to_list (Map), Pred(Key, Value) =:= true], > case length(M) of > 0 -> {}; > _ -> lists:nth(1, M) > end. ... > Problem? > The problem is efficiency. > It uses list comprehension and runs for all the elements in the list. The better solution would be to return after the first match. > > As a beginner to language, I do not know a better idiomatic way to solve this, so looking for better ideas and suggestions > You can use map_search_pred(Map, Pred) -> case lists:dropwhile(fun(X) -> not Pred(X) end, maps:to_list (Map)) of [Found | _] -> Found; [] -> false end. A few comments. - Using {} as a return value denoting ?not found? is unusual. My version mimicks lists:keyfind/2, which returns the matching object or ?false?. - Testing the length of the list using length/1 will force a count of all elements. In this case, you are only interested in checking if the list is non-empty, which is better done with a pattern-match. This also eliminates the need for calling lists:nth(1,M), which is also a more expensive option than hd(M). BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 8 11:07:31 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 8 Jan 2015 11:07:31 +0100 Subject: [erlang-questions] {error,emfile} In-Reply-To: <54AD5E69.4020106@myriadgroup.com> References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> <54AD5E69.4020106@myriadgroup.com> Message-ID: ERL_MAX_PORTS is deprecated, and has been substituted by the +Q option in vm.args. Thanks, r. On Wed, Jan 7, 2015 at 5:27 PM, Pamelard Stephane < stephane.pamelard@REDACTED> wrote: > Hi Roberto, > > Have you tried to increase ERL_MAX_PORTS env variable value? Its default > value is 1024. > Hope that could help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 8 11:10:51 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 8 Jan 2015 11:10:51 +0100 Subject: [erlang-questions] {error,emfile} In-Reply-To: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> Message-ID: It looks like the problem is related to the fact that I'm using Ubuntu's upstart to launch the Erlang process. Apparently, changing the file descriptors limit via ulimit or by /etc/security/limits.conf has no effect. Programs that launch on startup via ?upstart? do not get these limits applied to them: http://bryanmarty.com/2012/02/10/setting-nofile-limit-upstart/ The solution is to use the "limit" stanza in the script. Hope this helps anyone else with the same matter. Best, r. On Wed, Jan 7, 2015 at 5:20 PM, Jon Schneider wrote: > Could it be due to FD_SETSIZE ? Even if Erlang itself might be able to > handle many more sockets there are still things constrained by this. > > Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Jan 8 11:42:26 2015 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 8 Jan 2015 11:42:26 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: Reply in http://joearms.github.io/2015/01/08/Some_Performance-Measurements-On-Maps.html /Joe On Thu, Jan 8, 2015 at 8:47 AM, Ulf Wiger wrote: > > On 08 Jan 2015, at 04:56, Harit Himanshu > wrote: > > My attempt, looks like > > map_search_pred(Map, Pred) -> > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, Value) > =:= true], > case length(M) of > 0 -> {}; > _ -> lists:nth(1, M) > end. > > ... > > Problem? > The problem is efficiency. > It uses list comprehension and runs for all the elements in the list. The > better solution would be to return after the first match. > > As a beginner to language, I do not know a better idiomatic way to solve > this, so looking for better ideas and suggestions > > > You can use > > map_search_pred(Map, Pred) -> > case lists:dropwhile(fun(X) -> not Pred(X) end, maps:to_list(Map)) of > [Found | _] -> Found; > [] -> false > end. > > A few comments. > > - Using {} as a return value denoting ?not found? is unusual. My version > mimicks lists:keyfind/2, which returns the matching object or ?false?. > > - Testing the length of the list using length/1 will force a count of all > elements. In this case, you are only interested in checking if the list is > non-empty, which is better done with a pattern-match. This also eliminates > the need for calling lists:nth(1,M), which is also a more expensive option > than hd(M). > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From imantc@REDACTED Thu Jan 8 12:03:06 2015 From: imantc@REDACTED (Imants Cekusins) Date: Thu, 8 Jan 2015 12:03:06 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: Let's change the predicate so it does something with the found key-value pair (what would you do with the match when you found it?). Then you could maps:to_list and pass this side-effect predicate to lists:any. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jan 8 11:21:07 2015 From: bob@REDACTED (Robert Wilkinson) Date: Thu, 8 Jan 2015 11:21:07 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: <20150108102107.GB12145@fourtheye.org> On Wed, Jan 07, 2015 at 07:56:01PM -0800, Harit Himanshu wrote: > The more formal definition of problem is > > Write a function map_search_pred(Map, Pred) that returns the first element > {Key,Value} in the map for which Pred(Key, Value) is true. SNIP > *Problem?* > The problem is efficiency. > It uses list comprehension and runs for all the elements in the list. The > better solution would be to return after the first match. > > As a beginner to language, I do not know a better idiomatic way to solve > this, so looking for better ideas and suggestions > > Thank you Hello Harit I have some problem with your intent. A map is not ordered (afaik - or it will/may be implementation dependent) and so the concept of *first* is misguided? (it may depend on external factors) Maybe the function should return when *any* key/value pair satisfies the predicate. Sorry if I am being over pedantic! Bob From jesper.louis.andersen@REDACTED Thu Jan 8 13:45:26 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 08 Jan 2015 12:45:26 +0000 Subject: [erlang-questions] All possible internal states of Erlang/OTP random module are practically computable References: <20141223004648.GA34572@k2r.org> <54AD5032.2070909@erix.ericsson.se> Message-ID: Note the caveat: "if the underlying primitive is correctly implemented". FreeBSD dragon.lan 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014 root@REDACTED:/usr/obj/usr/src/sys/GENERIC amd64 [jlouis@REDACTED ~]$ ls -l /dev/*random crw-rw-rw- 1 root wheel 0xc Dec 25 17:28 /dev/random lrwxr-xr-x 1 root wheel 6 Dec 25 17:28 /dev/urandom -> random On FreeBSD, the randomness refers to the same source for a reason. Because there is no way you can "run out of random entropy". As such, there are no difference between "strong" and "normal" random bytes at all. Both sources are backed by the Yarrow generator. On Linux, you can, but Linux presents an appallingly bad interface :) On OpenBSD, you have an even better option: http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2 which lets you source entropy to use for your own CSPRNG in the user space of the process (OpenBSD uses a solution based on chacha20 currently). It has the advantage that you can't attack it with file descriptor exhaustion and other such shenanigans since it is a syscall. It was so useful, the Linux guys have added it: http://lists.openwall.net/linux-kernel/2014/07/17/235 You unfortunately can't get rid of the API semantics in Erlang since some older systems will definitely fail this. In case of an older system, the right approach is to crash. And probably never to call strong_rand_bytes ever on sane systems :) On Wed Jan 07 2015 at 4:26:43 PM Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > On 12/23/2014 03:56 PM, Jesper Louis Andersen wrote: > > 3. ... The 'strong_rand_bytes/1' function can > > return 'low_entropy' which is outright wrong and preposterous on modern > > machines. It never will, if the underlying random primitive is correctly > > implemented. The whole idea of "running out of entropy" is false. > > > > 'low_entropy' from strong_rand_bytes/1 is a direct mapping to RAND_bytes > returning error, which OpenSSL docs say can happen "... if the PRNG has not > been seeded with enough randomness to ensure an unpredictable byte > sequence." > > And the Linux man page for /dev/random and /dev/urandom says things like: > "When the entropy pool is empty, reads from /dev/random will block > until additional environmental noise is gathered.". > > So, it seems to me that "running out of entropy" is at least a valid > concept. > And doing "cat /dev/random" on my Linux machine sure do block > after a few hundred bytes of output. > > > /Sverker > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jan 8 14:00:49 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 08 Jan 2015 13:00:49 +0000 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true References: Message-ID: The current (Release 17.x) implementation of maps are deliberately kept simple. This has some severe performance problems, but it maximizes implementation flexibility. I think it is a good approach to the problem. * Make it right * Make it beautiful * Make it fast Of course, having a defined function for this: -spec maps:find(fun ((B) -> boolean()), map(A, B)) -> {ok, {A, B}} | not_found. Could be useful in the long run. But this is an iterative process that moves ahead one release every year :) On Thu Jan 08 2015 at 11:42:37 AM Joe Armstrong wrote: > Reply in > > http://joearms.github.io/2015/01/08/Some_Performance- > Measurements-On-Maps.html > > /Joe > > > On Thu, Jan 8, 2015 at 8:47 AM, Ulf Wiger wrote: > > > > On 08 Jan 2015, at 04:56, Harit Himanshu > > wrote: > > > > My attempt, looks like > > > > map_search_pred(Map, Pred) -> > > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, > Value) > > =:= true], > > case length(M) of > > 0 -> {}; > > _ -> lists:nth(1, M) > > end. > > > > ... > > > > Problem? > > The problem is efficiency. > > It uses list comprehension and runs for all the elements in the list. The > > better solution would be to return after the first match. > > > > As a beginner to language, I do not know a better idiomatic way to solve > > this, so looking for better ideas and suggestions > > > > > > You can use > > > > map_search_pred(Map, Pred) -> > > case lists:dropwhile(fun(X) -> not Pred(X) end, maps:to_list(Map)) of > > [Found | _] -> Found; > > [] -> false > > end. > > > > A few comments. > > > > - Using {} as a return value denoting ?not found? is unusual. My version > > mimicks lists:keyfind/2, which returns the matching object or ?false?. > > > > - Testing the length of the list using length/1 will force a count of all > > elements. In this case, you are only interested in checking if the list > is > > non-empty, which is better done with a pattern-match. This also > eliminates > > the need for calling lists:nth(1,M), which is also a more expensive > option > > than hd(M). > > > > BR, > > Ulf W > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > > http://feuerlabs.com > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Jan 8 15:20:15 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 8 Jan 2015 18:20:15 +0400 Subject: [erlang-questions] {error,emfile} In-Reply-To: References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> Message-ID: Wow. We are still using plain init.d script and plain limits.conf is working. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Thu Jan 8 15:24:01 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Thu, 8 Jan 2015 15:24:01 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: Thank you Jesper! =) 2015-01-08 14:00 GMT+01:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > The current (Release 17.x) implementation of maps are deliberately kept > simple. This has some severe performance problems, but it maximizes > implementation flexibility. I think it is a good approach to the problem. > > * Make it right > * Make it beautiful > * Make it fast > > Of course, having a defined function for this: > > -spec maps:find(fun ((B) -> boolean()), map(A, B)) -> {ok, {A, B}} | > not_found. > > Could be useful in the long run. But this is an iterative process that > moves ahead one release every year :) > > > On Thu Jan 08 2015 at 11:42:37 AM Joe Armstrong wrote: > >> Reply in >> >> http://joearms.github.io/2015/01/08/Some_Performance- >> Measurements-On-Maps.html >> >> /Joe >> >> >> On Thu, Jan 8, 2015 at 8:47 AM, Ulf Wiger wrote: >> > >> > On 08 Jan 2015, at 04:56, Harit Himanshu > > >> > wrote: >> > >> > My attempt, looks like >> > >> > map_search_pred(Map, Pred) -> >> > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, >> Value) >> > =:= true], >> > case length(M) of >> > 0 -> {}; >> > _ -> lists:nth(1, M) >> > end. >> > >> > ... >> > >> > Problem? >> > The problem is efficiency. >> > It uses list comprehension and runs for all the elements in the list. >> The >> > better solution would be to return after the first match. >> > >> > As a beginner to language, I do not know a better idiomatic way to solve >> > this, so looking for better ideas and suggestions >> > >> > >> > You can use >> > >> > map_search_pred(Map, Pred) -> >> > case lists:dropwhile(fun(X) -> not Pred(X) end, maps:to_list(Map)) >> of >> > [Found | _] -> Found; >> > [] -> false >> > end. >> > >> > A few comments. >> > >> > - Using {} as a return value denoting ?not found? is unusual. My version >> > mimicks lists:keyfind/2, which returns the matching object or ?false?. >> > >> > - Testing the length of the list using length/1 will force a count of >> all >> > elements. In this case, you are only interested in checking if the list >> is >> > non-empty, which is better done with a pattern-match. This also >> eliminates >> > the need for calling lists:nth(1,M), which is also a more expensive >> option >> > than hd(M). >> > >> > BR, >> > Ulf W >> > >> > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> > http://feuerlabs.com >> > >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.proctor@REDACTED Thu Jan 8 15:48:14 2015 From: steven.proctor@REDACTED (Steven Proctor) Date: Thu, 8 Jan 2015 08:48:14 -0600 Subject: [erlang-questions] Calling all blogs with Erlang content for PlanetErlang.com Message-ID: I realized I missed announcing PlanetErlang.com here, and feel the need to remedy that mistake. If you have a blog with a tag/category/label for entries related to the Erlang ecosystem, I would love for you to -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.proctor@REDACTED Thu Jan 8 15:58:59 2015 From: steven.proctor@REDACTED (Steven Proctor) Date: Thu, 8 Jan 2015 08:58:59 -0600 Subject: [erlang-questions] Calling all blogs with Erlang content for PlanetErlang.com Message-ID: I realized I missed announcing PlanetErlang.com here, and feel the need to remedy that mistake. If you have a blog with a tag/category/label for posts related to the Erlang ecosystem, I would love to have your rss feed for that tag as part of Planet Erlang. I have worked on bootstrapping it myself from those blogs I knew about that have a Erlang tag and feed, but want to include as many people in the Planet feed as possible. If you do Elixir, LFE, Joxa, Erlog, or any other languages, tools, "libraries", or applications around Erlang and the Erlang ecosystem, I would love to get your RSS feed added. To add your site, please submit a pull request to https://github.com/stevenproctor/planet-erlang. If that is not possible, you can send me the name of the Author/Site you would like it to show up as and the link to the RSS feed for your Erlang related posts and I will get it added to the repo. Thank you so much, --Proctor -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Thu Jan 8 18:46:15 2015 From: imantc@REDACTED (Imants Cekusins) Date: Thu, 8 Jan 2015 18:46:15 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: An iterator function with a break option would be very useful, too. Even the current maps:fold function could be modified to allow break: fold(Fun, Init, Map) -> Acc Types: Fun = fun((K, V, AccIn) -> AccOut) modify to: Fun = fun((K, V, AccIn) -> AccOut | {break, AccOut}) ? From mononcqc@REDACTED Thu Jan 8 19:02:30 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 8 Jan 2015 13:02:30 -0500 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: <20150108180227.GA7939@ferdair.local> On 01/08, Imants Cekusins wrote: > An iterator function with a break option would be very useful, too. > > Even the current maps:fold function could be modified to allow break: > > fold(Fun, Init, Map) -> Acc > > Types: > > Fun = fun((K, V, AccIn) -> AccOut) > > modify to: > Fun = fun((K, V, AccIn) -> AccOut | {break, AccOut}) > > ? try maps:fold(fun(K, V, Map) -> ..., throw({found, {K,V}) end, Init, Map0), not_found catch {found, Val} -> Val end. This gives you a similar result without anything else required. Alternatively you could just go 'catch maps:fold(...)', but in case of exceptions that aren't throws, you may yield unexpected results (a stacktrace) whereas the 'try ... catch' would just have crashed. From egil@REDACTED Thu Jan 8 19:10:22 2015 From: egil@REDACTED (=?windows-1252?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 8 Jan 2015 19:10:22 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: <54AEC80E.6090806@erlang.org> Obviously there is a need for iterators. Map comprehensions will not work well without it. It's already on the TODO list .. On 2015-01-08 18:46, Imants Cekusins wrote: > An iterator function with a break option would be very useful, too. > > Even the current maps:fold function could be modified to allow break: > > fold(Fun, Init, Map) -> Acc > > Types: > > Fun = fun((K, V, AccIn) -> AccOut) > > modify to: > Fun = fun((K, V, AccIn) -> AccOut | {break, AccOut}) > > ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From imantc@REDACTED Thu Jan 8 19:24:11 2015 From: imantc@REDACTED (Imants Cekusins) Date: Thu, 8 Jan 2015 19:24:11 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: <54AEC80E.6090806@erlang.org> References: <54AEC80E.6090806@erlang.org> Message-ID: Cheers Fred! Yep, this is an option. Some folks warn against using try & catch though. Great book btw :) Egil, is Accessing a single value V = M#{ K }. http://www.erlang.org/eeps/eep-0043.html on TODO list, too? Would be so handy. From wallentin.dahlberg@REDACTED Thu Jan 8 19:38:29 2015 From: wallentin.dahlberg@REDACTED (=?utf-8?Q?Bj=C3=B6rn-Egil?=) Date: Thu, 8 Jan 2015 19:38:29 +0100 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: <54AEC80E.6090806@erlang.org> Message-ID: If it's on there it's below the horizon atm. We, or perhaps I, have some issue with it's syntax and also it's function. I would rather see syntax and design for functor/lenses. It would be more useful. That is not in scope of the current maps implementation though. // egil Sent from my iPhone > 8 jan 2015 kl. 19:24 skrev Imants Cekusins : > > Cheers Fred! Yep, this is an option. Some folks warn against using try > & catch though. > > Great book btw :) > > > Egil, is Accessing a single value > > V = M#{ K }. > > http://www.erlang.org/eeps/eep-0043.html > > on TODO list, too? Would be so handy. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto.ostinelli@REDACTED Thu Jan 8 19:42:23 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Thu, 8 Jan 2015 19:42:23 +0100 Subject: [erlang-questions] {error,emfile} In-Reply-To: References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> Message-ID: <948E4058-A60B-4827-94F5-14ABDFAA1F02@widetag.com> Is this a suggestion to not use upstart, or just a honest astonishment for ubuntu's upstart choices? > On 08/gen/2015, at 15:20, Max Lapshin wrote: > > Wow. > > We are still using plain init.d script and plain limits.conf is working. > From harit.subscriptions@REDACTED Thu Jan 8 20:58:27 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Thu, 8 Jan 2015 11:58:27 -0800 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: Huge thanks @joe for detailed analysis on that. That made me go through some of the documentation and read more. I believe find_first is definitely a better way. When I said performance, I really meant traversing the complete list is un-necessary, early return is best way do do once {Key, Value} is found Thanks again + Harit Himanshu On Thu, Jan 8, 2015 at 2:42 AM, Joe Armstrong wrote: > Reply in > > > http://joearms.github.io/2015/01/08/Some_Performance-Measurements-On-Maps.html > > /Joe > > > On Thu, Jan 8, 2015 at 8:47 AM, Ulf Wiger wrote: > > > > On 08 Jan 2015, at 04:56, Harit Himanshu > > wrote: > > > > My attempt, looks like > > > > map_search_pred(Map, Pred) -> > > M = [{Key, Value} || {Key, Value} <- maps:to_list(Map), Pred(Key, > Value) > > =:= true], > > case length(M) of > > 0 -> {}; > > _ -> lists:nth(1, M) > > end. > > > > ... > > > > Problem? > > The problem is efficiency. > > It uses list comprehension and runs for all the elements in the list. The > > better solution would be to return after the first match. > > > > As a beginner to language, I do not know a better idiomatic way to solve > > this, so looking for better ideas and suggestions > > > > > > You can use > > > > map_search_pred(Map, Pred) -> > > case lists:dropwhile(fun(X) -> not Pred(X) end, maps:to_list(Map)) of > > [Found | _] -> Found; > > [] -> false > > end. > > > > A few comments. > > > > - Using {} as a return value denoting ?not found? is unusual. My version > > mimicks lists:keyfind/2, which returns the matching object or ?false?. > > > > - Testing the length of the list using length/1 will force a count of all > > elements. In this case, you are only interested in checking if the list > is > > non-empty, which is better done with a pattern-match. This also > eliminates > > the need for calling lists:nth(1,M), which is also a more expensive > option > > than hd(M). > > > > BR, > > Ulf W > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > > http://feuerlabs.com > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Jan 8 22:06:15 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 8 Jan 2015 22:06:15 +0100 Subject: [erlang-questions] {error,emfile} In-Reply-To: <948E4058-A60B-4827-94F5-14ABDFAA1F02@widetag.com> References: <5c826645e2d384b50523c4ebd804a5bb.squirrel@mail.jschneider.net> <948E4058-A60B-4827-94F5-14ABDFAA1F02@widetag.com> Message-ID: On Thu, Jan 8, 2015 at 7:42 PM, Roberto Ostinelli < roberto.ostinelli@REDACTED> wrote: > Is this a suggestion to not use upstart, or just a honest astonishment > for ubuntu's upstart choices? > > > limits.conf is also not used in systemd (on centos 7 at least). We had to set it using a service setting... - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jan 9 02:12:57 2015 From: ok@REDACTED (ok@REDACTED) Date: Fri, 9 Jan 2015 14:12:57 +1300 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: <140e14179c87d768db5d72553ba8ba98.squirrel@chasm.otago.ac.nz> > The more formal definition of problem is > > Write a function map_search_pred(Map, Pred) that returns the first element > {Key,Value} in the map for which Pred(Key, Value) is true. Can we start right at the beginning? Looking at "maps" through the lens of hashed collections in a range of other languages, what does "FIRST" *mean* for a hash? (In Smalltalk, for example, #findFirst: is defined on sequences, but *not* on dictionaries.) Basically, what I'm getting at here is "do you REALLY mean 'FIRST' or do you actually mean 'ANY ONE'?" >From the on-line documentation, "The fuction (sic.) [to_list/2] returns a list of pairs representing the key-value associations of Map, where the pairs, [{K1,V1}, ..., {Kn,Vn}], >>>> are returned in arbitrary order." That is, the "FIRST" as returned by maps:to_list/2 is *NOT* the first in any *semantic* order relevant to your program. Since to_list/2 takes O(|Map|) time and space, it might be better to do map_search_pred(Map, Pred) -> Filter = fun (K, V, {}) -> case Pred(K, V) of true -> {K,V} ; false -> {} end ; (_, _, State) -> State end, maps:fold(Filter, {}, Map). This will take O(|Map|) time, but after Pred returns true, it will not call Pred again. (I do hope maps:fold/3 does not call maps:to_list/1 behind the scenes...) Although the on-line documentation fails to say one way or the other, the order in which key/value pairs are passed to the filter is probably just as arbitary as the order of to_list/1, so "FIRST" is just as dodgy a concept here as in the other approach. If "first" is important to you, I can't help wondering why you are not using a data structure for which "first" is well defined. From ok@REDACTED Fri Jan 9 02:26:58 2015 From: ok@REDACTED (ok@REDACTED) Date: Fri, 9 Jan 2015 14:26:58 +1300 Subject: [erlang-questions] How to return first {Key, Value} from Map where Predicate is true In-Reply-To: References: Message-ID: <849da3811f0baa4a7023ac177b9357a7.squirrel@chasm.otago.ac.nz> > > Of course, having a defined function for this: > > -spec maps:find(fun ((B) -> boolean()), map(A, B)) -> {ok, {A, B}} | > not_found. > > Could be useful in the long run. It could, but it would not help with the original poster's problem, because he wanted to supply an (A,B) -> boolean() predicate, not a (B) -> boolean() one. I wonder if the original poster could somehow keep a list of all the keys, because [dummy || K <- Keys, case maps:get(K, Map) of V -> case Pred(K, V) of true -> throw({K,V}) ; false -> false end end] would do the job without any unwanted allocation, and the order of the Keys *might* be controlled in a way that gives 'first' a sensible meaning. (Of course there's a try/catch around this as well, which I couldn't be bothered typing.) From harit.subscriptions@REDACTED Fri Jan 9 14:54:07 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Fri, 9 Jan 2015 05:54:07 -0800 Subject: [erlang-questions] Reverses Order of Bytes in Binary Message-ID: This is another exercise in Programming Edition book that talk about reverses the order of bytes in a binary So my attempt looks like 58> B = <<1,2,3,4>>. <<1,2,3,4>> 59> binary_to_list(B). [1,2,3,4] 60> lists:reverse(binary_to_list(B)). [4,3,2,1] 61> list_to_binary(lists:reverse(binary_to_list(B))). <<4,3,2,1>> 62> Wondering if there is a better way available. Any recommendations? Thank you + Harit Himanshu -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jan 9 15:03:20 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Jan 2015 18:03:20 +0400 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: reverse(Bin) -> reverse(Bin, <<>>). reverse(<<>>, Acc) -> Acc; reverse(<>, Acc) -> reverse(Bin, <>). -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Fri Jan 9 15:04:01 2015 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 9 Jan 2015 15:04:01 +0100 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: If you have a small amount of bytes, nox showed an interesting solution a couple days ago that relies on endianness: 1> <> = <<1, 2, 3, 4>>. <<1,2,3,4>> 2> <>. <<4,3,2,1>> *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer On Fri, Jan 9, 2015 at 2:54 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > This is another exercise in Programming Edition book that talk about > > reverses the order of bytes in a binary > > So my attempt looks like > > 58> B = <<1,2,3,4>>. > > <<1,2,3,4>> > > 59> binary_to_list(B). > > [1,2,3,4] > > 60> lists:reverse(binary_to_list(B)). > > [4,3,2,1] > > 61> list_to_binary(lists:reverse(binary_to_list(B))). > > <<4,3,2,1>> > > 62> > > > Wondering if there is a better way available. Any recommendations? > > Thank you > + Harit Himanshu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jan 9 15:08:05 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 09 Jan 2015 15:08:05 +0100 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: <54AFE0C5.2080609@ninenines.eu> This doesn't reverse, this will return the same binary. :-) Would reverse if you put C before Acc every time but that would fill your memory quickly. On 01/09/2015 03:03 PM, Max Lapshin wrote: > reverse(Bin) -> > reverse(Bin, <<>>). > > reverse(<<>>, Acc) -> Acc; > reverse(<>, Acc) -> reverse(Bin, <>). > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From max.lapshin@REDACTED Fri Jan 9 15:18:26 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 9 Jan 2015 18:18:26 +0400 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: <54AFE0C5.2080609@ninenines.eu> References: <54AFE0C5.2080609@ninenines.eu> Message-ID: Ah, really. I should be ashamed. reverse(Bin) -> reverse(Bin, []). reverse(<<>>, Acc) -> list_to_binary(Acc); reverse(<>, Acc) -> reverse(Bin, [C|Acc]). -------------- next part -------------- An HTML attachment was scrubbed... URL: From kovyl2404@REDACTED Fri Jan 9 17:39:05 2015 From: kovyl2404@REDACTED (Viacheslav V. Kovalev) Date: Fri, 9 Jan 2015 20:39:05 +0400 Subject: [erlang-questions] Probably cpu_sup bug? Message-ID: Probably cpu_sup bug? Hello. I'am using Erlang E17 on Ubuntu boxes with kernel 3.16.0-28. Some time ago I've noticed strange messages in logs. > {{badmatch,{more,"~d",24,[51,1.090200e+02,1.199300e+02,1.487400e+02]}}, > [{cpu_sup,get_uint32_measurement,2,[{file,"cpu_sup.erl"},{line,226}]}, > {cpu_sup,measurement_server_loop,1,[{file,"cpu_sup.erl"},{line,585}]}]} It comes from this code https://github.com/erlang/otp/blob/maint/lib/os_mon/src/cpu_sup.erl#L226. Actually there is `io_lib:fread/2` failed when tried to parse input string read from `/proc/loadavg`. Looking on its input stack I believe that input string was "148.74 119.93 109.02 51/...". But `cpu_sup` reads EXACTLY 24 characters from file (thinking that load avg's are lesser than 100). Doesn't this looks buggy? Why not to rewrite this code as following (or something like that)? get_uint32_measurement(Request, #internal{os_type = {unix, linux}}) -> {ok,F} = file:open("/proc/loadavg",[read,raw]), {ok,D} = file:read_line(F), ok = file:close(F), {ok,[Load1,Load5,Load15,_PRun,PTotal],_} = io_lib:fread("~f ~f ~f ~d/~d", D), case Request of ?avg1 -> sunify(Load1); ?avg5 -> sunify(Load5); ?avg15 -> sunify(Load15); ?ping -> 4711; ?nprocs -> PTotal end; From tuncer.ayaz@REDACTED Fri Jan 9 19:16:46 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 9 Jan 2015 19:16:46 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: <54ABBC52.90408@ninenines.eu> References: <54AA6B03.2020804@ninenines.eu> <54AB190D.8070402@ninenines.eu> <54ABBC52.90408@ninenines.eu> Message-ID: On Tue, Jan 6, 2015 at 11:43 AM, Loic Hoguin wrote: > On 01/06/2015 11:13 AM, Tuncer Ayaz wrote: > > > > On Tue, Jan 6, 2015 at 12:06 AM, Loic Hoguin wrote: > > > > > > Yes that also adds to the load time, but it probably wouldn't be > > > too noticeable if erlc was loading only the modules it requires > > > instead of everything. I don't think it uses the network code, > > > dets, process > > > > > > It can use the network code if code is not loaded from disk. > > Erlang's distributed nature is the reason for some indirection you > > tried to circumvent in, for example, file operations :). > > > But erlc only compiles *files* doesn't it? It doesn't start > distribution or anything so there should be no indirection? I was referring to the fact that erl_prim_loader can use either efile or inet as the loader, and that this might pull in more modules. > > > groups and so on, and it may also not need the whole OTP > > > application framework. > > > > > > I will play with generating a smaller boot file when I get the > > > time and see what happens. If it can be made closer to 0s this > > > would allow me to just use %.erl: %.beam rules, something that's > > > been on my wishlist from the start. > > > > > > Maybe the resulting boot file can be used for escript as well. > > > Well "escript" wouldn't be part of the boot file if erlc doesn't > need it, so my guess is no. :-) Given that you're using erl_compile.beam, I wouldn't expect too much more to add for getting escript running, so if you document the process, we can try the same for escript and maybe make escript use a custom boot file. > The hard part is figuring out what it actually needs. > > > Also, the speedups we've discussed also apply to older versions > > and are not fixing the overhead introduced between R13 and now. > > So, if you're serious about this, and want to invest time, I'd > > suggest to profile R13 against R14 and R15 with your trace/profile > > tool of choice and see where time in (mostly) C and maybe Erlang > > is spent since >=R14. > > > I wish I could invest time now but this and the boot file experiment > will probably have to wait for a while. Maybe we should add a page on erlang.org or the github wiki with interesting projects for people who are actively looking for challenges to get into otp.git. Bruce? From roberto@REDACTED Fri Jan 9 20:36:15 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 9 Jan 2015 20:36:15 +0100 Subject: [erlang-questions] Cowboy monitoring long-lived HTTP connections Message-ID: Dear list, I am using cowboy to manage long-lived HTTP requests, that I use to send server generated events. When a request comes in, I add it to a process registry. I can then send messages to the request process, that takes care of sending it down the TCP pipe. The question is: when one of these processes dies, also because of an internal crash, I want to ensure that I remove it's pid from the process registry. The question is: what is the best practice to do this? Is the terminate function called also when a process dies? If so, I could just do: init(_Type, Req, []) -> ets:insert(?PID_TABLE, {self()}), {ok, Req, undefined}. handle(Req, _ReqState) -> [handle request] terminate(_Reason, _Req, _State) -> ets:delete(?PID_TABLE, {self()}), ok. Or do I need to monitor it with some kind of registry, for instance: init(_Type, Req, []) -> registry:register(self()), {ok, Req, undefined}. handle(Req, _ReqState) -> [handle request] terminate(_Reason, _Req, _State) -> ok. And registry is a gen_server: register(Pid) -> gen_server:call(?MODULE,{register, Pid}). ... handle_call({register, Pid}, _From, State) -> ets:insert(?PID_TABLE, {Pid}), erlang:monitor(process, Pid), {reply, ok, State}; ... handle_info({'DOWN', _Ref, process, Pid, _Reason}, State) -> ets:delete(?PID_TABLE, {Pid}), {noreply, State}. Any recommendations? Thank you ^^_ r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Jan 9 22:43:25 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Fri, 09 Jan 2015 22:43:25 +0100 Subject: [erlang-questions] Cowboy monitoring long-lived HTTP connections In-Reply-To: References: Message-ID: <54B04B7D.2060502@ninenines.eu> Terminate is not recommended for this and will become an optional callback because of how rarely it is actually useful. Cleaning up inside the process is never perfect because when a process is exited with reason 'kill' it ends immediately, with no way to call any terminate function. A supervisor will use 'kill' when a child is configured with 'brutal_kill' or when the shutdown timeout times out, for example. Then you end up with something that will never be cleaned up. Monitoring is the right solution, but there's usually no need to do it yourself. Your registry, and the second solution in your email, is basically what gproc properties give you: https://github.com/uwiger/gproc#use-case-pubsub-patterns Gproc will take care of monitoring, ets table and sending for you. On 01/09/2015 08:36 PM, Roberto Ostinelli wrote: > Dear list, > I am using cowboy to manage long-lived HTTP requests, that I use to send > server generated events. > > When a request comes in, I add it to a process registry. I can then send > messages to the request process, that takes care of sending it down the > TCP pipe. > > The question is: when one of these processes dies, also because of an > internal crash, I want to ensure that I remove it's pid from the process > registry. > > The question is: what is the best practice to do this? Is the terminate > function called also when a process dies? If so, I could just do: > > init(_Type, Req, []) -> > ets:insert(?PID_TABLE, {self()}), > {ok, Req, undefined}. > > handle(Req, _ReqState) -> > [handle request] > > terminate(_Reason, _Req, _State) -> > ets:delete(?PID_TABLE, {self()}), > ok. > > > Or do I need to monitor it with some kind of registry, for instance: > > init(_Type, Req, []) -> > registry:register(self()), > {ok, Req, undefined}. > > handle(Req, _ReqState) -> > [handle request] > > terminate(_Reason, _Req, _State) -> > ok. > > > And registry is a gen_server: > > register(Pid) -> > gen_server:call(?MODULE,{register, Pid}). > > ... > > handle_call({register, Pid}, _From, State) -> > ets:insert(?PID_TABLE, {Pid}), > erlang:monitor(process, Pid), > {reply, ok, State}; > > ... > > handle_info({'DOWN', _Ref, process, Pid, _Reason}, State) -> > ets:delete(?PID_TABLE, {Pid}), > {noreply, State}. > > > Any recommendations? > > Thank you ^^_ > r. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From n.oxyde@REDACTED Sat Jan 10 11:40:21 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 10 Jan 2015 11:40:21 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <54AB190D.8070402@ninenines.eu> <54ABBC52.90408@ninenines.eu> Message-ID: Le 9 janv. 2015 ? 19:16, Tuncer Ayaz a ?crit : > Maybe we should add a page on erlang.org or the github wiki with > interesting projects for people who are actively looking for > challenges to get into otp.git. Bruce? I'm no Bruce, but I use my own fork's issues to track some things I want to do or see completed: https://github.com/nox/otp/issues From max.lapshin@REDACTED Sat Jan 10 12:01:01 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 10 Jan 2015 15:01:01 +0400 Subject: [erlang-questions] Cowboy monitoring long-lived HTTP connections In-Reply-To: <54B04B7D.2060502@ninenines.eu> References: <54B04B7D.2060502@ninenines.eu> Message-ID: terminate should never be used as a reliable callback. Our production-tested gen_tracker registrator: https://github.com/erlyvideo/gen_tracker adds after_terminate callback that will be called with some metadata _after_ process dies. But your case will be handled automatically in gen_tracker and in gproc. -------------- next part -------------- An HTML attachment was scrubbed... URL: From seriy.pr@REDACTED Sat Jan 10 12:24:54 2015 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Sat, 10 Jan 2015 14:24:54 +0300 Subject: [erlang-questions] Reverses Order of Bytes in Binary Message-ID: Probably this one will be more effective reverse(Bin) -> reverse(Bin, byte_size(Bin)). reverse(_, 0) -> <<>>; reverse(Bin, Pos) -> <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sat Jan 10 12:30:55 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Sat, 10 Jan 2015 11:30:55 +0000 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: Remember lists:reverse/1 is implemented in C & does some non-erlangy things (IIRC) so should be jolly fast & efficient. Unless binary_to_list/1 is v bad the OP's version is the best so far (imho). Best wishes Ivan -- festina lente > On 10 Jan 2015, at 11:24, ?????? ???????? wrote: > > Probably this one will be more effective > > reverse(Bin) -> > reverse(Bin, byte_size(Bin)). > > reverse(_, 0) -> <<>>; > reverse(Bin, Pos) -> > <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From seriy.pr@REDACTED Sat Jan 10 12:46:50 2015 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Sat, 10 Jan 2015 14:46:50 +0300 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: Ok, looks like topicstarter's solution is the fastest one: ======================= -module(bin_reverse). -export([reverse_test/1]). reverse_test(Bin) -> [ {"body_pos", timer:tc(fun reverse_body_pos/1, [Bin])}, {"tail_pos", timer:tc(fun reverse_tail_pos/1, [Bin])}, {"list", timer:tc(fun reverse_list/1, [Bin])}, {"list_to_bin", timer:tc(fun reverse_list_to_bin/1, [Bin])} ]. reverse_tail_pos(Bin) -> reverse1(Bin, byte_size(Bin), <<>>). reverse1(_, 0, Acc) -> Acc; reverse1(Bin, Pos, Acc) -> reverse1(Bin, Pos - 1, <>). reverse_body_pos(Bin) -> reverse2(Bin, byte_size(Bin)). reverse2(_, 0) -> <<>>; reverse2(Bin, Pos) -> <<(binary:at(Bin, Pos - 1)), (reverse2(Bin, Pos - 1))/binary >>. reverse_list(Bin) -> list_to_binary(lists:reverse(binary_to_list(Bin))). reverse_list_to_bin(Bin) -> reverse3(Bin, []). reverse3(<<>>, Acc) -> list_to_binary(Acc); reverse3(<>, Acc) -> reverse3(Bin, [C|Acc]). ======================= 24> bin_reverse:reverse_test(crypto:rand_bytes(100)). [{"body_pos", {32, <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, 85,33,206,130,164,3,242,174,157,...>>}}, {"tail_pos", {14, <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, 85,33,206,130,164,3,242,174,...>>}}, {"list", {3, <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, 85,33,206,130,164,3,242,...>>}}, {"list_to_bin", {13, <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, 85,33,206,130,164,3,...>>}}] 2015-01-10 14:30 GMT+03:00 Ivan Uemlianin : > Remember lists:reverse/1 is implemented in C & does some non-erlangy > things (IIRC) so should be jolly fast & efficient. Unless binary_to_list/1 > is v bad the OP's version is the best so far (imho). > > Best wishes > > Ivan > > > -- > festina lente > > > > On 10 Jan 2015, at 11:24, ?????? ???????? wrote: > > > > Probably this one will be more effective > > > > reverse(Bin) -> > > reverse(Bin, byte_size(Bin)). > > > > reverse(_, 0) -> <<>>; > > reverse(Bin, Pos) -> > > <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Sat Jan 10 13:06:30 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Sat, 10 Jan 2015 13:06:30 +0100 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: Async version so results are less skewed by GC. With a larger bin it is more clear what the best solution is: bin_reverse:reverse_test(crypto:rand_bytes(100000)). "list"=15515 "tail_pos"=24061 "list_to_bin"=28627 "body_pos"=4781891 -module(bin_reverse). -export([reverse_test/1]). reverse_test(Bin) -> [ async("body_pos",fun reverse_body_pos/1,Bin), async("tail_pos",fun reverse_tail_pos/1,Bin), async("list",fun reverse_list/1,Bin), async("list_to_bin",fun reverse_list_to_bin/1,Bin) ]. async(Name,Fun,Bin) -> spawn(fun() -> {T,_} = timer:tc(Fun,[Bin]), io:format("~p=~p~n",[Name,T]) end). reverse_tail_pos(Bin) -> reverse1(Bin, byte_size(Bin), <<>>). reverse1(_, 0, Acc) -> Acc; reverse1(Bin, Pos, Acc) -> reverse1(Bin, Pos - 1, <>). reverse_body_pos(Bin) -> reverse2(Bin, byte_size(Bin)). reverse2(_, 0) -> <<>>; reverse2(Bin, Pos) -> <<(binary:at(Bin, Pos - 1)), (reverse2(Bin, Pos - 1))/binary >>. reverse_list(Bin) -> list_to_binary(lists:reverse(binary_to_list(Bin))). reverse_list_to_bin(Bin) -> reverse3(Bin, []). reverse3(<<>>, Acc) -> list_to_binary(Acc); reverse3(<>, Acc) -> reverse3(Bin, [C|Acc]). On 10 Jan 2015, at 12:46, ?????? ???????? wrote: > Ok, looks like topicstarter's solution is the fastest one: > > ======================= > -module(bin_reverse). > -export([reverse_test/1]). > > > reverse_test(Bin) -> > [ > {"body_pos", timer:tc(fun reverse_body_pos/1, [Bin])}, > {"tail_pos", timer:tc(fun reverse_tail_pos/1, [Bin])}, > {"list", timer:tc(fun reverse_list/1, [Bin])}, > {"list_to_bin", timer:tc(fun reverse_list_to_bin/1, [Bin])} > ]. > > > > reverse_tail_pos(Bin) -> > reverse1(Bin, byte_size(Bin), <<>>). > > reverse1(_, 0, Acc) -> Acc; > reverse1(Bin, Pos, Acc) -> > reverse1(Bin, Pos - 1, <>). > > > > reverse_body_pos(Bin) -> > reverse2(Bin, byte_size(Bin)). > > reverse2(_, 0) -> <<>>; > reverse2(Bin, Pos) -> > <<(binary:at(Bin, Pos - 1)), (reverse2(Bin, Pos - 1))/binary >>. > > > > reverse_list(Bin) -> > list_to_binary(lists:reverse(binary_to_list(Bin))). > > > > reverse_list_to_bin(Bin) -> > reverse3(Bin, []). > > reverse3(<<>>, Acc) -> list_to_binary(Acc); > reverse3(<>, Acc) -> reverse3(Bin, [C|Acc]). > > ======================= > > 24> bin_reverse:reverse_test(crypto:rand_bytes(100)). > [{"body_pos", > {32, > <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, > 85,33,206,130,164,3,242,174,157,...>>}}, > {"tail_pos", > {14, > <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, > 85,33,206,130,164,3,242,174,...>>}}, > {"list", > {3, > <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, > 85,33,206,130,164,3,242,...>>}}, > {"list_to_bin", > {13, > <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, > 85,33,206,130,164,3,...>>}}] > > 2015-01-10 14:30 GMT+03:00 Ivan Uemlianin : > Remember lists:reverse/1 is implemented in C & does some non-erlangy things (IIRC) so should be jolly fast & efficient. Unless binary_to_list/1 is v bad the OP's version is the best so far (imho). > > Best wishes > > Ivan > > > -- > festina lente > > > > On 10 Jan 2015, at 11:24, ?????? ???????? wrote: > > > > Probably this one will be more effective > > > > reverse(Bin) -> > > reverse(Bin, byte_size(Bin)). > > > > reverse(_, 0) -> <<>>; > > reverse(Bin, Pos) -> > > <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sat Jan 10 13:28:22 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Sat, 10 Jan 2015 12:28:22 +0000 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: <148261D6-53B5-40CE-9430-CC6F6C3C97EC@llaisdy.com> Testament to what a friendly language erlang is: the learner effortlessly arrives at the most fitting solution. -- festina lente > On 10 Jan 2015, at 12:06, Sergej Jurecko wrote: > > Async version so results are less skewed by GC. > With a larger bin it is more clear what the best solution is: > > bin_reverse:reverse_test(crypto:rand_bytes(100000)). > "list"=15515 > "tail_pos"=24061 > "list_to_bin"=28627 > "body_pos"=4781891 > > -module(bin_reverse). > -export([reverse_test/1]). > > > reverse_test(Bin) -> > [ > async("body_pos",fun reverse_body_pos/1,Bin), > async("tail_pos",fun reverse_tail_pos/1,Bin), > async("list",fun reverse_list/1,Bin), > async("list_to_bin",fun reverse_list_to_bin/1,Bin) > ]. > > async(Name,Fun,Bin) -> > spawn(fun() -> > {T,_} = timer:tc(Fun,[Bin]), > io:format("~p=~p~n",[Name,T]) > end). > > > reverse_tail_pos(Bin) -> > reverse1(Bin, byte_size(Bin), <<>>). > > reverse1(_, 0, Acc) -> Acc; > reverse1(Bin, Pos, Acc) -> > reverse1(Bin, Pos - 1, <>). > > > > reverse_body_pos(Bin) -> > reverse2(Bin, byte_size(Bin)). > > reverse2(_, 0) -> <<>>; > reverse2(Bin, Pos) -> > <<(binary:at(Bin, Pos - 1)), (reverse2(Bin, Pos - 1))/binary >>. > > > > reverse_list(Bin) -> > list_to_binary(lists:reverse(binary_to_list(Bin))). > > > > reverse_list_to_bin(Bin) -> > reverse3(Bin, []). > > reverse3(<<>>, Acc) -> list_to_binary(Acc); > reverse3(<>, Acc) -> reverse3(Bin, [C|Acc]). > > >> On 10 Jan 2015, at 12:46, ?????? ???????? wrote: >> >> Ok, looks like topicstarter's solution is the fastest one: >> >> ======================= >> -module(bin_reverse). >> -export([reverse_test/1]). >> >> >> reverse_test(Bin) -> >> [ >> {"body_pos", timer:tc(fun reverse_body_pos/1, [Bin])}, >> {"tail_pos", timer:tc(fun reverse_tail_pos/1, [Bin])}, >> {"list", timer:tc(fun reverse_list/1, [Bin])}, >> {"list_to_bin", timer:tc(fun reverse_list_to_bin/1, [Bin])} >> ]. >> >> >> >> reverse_tail_pos(Bin) -> >> reverse1(Bin, byte_size(Bin), <<>>). >> >> reverse1(_, 0, Acc) -> Acc; >> reverse1(Bin, Pos, Acc) -> >> reverse1(Bin, Pos - 1, <>). >> >> >> >> reverse_body_pos(Bin) -> >> reverse2(Bin, byte_size(Bin)). >> >> reverse2(_, 0) -> <<>>; >> reverse2(Bin, Pos) -> >> <<(binary:at(Bin, Pos - 1)), (reverse2(Bin, Pos - 1))/binary >>. >> >> >> >> reverse_list(Bin) -> >> list_to_binary(lists:reverse(binary_to_list(Bin))). >> >> >> >> reverse_list_to_bin(Bin) -> >> reverse3(Bin, []). >> >> reverse3(<<>>, Acc) -> list_to_binary(Acc); >> reverse3(<>, Acc) -> reverse3(Bin, [C|Acc]). >> >> ======================= >> >> 24> bin_reverse:reverse_test(crypto:rand_bytes(100)). >> [{"body_pos", >> {32, >> <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, >> 85,33,206,130,164,3,242,174,157,...>>}}, >> {"tail_pos", >> {14, >> <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, >> 85,33,206,130,164,3,242,174,...>>}}, >> {"list", >> {3, >> <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, >> 85,33,206,130,164,3,242,...>>}}, >> {"list_to_bin", >> {13, >> <<247,128,13,6,1,53,110,148,74,210,196,126,234,115,180, >> 85,33,206,130,164,3,...>>}}] >> >> 2015-01-10 14:30 GMT+03:00 Ivan Uemlianin : >>> Remember lists:reverse/1 is implemented in C & does some non-erlangy things (IIRC) so should be jolly fast & efficient. Unless binary_to_list/1 is v bad the OP's version is the best so far (imho). >>> >>> Best wishes >>> >>> Ivan >>> >>> >>> -- >>> festina lente >>> >>> >>> > On 10 Jan 2015, at 11:24, ?????? ???????? wrote: >>> > >>> > Probably this one will be more effective >>> > >>> > reverse(Bin) -> >>> > reverse(Bin, byte_size(Bin)). >>> > >>> > reverse(_, 0) -> <<>>; >>> > reverse(Bin, Pos) -> >>> > <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sat Jan 10 13:32:14 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 10 Jan 2015 13:32:14 +0100 Subject: [erlang-questions] Why is erlc so quick to start? In-Reply-To: References: <54AA6B03.2020804@ninenines.eu> <54AB190D.8070402@ninenines.eu> <54ABBC52.90408@ninenines.eu> Message-ID: On Sat, Jan 10, 2015 at 11:40 AM, Anthony Ramine wrote: > Le 9 janv. 2015 a 19:16, Tuncer Ayaz ecrit: > > > Maybe we should add a page on erlang.org or the github wiki with > > interesting projects for people who are actively looking for > > challenges to get into otp.git. Bruce? > > I'm no Bruce, but I use my own fork's issues to track some things I > want to do or see completed: > > https://github.com/nox/otp/issues > Yeah, seen that. From what I heard there's a plan for an official public ticket system, but I'm not sure if it's the right place for project ideas with included implementation/design proposals. From freeakk@REDACTED Sat Jan 10 14:07:01 2015 From: freeakk@REDACTED (Michael Uvarov) Date: Sat, 10 Jan 2015 14:07:01 +0100 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: there's another way: reverse(Bin) when is_binary(Bin) -> ????S = bit_size(Bin), ????<> = Bin, ????<>. On January 10, 2015 12:24:54 PM CET, "?????? ????????" wrote: >Probably this one will be more effective > >reverse(Bin) -> > reverse(Bin, byte_size(Bin)). > >reverse(_, 0) -> <<>>; >reverse(Bin, Pos) -> > <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Sat Jan 10 16:14:56 2015 From: mihai@REDACTED (Mihai Balea) Date: Sat, 10 Jan 2015 10:14:56 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: Message-ID: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> > On Jan 5, 2015, at 11:28 AM, xu xiut wrote: > > I am looking at using Erlang for an exchange. If I'm lucky, my transaction volume will be 100 - 150k day or about 4000 transactions/hour, about 1.5/sec At this rate, maybe Erlang wouldn't be necessary, however, would it be the case that Erlang provides better semantics surrounding failure compared to other languages and their ecosystems, and so, even then it would be a great tool choice? > > Is Erlang ideal for scaling out such an exchange? I wonder what is the minimum number of trades a second that would be difficult for erlang to process. I realize it's difficult to answer given the trade execution path isn't described, but it's the first question that comes to mind when I started thinking about the language of choice. > > Any thoughts or suggestions are welcomed, and thank you for letting me share this here! You don?t provide enough information to determine, with a reasonable degree of confidence, if Erlang would be a good fit for your project. Throughput is only one parameter of an exchange, but equally important are latency and burst rate. The failure/recovery semantics of Erlang, while great for many situations, might not be appropriate for an exchange - again, depending on the particulars of your use case. It might turn out that crashing and restarting is not appropriate for you. To put it in perspective, if you lose a message in a chat application it is bad, but not disastrous. If you lose a trade in an exchange, people lose money, and they tend to be _very_ unhappy about that. One thing though, in the hands of an experienced person, Erlang can be used as fast prototyping tool. It?s not a bad place to start for this kind of projects. You can whip up a proof of concept reasonably quickly and then you will know better if this is a good fit. Is this your first foray into Erlang and exchange development? If yes, you will be fighting two new and peculiar domains at the same time, in which case I?d suggest you start with something more familiar to you. If you do have experience with either Erlang or exchange development then maybe give it a shot :) I know my answer is not very specific, but i hope it helps :) Mihai From the.silly.sad@REDACTED Sat Jan 10 16:21:16 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Sat, 10 Jan 2015 16:21:16 +0100 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> Message-ID: <54B1436C.7060008@gmail.com> > The failure/recovery semantics of Erlang, while great for many situations, might not be appropriate for an exchange - again, depending on the particulars of your use case. It might turn out that crashing and restarting is not appropriate for you. To put it in perspective, if you lose a message in a chat application it is bad, but not disastrous. If you lose a trade in an exchange, people lose money, and they tend to be _very_ unhappy about that. i believe this particular problem might be easily shifted up to the next level of implementation. there is always an option to organize some sort of two-phase transactions between clients, so that minimizing server's involvement... distribute calculations wherever possible :) From roberto.ostinelli@REDACTED Sat Jan 10 17:56:15 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Sat, 10 Jan 2015 17:56:15 +0100 Subject: [erlang-questions] Cowboy monitoring long-lived HTTP connections In-Reply-To: References: <54B04B7D.2060502@ninenines.eu> Message-ID: Max, Lo?c, Thank you for confirming my thoughts. Best, r. > On 10/gen/2015, at 12:01, Max Lapshin wrote: > > terminate should never be used as a reliable callback. > > > Our production-tested gen_tracker registrator: https://github.com/erlyvideo/gen_tracker adds after_terminate callback > that will be called with some metadata _after_ process dies. > > But your case will be handled automatically in gen_tracker and in gproc. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 10 20:50:18 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 10 Jan 2015 11:50:18 -0800 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: Thanks a lot all, that was the only know I knew based on what I found in documentation, but I learnt lot other ways to do it. Plus, One thing I learnt is how to measure performance, so Thank you all for your time and effort + Harit On Sat, Jan 10, 2015 at 5:07 AM, Michael Uvarov wrote: > there's another way: > > reverse(Bin) when is_binary(Bin) -> > > S = bit_size(Bin), > > <> = Bin, > > <>. > > > > > On January 10, 2015 12:24:54 PM CET, "?????? ????????" > wrote: > >> Probably this one will be more effective >> >> reverse(Bin) -> >> reverse(Bin, byte_size(Bin)). >> >> reverse(_, 0) -> <<>>; >> reverse(Bin, Pos) -> >> <<(binary:at(Bin, Pos - 1)), (reverse(Bin, Pos - 1))/binary >>. >> >> ------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 10 20:54:20 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 10 Jan 2015 11:54:20 -0800 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <54B1436C.7060008@gmail.com> References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> Message-ID: This is very interesting. I do not want to divert the discussion somewhere else, but could someone point to resources about how exchange/trade systems work/implemented? I am learning Erlang and surely interested in building such systems for educational purposes. Apologies for diversion + Harit On Sat, Jan 10, 2015 at 7:21 AM, the.silly.sad wrote: > The failure/recovery semantics of Erlang, while great for many situations, >> might not be appropriate for an exchange - again, depending on the >> particulars of your use case. It might turn out that crashing and >> restarting is not appropriate for you. To put it in perspective, if you >> lose a message in a chat application it is bad, but not disastrous. If you >> lose a trade in an exchange, people lose money, and they tend to be _very_ >> unhappy about that. >> > > i believe this particular problem might be easily shifted up to the next > level of implementation. > > there is always an option to organize some sort of two-phase transactions > between clients, so that minimizing server's involvement... > > distribute calculations wherever possible :) > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowdhury918@REDACTED Sun Jan 11 02:24:07 2015 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Sat, 10 Jan 2015 20:24:07 -0500 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr Message-ID: All, I have to implement an websocket server in Erlang for video streaming. (An Android client will send continous snapshot to Erlang server and Erlang Server will send it to an User Interface which is developed in HTML5). For this Erlang server, what will be better? Mochiweb, cowboy or something different? What will scale better? Can someone please through some light on it? Any help will be highly appreciated. Thanks. - Akash -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Jan 11 03:41:52 2015 From: bob@REDACTED (Bob Ippolito) Date: Sun, 11 Jan 2015 13:41:52 +1100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: I've used cowboy for most of the websocket stuff I've done in Erlang. The websocket support in mochiweb is newer and less mature. I haven't done any benchmarks on it. On Sun, Jan 11, 2015 at 12:24 PM, Akash Chowdhury wrote: > All, > I have to implement an websocket server in Erlang for video streaming. (An > Android client will send continous snapshot to Erlang server and Erlang > Server will send it to an User Interface which is developed in HTML5). > For this Erlang server, what will be better? Mochiweb, cowboy or something > different? What will scale better? Can someone please through some light on > it? > Any help will be highly appreciated. > Thanks. > - > Akash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 11 03:46:30 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 11 Jan 2015 03:46:30 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: Yaws too handles websockets. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Jan 11 04:05:22 2015 From: ok@REDACTED (ok@REDACTED) Date: Sun, 11 Jan 2015 16:05:22 +1300 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: I'm on holiday, so haven't been following mail regularly. Why is reversing a binary useful? How many times is the reversed binary to be traversed? Is it practical to access the binary in reverse order instead, acting _as if_ it had been reversed? From harit.subscriptions@REDACTED Sun Jan 11 04:24:26 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 10 Jan 2015 19:24:26 -0800 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: Message-ID: If this question is for me @ok, this is one of the exercise in Programming Erlang 2nd Edition. I am learning Erlang and sincerely trying my solutions, but also asking questions were I am very sure my solution is not correct or efficient. Thank you + Harit Himanshu On Sat, Jan 10, 2015 at 7:05 PM, wrote: > I'm on holiday, so haven't been following mail regularly. > Why is reversing a binary useful? > How many times is the reversed binary to be traversed? > Is it practical to access the binary in reverse order > instead, acting _as if_ it had been reversed? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Jan 11 05:45:22 2015 From: bob@REDACTED (Bob Ippolito) Date: Sun, 11 Jan 2015 15:45:22 +1100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: After looking a bit more closely at this code (to write tests for and fix an issue) I would not recommend using mochiweb's implementation of websocket at this time. It's a contributed feature without many tests and it does not appear to implement the full RFC spec. I can also see some edge cases where incoming data might be lost (when websocket frames are fragmented across http packets). The only really good reason to use this implementation is if you really like mochiweb (perhaps for legacy reasons) *and* you are willing to fix this code if it's broken for your use case. On Sun, Jan 11, 2015 at 1:41 PM, Bob Ippolito wrote: > I've used cowboy for most of the websocket stuff I've done in Erlang. The > websocket support in mochiweb is newer and less mature. I haven't done any > benchmarks on it. > > On Sun, Jan 11, 2015 at 12:24 PM, Akash Chowdhury > wrote: > >> All, >> I have to implement an websocket server in Erlang for video streaming. >> (An Android client will send continous snapshot to Erlang server and Erlang >> Server will send it to an User Interface which is developed in HTML5). >> For this Erlang server, what will be better? Mochiweb, cowboy or >> something different? What will scale better? Can someone please through >> some light on it? >> Any help will be highly appreciated. >> Thanks. >> - >> Akash >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jan 11 07:13:34 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 11 Jan 2015 10:13:34 +0400 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: sorry, but how is websocket related to video streaming? -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 11 10:20:19 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 11 Jan 2015 10:20:19 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: Yaws has functions for streaming, too ;) From eric.pailleau@REDACTED Sun Jan 11 10:27:17 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 11 Jan 2015 10:27:17 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr Message-ID: Hi, we come back to still the same problem, what RFC (or part of) all those webservers are handling ? or claim to. ? Envoy? depuis mon mobile ? Eric Bob Ippolito a ?crit?: >After looking a bit more closely at this code (to write tests for and fix >an issue) I would not recommend using mochiweb's implementation of >websocket at this time. It's a contributed feature without many tests and >it does not appear to implement the full RFC spec. I can also see some edge >cases where incoming data might be lost (when websocket frames are >fragmented across http packets). The only really good reason to use this >implementation is if you really like mochiweb (perhaps for legacy reasons) >*and* you are willing to fix this code if it's broken for your use case. > >On Sun, Jan 11, 2015 at 1:41 PM, Bob Ippolito wrote: > >> I've used cowboy for most of the websocket stuff I've done in Erlang. The >> websocket support in mochiweb is newer and less mature. I haven't done any >> benchmarks on it. >> >> On Sun, Jan 11, 2015 at 12:24 PM, Akash Chowdhury > > wrote: >> >>> All, >>> I have to implement an websocket server in Erlang for video streaming. >>> (An Android client will send continous snapshot to Erlang server and Erlang >>> Server will send it to an User Interface which is developed in HTML5). >>> For this Erlang server, what will be better? Mochiweb, cowboy or >>> something different? What will scale better? Can someone please through >>> some light on it? >>> Any help will be highly appreciated. >>> Thanks. >>> - >>> Akash >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 11 10:50:57 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 11 Jan 2015 10:50:57 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: Yaws: rfc6455 (sockets) http://yaws.hyber.org/websockets.yaws rfc2326 (streaming) https://github.com/klacke/yaws/blob/master/src/yaws_api.erl From eric.pailleau@REDACTED Sun Jan 11 11:08:16 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 11 Jan 2015 11:08:16 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr Message-ID: Hi, it is probably possible to find this, yes, there or there, but MHO, each webserver developper team should maintain a clear table on a web page for this. Yaws'documentation, at least , contains clear reference to rfcs and witch section. This would help people to choose without having to read the code and rfc in parallel. One of Erlang power is its short development time. Having to dig in the code is not a gain of time. And all people are not web specialists to do so. regards ? Envoy? depuis mon mobile ? Eric Imants Cekusins a ?crit?: >Yaws: > >rfc6455 (sockets) >http://yaws.hyber.org/websockets.yaws > >rfc2326 (streaming) >https://github.com/klacke/yaws/blob/master/src/yaws_api.erl From essen@REDACTED Sun Jan 11 11:15:40 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sun, 11 Jan 2015 11:15:40 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: <54B24D4C.8020703@ninenines.eu> On 01/11/2015 11:08 AM, Eric Pailleau wrote: > Hi, > it is probably possible to find this, yes, there or there, but MHO, each webserver developper team should maintain a clear table on a web page for this. That's something I intend to fix from Cowboy 2 onward. I have made a wishlist so far (https://gist.github.com/essen/12eb360f0717911d94ce) and started noting down points from RFCs that Cowboy (will) support (e.g. https://github.com/ninenines/cowboy/blob/master/doc/src/specs/rfc7230_server.ezdoc). It will in time be a full RFC digest and each point will have a test. The one I link above is probably the biggest RFC, Cowboy only caring about header value syntax in many other RFCs. -- Lo?c Hoguin http://ninenines.eu From imantc@REDACTED Sun Jan 11 11:27:59 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 11 Jan 2015 11:27:59 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: <54B24D4C.8020703@ninenines.eu> References: <54B24D4C.8020703@ninenines.eu> Message-ID: Agree with Eric: documentation is very, very important for app / lib adoption. Apps without documentation are pretty much useless. From bchesneau@REDACTED Sun Jan 11 11:33:44 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sun, 11 Jan 2015 11:33:44 +0100 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: On Tue, Dec 16, 2014 at 11:33 PM, Max Lapshin wrote: > Hi, Andreas. > > I'm implementing server-side for webrtc now and I want know if it is > possible to work with you on this DTLS stuff? > > > Did you made same progress? I can help in review/tests if needed :) - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Jan 11 11:57:00 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 11 Jan 2015 11:57:00 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr Message-ID: Hi Lo?c, glad to read this. I add this will generate a sane emulation between each developper team/community. regards. ? Envoy? depuis mon mobile ? Eric Lo?c Hoguin a ?crit?: >On 01/11/2015 11:08 AM, Eric Pailleau wrote: >> Hi, >> it is probably possible to find this, yes, there or there, but MHO, each webserver developper team should maintain a clear table on a web page for this. > >That's something I intend to fix from Cowboy 2 onward. I have made a >wishlist so far (https://gist.github.com/essen/12eb360f0717911d94ce) and >started noting down points from RFCs that Cowboy (will) support (e.g. >https://github.com/ninenines/cowboy/blob/master/doc/src/specs/rfc7230_server.ezdoc). >It will in time be a full RFC digest and each point will have a test. > >The one I link above is probably the biggest RFC, Cowboy only caring >about header value syntax in many other RFCs. > >-- >Lo?c Hoguin >http://ninenines.eu From eric.pailleau@REDACTED Sun Jan 11 12:34:08 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sun, 11 Jan 2015 12:34:08 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr Message-ID: <7uc010ip9t6fwppnvsxm0pd5.1420975658021@email.android.com> Hi, forgot to say. the table should indicate from which version the rfc is handle. BTW it is a lack in Erlang OTP documentation : from which version a module/function is available ? this would help to define the minimal erts/otp version, i.e minimal erl version a project needs. a powerfull skill of eclipse when developping an Android app is to maintain autmatically the API version in the manifest, automagically, from class/method use. (Kostis, this could be a dialyzer killer feature :-) ....) ? Envoy? depuis mon mobile ? Eric Eric Pailleau a ?crit?: >Hi Lo?c, > >glad to read this. >I add this will generate a sane emulation between each developper team/community. >regards. > > >? Envoy? depuis mon mobile ? Eric > >Lo?c Hoguin a ?crit?: > >>On 01/11/2015 11:08 AM, Eric Pailleau wrote: >>> Hi, >>> it is probably possible to find this, yes, there or there, but MHO, each webserver developper team should maintain a clear table on a web page for this. >> >>That's something I intend to fix from Cowboy 2 onward. I have made a >>wishlist so far (https://gist.github.com/essen/12eb360f0717911d94ce) and >>started noting down points from RFCs that Cowboy (will) support (e.g. >>https://github.com/ninenines/cowboy/blob/master/doc/src/specs/rfc7230_server.ezdoc). >>It will in time be a full RFC digest and each point will have a test. >> >>The one I link above is probably the biggest RFC, Cowboy only caring >>about header value syntax in many other RFCs. >> >>-- >>Lo?c Hoguin >>http://ninenines.eu >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Sun Jan 11 12:41:35 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 11 Jan 2015 15:41:35 +0400 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: I haven't made any progress, yet, because it were extremely long new year holidays. Right now I'm looking at different C libraries, perhaps it would be possible to find out something from there. On Sun, Jan 11, 2015 at 1:33 PM, Benoit Chesneau wrote: > > > On Tue, Dec 16, 2014 at 11:33 PM, Max Lapshin > wrote: > >> Hi, Andreas. >> >> I'm implementing server-side for webrtc now and I want know if it is >> possible to work with you on this DTLS stuff? >> >> >> Did you made same progress? I can help in review/tests if needed :) > > - benoit > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sun Jan 11 14:21:56 2015 From: bchesneau@REDACTED (bchesneau@REDACTED) Date: Sun, 11 Jan 2015 14:21:56 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: <54B24D4C.8020703@ninenines.eu> Message-ID: > On 11 Jan 2015, at 11:27, Imants Cekusins wrote: > > Agree with Eric: documentation is very, very important for app / lib adoption. > > Apps without documentation are pretty much useless. And any help and feedback to improve a documentation is always appreciated. - benoit From mihai@REDACTED Sun Jan 11 14:46:48 2015 From: mihai@REDACTED (Mihai Balea) Date: Sun, 11 Jan 2015 08:46:48 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> Message-ID: <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> > On Jan 10, 2015, at 2:54 PM, Harit Himanshu wrote: > > This is very interesting. I do not want to divert the discussion somewhere else, but could someone point to resources about how exchange/trade systems work/implemented? I am learning Erlang and surely interested in building such systems for educational purposes. > > Apologies for diversion > + Harit Here?s one article about the subject: http://martinfowler.com/articles/lmax.html It has nothing to do with Erlang - my apologies to the list - but it is very interesting, from a technical point of view. Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sun Jan 11 15:59:46 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sun, 11 Jan 2015 06:59:46 -0800 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: Thanks Mihai I will take a look Thanks On Sun, Jan 11, 2015 at 5:46 AM, Mihai Balea wrote: > > On Jan 10, 2015, at 2:54 PM, Harit Himanshu > wrote: > > This is very interesting. I do not want to divert the discussion somewhere > else, but could someone point to resources about how exchange/trade systems > work/implemented? I am learning Erlang and surely interested in building > such systems for educational purposes. > > Apologies for diversion > + Harit > > > Here?s one article about the subject: > http://martinfowler.com/articles/lmax.html > It has nothing to do with Erlang - my apologies to the list - but it is > very interesting, from a technical point of view. > > Mihai > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sun Jan 11 16:09:10 2015 From: t@REDACTED (Tristan Sloughter) Date: Sun, 11 Jan 2015 09:09:10 -0600 Subject: [erlang-questions] Systools and Release Handler Message-ID: <1420988950.3910887.212436217.6912B5A1@webmail.messagingengine.com> I'm hoping the OTP team can shed some light on what I consider an oddity in systools and release_handler. I'm only bringing up one at this time, it is the most often brought up confusion for users when using relx that traces back to systools and release_handler. systools make_script creates a .boot file with the name of the release. When make_tar is called on the release that boot script is renamed start.boot release_handler relies on it being named start.boot This means that a script for starting a node must check "does RelName.boot exist?" to know which name to pass to -boot. A small amount of code, but why bother ever having the boot script not named start.boot? There are some other confusing points when it comes to release_handler but I'll save those for later (mostly because I haven't kept track). But besides simply "why" I'm also wondering what the tools you use do to handle this? Separate ways of starting in development than deployment? Switches in scripts like I mentioned? And would modifications to systools to always use one name or the other be accepted? -- Tristan Sloughter t@REDACTED From imantc@REDACTED Sun Jan 11 16:52:11 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 11 Jan 2015 16:52:11 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: <54B24D4C.8020703@ninenines.eu> Message-ID: > help and feedback to improve a documentation would this help: is documentation for domestic appliances - a good example? 1) intro: what the appliance (app) does; 2) how to unpack (install) and configure; 3) what every button on the remote (api function) does, ideally with examples, ideally grouped by topics; 4) how to tell if something goes wrong (where to look for log and how to interpret error messages) and what to do then. Not to point at any specific product, but some apps skip a few of these topics. From harit.subscriptions@REDACTED Sun Jan 11 20:48:56 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sun, 11 Jan 2015 11:48:56 -0800 Subject: [erlang-questions] Ring Benchmark Problem Message-ID: Hello there, I need help with code review on my attempt to following problem Write a ring benchmark. Create N processes in a ring. Send a message round the ring M times so that a total of N * M messages get sent. Time how long this takes for different values of N and M. I have not timed this yet, and guidance on recommended ways to time is very much appreciated. For now, following is the code for ring that I wrote. Please let me know if there are any shortcomings or code could be written in an much idiomatic way. Thanks a lot + Harit *Code* -module(ring). -author("harith"). %% API -export([message/2]). % create ring of N processes and % send M messages between them message(N, M) when is_integer(N), is_integer(M), N > 0, M > 0 -> Ring = create_ring(N), [Start | T] = Ring, Start ! {T, Ring, 1, M}. create_ring(N) -> Processes = [spawn(fun() -> loop() end) || _ <- lists:seq(1, N)], [H | _] = Processes, lists:append(Processes, [H]). loop() -> receive {[H | T], _L, CurrentMessage, M} -> io:format("~p received ~p~n", [self(), CurrentMessage]), H ! {T, _L, CurrentMessage, M}, loop(); {[], Ring, CurrentMessage, M} -> io:format("~p received ~p with empty list~n", [self(), CurrentMessage]), case CurrentMessage < M of true -> [_ | [Next | T]] = Ring, NewMessage = CurrentMessage + 1, io:format("sending message ~p to ~p~n", [NewMessage, Next]), Next ! {T, Ring, NewMessage, M}; false -> io:format("done sending ~p messages in ~p ring, taking rest now.~n",[M, Ring]) end, loop() end. *Output* 1> ring:message(4, 3). <0.33.0> received 1 {[<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], 1,3} <0.34.0> received 1 <0.35.0> received 1 2> <0.36.0> received 1 2> <0.33.0> received 1 with empty list 2> sending message 2 to <0.34.0> 2> <0.34.0> received 2 2> <0.35.0> received 2 2> <0.36.0> received 2 2> <0.33.0> received 2 with empty list 2> sending message 3 to <0.34.0> 2> <0.34.0> received 3 2> <0.35.0> received 3 2> <0.36.0> received 3 2> <0.33.0> received 3 with empty list 2> done sending 3 messages in [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>] ring, taking rest now. 2> -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoine.koener@REDACTED Mon Jan 12 09:05:58 2015 From: antoine.koener@REDACTED (Antoine Koener) Date: Mon, 12 Jan 2015 09:05:58 +0100 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: Maybe, if you don't know about this project, you should have a look to: https://code.google.com/p/telepresence/ http://conf-call.org which is an open source realtime video multiplexer using webrtc and is compatible with any sip client, The build instructions is located at http://conf-call.org/technical-guide.pdf?svn=2 I've successfully built it and tested it, but I've didn't made any real benchmarks for now... On Sun, Jan 11, 2015 at 12:41 PM, Max Lapshin wrote: > I haven't made any progress, yet, because it were extremely long new year > holidays. > > Right now I'm looking at different C libraries, perhaps it would be > possible to find out something from there. > > > On Sun, Jan 11, 2015 at 1:33 PM, Benoit Chesneau > wrote: > >> >> >> On Tue, Dec 16, 2014 at 11:33 PM, Max Lapshin >> wrote: >> >>> Hi, Andreas. >>> >>> I'm implementing server-side for webrtc now and I want know if it is >>> possible to work with you on this DTLS stuff? >>> >>> >>> Did you made same progress? I can help in review/tests if needed :) >> >> - benoit >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Mon Jan 12 09:52:02 2015 From: davidnwelton@REDACTED (David Welton) Date: Mon, 12 Jan 2015 09:52:02 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: > sorry, but how is websocket related to video streaming? To quote Bill and Ted's Excellent Adventure, "listen to this dude Max - he knows what he's talking about!". He's very knowledgeable about video streaming, having created: http://erlyvideo.org/ Cowboy does web sockets fine, but are you sure you're approaching the problem the right way? -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From tty.erlang@REDACTED Mon Jan 12 10:10:00 2015 From: tty.erlang@REDACTED (T Ty) Date: Mon, 12 Jan 2015 09:10:00 +0000 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: The LMAX architecture is an easy fit for Erlang and one which an Erlanger would naturally arrive to. In terms of 'Is Erlang ideal for a global exchange' I would say yes it is. However what is more important is knowing the transaction profile and a rough idea of the language agnostic architecture that fits that transaction profile. In many cases the mapping from this agnostic architecture to Erlang is seamless and if it isn't seamless then you are looking at another implementation language. On Sun, Jan 11, 2015 at 2:59 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > Thanks Mihai > > I will take a look > > Thanks > > On Sun, Jan 11, 2015 at 5:46 AM, Mihai Balea wrote: > >> >> On Jan 10, 2015, at 2:54 PM, Harit Himanshu < >> harit.subscriptions@REDACTED> wrote: >> >> This is very interesting. I do not want to divert the discussion >> somewhere else, but could someone point to resources about how >> exchange/trade systems work/implemented? I am learning Erlang and surely >> interested in building such systems for educational purposes. >> >> Apologies for diversion >> + Harit >> >> >> Here?s one article about the subject: >> http://martinfowler.com/articles/lmax.html >> It has nothing to do with Erlang - my apologies to the list - but it is >> very interesting, from a technical point of view. >> >> Mihai >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Jan 12 10:23:41 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 12 Jan 2015 10:23:41 +0100 Subject: [erlang-questions] mochiweb vs. cowboy for websocket serevr In-Reply-To: References: Message-ID: On Mon, Jan 12, 2015 at 9:52 AM, David Welton wrote: > > sorry, but how is websocket related to video streaming? > > To quote Bill and Ted's Excellent Adventure, "listen to this dude Max > - he knows what he's talking about!". He's very knowledgeable about > video streaming, having created: http://erlyvideo.org/ > > Cowboy does web sockets fine, but are you sure you're approaching the > problem the right way? > Maybe people could have different ideas though ;) I would rather ask for which purpose the websockets will be used. - benoit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 11:40:04 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 11:40:04 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 Message-ID: Dear list, I've probably missed out something since I'm not understanding what is going here. I'm basically just trying to send messages periodically, though I found out that I wasn't seeing the behavior that I was expecting. It looks like the timer has some erratic behavior. I've stripped out the code to the bare minimum and it looks like this: -module(test). -export([sleep_periodical/2]). -export([sleep_loop/2]). sleep_periodical(Num, IntervalMs) -> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), io:format("Finished in ~p microseconds.~n", [Time]). sleep_loop(Num, IntervalMs) -> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, lists:foreach(SleepFun, lists:seq(1, Num)). When I run this code, I see the following: Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V6.3 (abort with ^G) 1> c(test). {ok,test} 2> test:sleep_periodical(1000, 10). Finished in 12257397 microseconds. ok 3> test:sleep_periodical(2000, 10). Finished in 24518070 microseconds. ok 4> test:sleep_periodical(10000, 1). Finished in 20000280 microseconds. ok 5> test:sleep_periodical(20000, 1). Finished in 40000685 microseconds. ok So: - Looping 1,000 times and sleeping 10 ms each time takes 12.26 seconds (instead of 10). - Looping 2,000 times and sleeping 10 ms each time takes 24.52 seconds (instead of 20). - Looping 10,000 times and sleeping 1 ms each time takes 20.00 seconds (instead of 10). - Looping 20,000 times and sleeping 1 ms each time takes 40.00 seconds (instead of 20). Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it takes 100% (double). Can some kind soul explain what I'm doing wrong here and how to achieve the desired results? I'm on OSX, Erlang 17.4. Help! ^^_ r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 11:46:40 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 11:46:40 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Additionally: 6> test:sleep_periodical(1000, 1). Finished in 1999097 microsecond So: - Looping 1,000 times and sleeping 1 ms each time takes 2.00 seconds (instead of 1). Looks like it's ms related, not total count. Best, r. On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli wrote: > Dear list, > I've probably missed out something since I'm not understanding what is > going here. > > I'm basically just trying to send messages periodically, though I found > out that I wasn't seeing the behavior that I was expecting. It looks like > the timer has some erratic behavior. > > I've stripped out the code to the bare minimum and it looks like this: > > -module(test). > -export([sleep_periodical/2]). > -export([sleep_loop/2]). > > sleep_periodical(Num, IntervalMs) -> > {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), > io:format("Finished in ~p microseconds.~n", [Time]). > > sleep_loop(Num, IntervalMs) -> > SleepFun = fun(_) -> timer:sleep(IntervalMs) end, > lists:foreach(SleepFun, lists:seq(1, Num)). > > > When I run this code, I see the following: > > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V6.3 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:sleep_periodical(1000, 10). > Finished in 12257397 microseconds. > ok > 3> test:sleep_periodical(2000, 10). > Finished in 24518070 microseconds. > ok > 4> test:sleep_periodical(10000, 1). > Finished in 20000280 microseconds. > ok > 5> test:sleep_periodical(20000, 1). > Finished in 40000685 microseconds. > ok > > So: > > - Looping 1,000 times and sleeping 10 ms each time takes 12.26 seconds > (instead of 10). > - Looping 2,000 times and sleeping 10 ms each time takes 24.52 seconds > (instead of 20). > - Looping 10,000 times and sleeping 1 ms each time takes 20.00 seconds > (instead of 10). > - Looping 20,000 times and sleeping 1 ms each time takes 40.00 seconds > (instead of 20). > > Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it > takes 100% (double). > > Can some kind soul explain what I'm doing wrong here and how to achieve > the desired results? > > I'm on OSX, Erlang 17.4. > > Help! ^^_ > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jan 12 12:08:22 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 12 Jan 2015 15:08:22 +0400 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.dephily@REDACTED Mon Jan 12 12:10:39 2015 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 12 Jan 2015 12:10:39 +0100 Subject: [erlang-questions] Reverses Order of Bytes in Binary In-Reply-To: References: <54AFE0C5.2080609@ninenines.eu> Message-ID: <2718842.kXzbkYpmtH@moltowork> On Friday 09 January 2015 18:18:26 Max Lapshin wrote: > Ah, really. I should be ashamed. > > reverse(Bin) -> > reverse(Bin, []). > > reverse(<<>>, Acc) -> list_to_binary(Acc); > reverse(<>, Acc) -> reverse(Bin, [C|Acc]). I microbenched various implementations with small and large binaries trying to get rid of the intermediate list representation and was really surprised by the results : l(B) -> list_to_binary(lists:reverse(binary_to_list(B))). b(Bin) -> b(Bin, []). b(<<>>, Acc) -> list_to_binary(Acc); b(<>, Acc) -> b(Bin, [C|Acc]). b1(Bin) -> b1(Bin, <<>>). b1(<<>>, Acc) -> Acc; b1(<>, Acc) -> b1(Bin, <>). b2(B) -> b2(B,byte_size(B)-1,<<>>). b2(B,0,Acc) -> <>; b2(B,N,Acc) -> b2(B,N-1,<>). I consistently get l/1 fastest: l 10225 b 14247 b1 339329 b2 50249 I'm surprised that the list/binary conversions are that cheap (compared to directly parsing/building the binary) because of all the small memory allocations they need to do. Is there some native optimisation they use that plain erlang can't use ? Typically, does list_to_binary/1 construct the binary in a more efficient way than is possible from plain erlang ? When decontructing l/1 into its 3 steps I find that binary_to_list takes just slightly longer than reverse, and list_to_binary takes a good bit less. For what it's worth, binary:reverse/1 could probably be much faster if implemented as a bif. I wonder if it'd be worth it to have some kind of binary:preallocate/2 function. -- Vincent de Phily From sergej.jurecko@REDACTED Mon Jan 12 12:12:47 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 12 Jan 2015 12:12:47 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: timer:sleep is not even remotely as accurate for those low numbers. It can not go lower than 20ms or so from what i remember. I have had this issue once when I needed low interval for timer. I used an external c app which printed to stdout every 5ms. I dont think there is any way in erlang to get a timer that low. Sergej On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" wrote: > Additionally: > > 6> test:sleep_periodical(1000, 1). > Finished in 1999097 microsecond > > So: > > - Looping 1,000 times and sleeping 1 ms each time takes 2.00 seconds > (instead of 1). > > Looks like it's ms related, not total count. > > Best, > r. > > > > On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli > wrote: > >> Dear list, >> I've probably missed out something since I'm not understanding what is >> going here. >> >> I'm basically just trying to send messages periodically, though I found >> out that I wasn't seeing the behavior that I was expecting. It looks like >> the timer has some erratic behavior. >> >> I've stripped out the code to the bare minimum and it looks like this: >> >> -module(test). >> -export([sleep_periodical/2]). >> -export([sleep_loop/2]). >> >> sleep_periodical(Num, IntervalMs) -> >> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >> io:format("Finished in ~p microseconds.~n", [Time]). >> >> sleep_loop(Num, IntervalMs) -> >> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >> lists:foreach(SleepFun, lists:seq(1, Num)). >> >> >> When I run this code, I see the following: >> >> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] >> [hipe] [kernel-poll:false] [dtrace] >> >> Eshell V6.3 (abort with ^G) >> 1> c(test). >> {ok,test} >> 2> test:sleep_periodical(1000, 10). >> Finished in 12257397 microseconds. >> ok >> 3> test:sleep_periodical(2000, 10). >> Finished in 24518070 microseconds. >> ok >> 4> test:sleep_periodical(10000, 1). >> Finished in 20000280 microseconds. >> ok >> 5> test:sleep_periodical(20000, 1). >> Finished in 40000685 microseconds. >> ok >> >> So: >> >> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >> seconds (instead of 10). >> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >> seconds (instead of 20). >> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >> seconds (instead of 10). >> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >> seconds (instead of 20). >> >> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it >> takes 100% (double). >> >> Can some kind soul explain what I'm doing wrong here and how to achieve >> the desired results? >> >> I'm on OSX, Erlang 17.4. >> >> Help! ^^_ >> r. >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Mon Jan 12 12:19:02 2015 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 12 Jan 2015 13:19:02 +0200 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: <8F54BA6C-5EC5-4F35-88C4-246129AECB76@gmail.com> Hello, timer:sleep() is implemented as receive after T -> ok end. It does not give you microsecond precision so that error is aggregated. secondly, lists:foreach(SleepFun, lists:seq(1, Num)) does not look performance friendly for big Num try tail recursion it gives better results. sleep_loop(0, _) -> ok; sleep_loop(N, IntervalMs) -> receive after IntervalMs -> ok end, sleep_loop(N - 1, IntervalMs). - Dmitry > On 12 Jan 2015, at 12:40, Roberto Ostinelli wrote: > > Dear list, > I've probably missed out something since I'm not understanding what is going here. > > I'm basically just trying to send messages periodically, though I found out that I wasn't seeing the behavior that I was expecting. It looks like the timer has some erratic behavior. > > I've stripped out the code to the bare minimum and it looks like this: > > -module(test). > -export([sleep_periodical/2]). > -export([sleep_loop/2]). > > sleep_periodical(Num, IntervalMs) -> > {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), > io:format("Finished in ~p microseconds.~n", [Time]). > > sleep_loop(Num, IntervalMs) -> > SleepFun = fun(_) -> timer:sleep(IntervalMs) end, > lists:foreach(SleepFun, lists:seq(1, Num)). > > > When I run this code, I see the following: > > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V6.3 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:sleep_periodical(1000, 10). > Finished in 12257397 microseconds. > ok > 3> test:sleep_periodical(2000, 10). > Finished in 24518070 microseconds. > ok > 4> test:sleep_periodical(10000, 1). > Finished in 20000280 microseconds. > ok > 5> test:sleep_periodical(20000, 1). > Finished in 40000685 microseconds. > ok > > So: > Looping 1,000 times and sleeping 10 ms each time takes 12.26 seconds (instead of 10). > Looping 2,000 times and sleeping 10 ms each time takes 24.52 seconds (instead of 20). > Looping 10,000 times and sleeping 1 ms each time takes 20.00 seconds (instead of 10). > Looping 20,000 times and sleeping 1 ms each time takes 40.00 seconds (instead of 20). > Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it takes 100% (double). > > Can some kind soul explain what I'm doing wrong here and how to achieve the desired results? > > I'm on OSX, Erlang 17.4. > > Help! ^^_ > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 12:20:06 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 12:20:06 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Indeed. From very basic try-outs: 1> test:sleep_periodical(6, 1000). Finished in 6022381 microseconds. ok 2> test:sleep_periodical(12, 500). Finished in 6042683 microseconds. ok 3> test:sleep_periodical(24, 250). Finished in 6062096 microseconds. ok 4> test:sleep_periodical(48, 125). Finished in 6133169 microseconds. ok 5> test:sleep_periodical(96, 62). Finished in 6320419 microseconds. ok 6> test:sleep_periodical(192, 31). Finished in 6492321 microseconds. ok 7> test:sleep_periodical(384, 16). Finished in 7327693 microseconds. ok So it somewhat is ok until we get in the 125ms range. On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko wrote: > timer:sleep is not even remotely as accurate for those low numbers. It can > not go lower than 20ms or so from what i remember. > > I have had this issue once when I needed low interval for timer. I used an > external c app which printed to stdout every 5ms. I dont think there is any > way in erlang to get a timer that low. > > Sergej > On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" wrote: > >> Additionally: >> >> 6> test:sleep_periodical(1000, 1). >> Finished in 1999097 microsecond >> >> So: >> >> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 seconds >> (instead of 1). >> >> Looks like it's ms related, not total count. >> >> Best, >> r. >> >> >> >> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli >> wrote: >> >>> Dear list, >>> I've probably missed out something since I'm not understanding what is >>> going here. >>> >>> I'm basically just trying to send messages periodically, though I found >>> out that I wasn't seeing the behavior that I was expecting. It looks like >>> the timer has some erratic behavior. >>> >>> I've stripped out the code to the bare minimum and it looks like this: >>> >>> -module(test). >>> -export([sleep_periodical/2]). >>> -export([sleep_loop/2]). >>> >>> sleep_periodical(Num, IntervalMs) -> >>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>> io:format("Finished in ~p microseconds.~n", [Time]). >>> >>> sleep_loop(Num, IntervalMs) -> >>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>> lists:foreach(SleepFun, lists:seq(1, Num)). >>> >>> >>> When I run this code, I see the following: >>> >>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] >>> [hipe] [kernel-poll:false] [dtrace] >>> >>> Eshell V6.3 (abort with ^G) >>> 1> c(test). >>> {ok,test} >>> 2> test:sleep_periodical(1000, 10). >>> Finished in 12257397 microseconds. >>> ok >>> 3> test:sleep_periodical(2000, 10). >>> Finished in 24518070 microseconds. >>> ok >>> 4> test:sleep_periodical(10000, 1). >>> Finished in 20000280 microseconds. >>> ok >>> 5> test:sleep_periodical(20000, 1). >>> Finished in 40000685 microseconds. >>> ok >>> >>> So: >>> >>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>> seconds (instead of 10). >>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>> seconds (instead of 20). >>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>> seconds (instead of 10). >>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>> seconds (instead of 20). >>> >>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it >>> takes 100% (double). >>> >>> Can some kind soul explain what I'm doing wrong here and how to achieve >>> the desired results? >>> >>> I'm on OSX, Erlang 17.4. >>> >>> Help! ^^_ >>> r. >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Jan 12 12:43:06 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 12 Jan 2015 11:43:06 +0000 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 References: Message-ID: Hi Roberto, I'm not sure what you would expect here. timer:sleep/1 has an implementation like this: sleep(T) -> receive after T -> ok end. which is what other people told you also. In general, such timing constructs have no guarantee about when they will fire, only that they will fire after the time has elapsed. So what you are seeing is somewhat expected behaviour. The more fine-grained and the more timers you have, the less precise things will get. In a way, this is one of the properties which makes Erlang a soft-realtime system and not a hard-realtime one, although the precise definition of soft/hard realtime are subtly different. I believe there were some effort being made by the OTP team to address lock contention around the timer wheel, so the code might change behaviour by quite a lot in the future. It might even have to get worse for single-process timers in order to become better for multi-process timers. On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli wrote: > Indeed. From very basic try-outs: > > 1> test:sleep_periodical(6, 1000). > Finished in 6022381 microseconds. > ok > 2> test:sleep_periodical(12, 500). > Finished in 6042683 microseconds. > ok > 3> test:sleep_periodical(24, 250). > Finished in 6062096 microseconds. > ok > 4> test:sleep_periodical(48, 125). > Finished in 6133169 microseconds. > ok > 5> test:sleep_periodical(96, 62). > Finished in 6320419 microseconds. > ok > 6> test:sleep_periodical(192, 31). > Finished in 6492321 microseconds. > ok > 7> test:sleep_periodical(384, 16). > Finished in 7327693 microseconds. > ok > > So it somewhat is ok until we get in the 125ms range. > > > > On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko > wrote: > >> timer:sleep is not even remotely as accurate for those low numbers. It >> can not go lower than 20ms or so from what i remember. >> >> I have had this issue once when I needed low interval for timer. I used >> an external c app which printed to stdout every 5ms. I dont think there is >> any way in erlang to get a timer that low. >> >> Sergej >> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >> wrote: >> >>> Additionally: >>> >>> 6> test:sleep_periodical(1000, 1). >>> Finished in 1999097 microsecond >>> >>> So: >>> >>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 seconds >>> (instead of 1). >>> >>> Looks like it's ms related, not total count. >>> >>> Best, >>> r. >>> >>> >>> >>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli >> > wrote: >>> >>>> Dear list, >>>> I've probably missed out something since I'm not understanding what is >>>> going here. >>>> >>>> I'm basically just trying to send messages periodically, though I found >>>> out that I wasn't seeing the behavior that I was expecting. It looks like >>>> the timer has some erratic behavior. >>>> >>>> I've stripped out the code to the bare minimum and it looks like this: >>>> >>>> -module(test). >>>> -export([sleep_periodical/2]). >>>> -export([sleep_loop/2]). >>>> >>>> sleep_periodical(Num, IntervalMs) -> >>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>> >>>> sleep_loop(Num, IntervalMs) -> >>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>> >>>> >>>> When I run this code, I see the following: >>>> >>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] >>>> [hipe] [kernel-poll:false] [dtrace] >>>> >>>> Eshell V6.3 (abort with ^G) >>>> 1> c(test). >>>> {ok,test} >>>> 2> test:sleep_periodical(1000, 10). >>>> Finished in 12257397 microseconds. >>>> ok >>>> 3> test:sleep_periodical(2000, 10). >>>> Finished in 24518070 microseconds. >>>> ok >>>> 4> test:sleep_periodical(10000, 1). >>>> Finished in 20000280 microseconds. >>>> ok >>>> 5> test:sleep_periodical(20000, 1). >>>> Finished in 40000685 microseconds. >>>> ok >>>> >>>> So: >>>> >>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>> seconds (instead of 10). >>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>> seconds (instead of 20). >>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>> seconds (instead of 10). >>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>> seconds (instead of 20). >>>> >>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>> it takes 100% (double). >>>> >>>> Can some kind soul explain what I'm doing wrong here and how to achieve >>>> the desired results? >>>> >>>> I'm on OSX, Erlang 17.4. >>>> >>>> Help! ^^_ >>>> r. >>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Mon Jan 12 14:02:58 2015 From: community-manager@REDACTED (Bruce Yinhe) Date: Mon, 12 Jan 2015 14:02:58 +0100 Subject: [erlang-questions] New Erlang job openings Message-ID: Hi, Erlang Central has currently 42 Erlang-related job openings. https://erlangcentral.org/jobs To subscribe for weekly updates on new Erlang jobs - Register on Erlang Central. https://erlangcentral.org/login/?action=register To publish your Erlang job openings free of charge: https://erlangcentral.org/jobs/add *Europe:* UK Home Based Role - Middleware Developer at bet365 - Stoke-on-Trent, UK https://erlangcentral.org/uk-home-based-role-middleware-developer-bet365/ Lecturer/Senior Lecturer/Reader/Chair at University of Sheffield - Sheffield, UK https://erlangcentral.org/lecturer-senior-lecturer-reader-chair-university-of-sheffield/ Software developer at Oxford Knight - London, UK https://erlangcentral.org/software-developer-java-c-scala-clojure-erlang-functional-programming-london/ Erlang Specialist at Modis - Rickmansworth, Hertfordshire, UK https://erlangcentral.org/erlang-specialist-modis/ Erlang developer at IoP - anywhere https://erlangcentral.org/erlang-developer-ambitious-startup-internet-of-people/ Experienced Erlang developer for an ambitious Startup at Startors - Remote work possible https://erlangcentral.org/experienced-erlang-developer-for-an-ambitious-startup/ Senior Software Engineer ? Core Tech at DemonWare - Dublin, Ireland https://erlangcentral.org/senior-software-engineer-core-tech-demonware/ Software Engineer - Build Engineering at DemonWare - Dublin, Ireland https://erlangcentral.org/software-engineer-build-engineering-demonware/ Senior Engineer with DevOps Focus at Campanja - Stockholm, Sweden https://erlangcentral.org/senior-engineer-with-devops-focus-campanja/ Senior Erlang Software Engineer at Cisco - Stockholm, Sweden https://erlangcentral.org/senior-erlang-software-engineer-cisco-tail-f/ Backend Engineer at Scycho - Stockholm, Sweden https://erlangcentral.org/backend-engineer-scypho/ Backend Engineer at Centralway - Zurich, Switzerland https://erlangcentral.org/backend-engineer-centralway/ Back-end Developer at NGTI - Rotterdam, The Netherlands https://erlangcentral.org/back-end-developer-ngti-nl/ *Americas:* Erlang Developer at Sqor Sports - San Francisco, CA https://erlangcentral.org/erlang-developer-sqor/ Erlang Developer at Inaka - Buenos Aires, Argentina https://erlangcentral.org/erlang-developer-inaka/ Infrastructure Engineer at Getaround - San Francisco, CA https://erlangcentral.org/infrastructure-engineer-getaround/ Senior Software Engineer, Erlang Specialist at AdRoll - San Francisco, CA https://erlangcentral.org/senior-software-engineer-erlang-specialist-adroll/ Backend Engineer at issuu - Palo Alto, CA https://erlangcentral.org/backend-engineer-issuu/ Senior Erlang Engineer at Machine Zone - Palo Alto, CA https://erlangcentral.org/senior-erlang-engineer-machine-zone/ Erlang Back End Engineer at TigerText - Santa Monica, CA https://erlangcentral.org/erlang-developer-tigertext/ Senior Software Engineer ? Distributed Systems at Couchbase - Mountain View, CA https://erlangcentral.org/senior-software-engineer-distributed-systems-couchbase/ Backend Engineer ? Relo Offered at CyberCoders - Los Angeles, CA; Santa Monica, CA https://erlangcentral.org/backend-engineer-relo-offered-erlang-java-otp-python-rubyonrails-elixir-cybercoders/ Elixir / Erlang Developer at Vitamin Talent - Austin, Texas (open to remote) https://erlangcentral.org/elixir-erlang-developer-vitamin-talent/ *Asia & Oceania * Software Engineer ? New Grad at ShoreTel - Canberra, Australia https://erlangcentral.org/software-engineer-new-grad-shoretel/ Lead Cloud Solution Architect at Chaatz Limited - Wan Chai, Hong Kong https://erlangcentral.org/lead-cloud-solution-architect-chaatz/ System Analyst (Erlang software engineer) at Chaatz Limited - Wan Chai,Hong Kong https://erlangcentral.org/system-analyst-erlang-software-engineer-chaatz/ Cheers, Bruce Yinhe ErlangCentral.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Mon Jan 12 14:10:19 2015 From: darach@REDACTED (Darach Ennis) Date: Mon, 12 Jan 2015 13:10:19 +0000 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Hi Roberto, If you want to avoid clock slewing or you need a timely and periodic event source you can achieve this somewhat as follows, although the right solution will of course depend on multiple factors. -define(INTERVAL, 1000). % One second main(_) -> loop(now_ms(),0). trigger(Now,Delta,Count) -> io:format("At: ~p drift:~p rate:~p/sec~n", [Now, Delta - ?INTERVAL, Count]). loop(Epoch,Count) -> Now = now_ms(), Delta = Now - Epoch, case Delta >= ?INTERVAL of true -> trigger(Now,Delta,Count), loop(Epoch+?INTERVAL,0); false -> loop(Epoch,Count+1) end. now_ms() -> {Mega,Secs,Micros} = os:timestamp(), (Mega*1000000 + Secs)*1000 + (Micros div 1000). This loops at a rate of ~7 million loops/second on my laptop and fires without slew or significant drift consistently calling the trigger/3 function every second. The rub is I burn a little more core/process for the privilege than with mechanism that induce slew such as with timer or erlang:send_after directly. C'est la vie. Accuracy and timeliness aren't free. In a gen_server you could use a combination of erlang:send_after and a clock based on os:timestamp and an epoch as above so that you end up with a clock that is 'accurate enough' for most use cases, and cheap enough for whatever value you ascribe to such tradeoffs given the specifics of your use case. Your mileage may vary. Cheers, Darach. On Mon, Jan 12, 2015 at 11:43 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi Roberto, > > I'm not sure what you would expect here. timer:sleep/1 has an > implementation like this: > > sleep(T) -> > receive > after T -> ok > end. > > which is what other people told you also. In general, such timing > constructs have no guarantee about when they will fire, only that they will > fire after the time has elapsed. So what you are seeing is somewhat > expected behaviour. The more fine-grained and the more timers you have, the > less precise things will get. In a way, this is one of the properties which > makes Erlang a soft-realtime system and not a hard-realtime one, although > the precise definition of soft/hard realtime are subtly different. > > I believe there were some effort being made by the OTP team to address > lock contention around the timer wheel, so the code might change behaviour > by quite a lot in the future. It might even have to get worse for > single-process timers in order to become better for multi-process timers. > > > On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli > wrote: > >> Indeed. From very basic try-outs: >> >> 1> test:sleep_periodical(6, 1000). >> Finished in 6022381 microseconds. >> ok >> 2> test:sleep_periodical(12, 500). >> Finished in 6042683 microseconds. >> ok >> 3> test:sleep_periodical(24, 250). >> Finished in 6062096 microseconds. >> ok >> 4> test:sleep_periodical(48, 125). >> Finished in 6133169 microseconds. >> ok >> 5> test:sleep_periodical(96, 62). >> Finished in 6320419 microseconds. >> ok >> 6> test:sleep_periodical(192, 31). >> Finished in 6492321 microseconds. >> ok >> 7> test:sleep_periodical(384, 16). >> Finished in 7327693 microseconds. >> ok >> >> So it somewhat is ok until we get in the 125ms range. >> >> >> >> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >> sergej.jurecko@REDACTED> wrote: >> >>> timer:sleep is not even remotely as accurate for those low numbers. It >>> can not go lower than 20ms or so from what i remember. >>> >>> I have had this issue once when I needed low interval for timer. I used >>> an external c app which printed to stdout every 5ms. I dont think there is >>> any way in erlang to get a timer that low. >>> >>> Sergej >>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>> wrote: >>> >>>> Additionally: >>>> >>>> 6> test:sleep_periodical(1000, 1). >>>> Finished in 1999097 microsecond >>>> >>>> So: >>>> >>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>> seconds (instead of 1). >>>> >>>> Looks like it's ms related, not total count. >>>> >>>> Best, >>>> r. >>>> >>>> >>>> >>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>> roberto@REDACTED> wrote: >>>> >>>>> Dear list, >>>>> I've probably missed out something since I'm not understanding what is >>>>> going here. >>>>> >>>>> I'm basically just trying to send messages periodically, though I >>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>> like the timer has some erratic behavior. >>>>> >>>>> I've stripped out the code to the bare minimum and it looks like this: >>>>> >>>>> -module(test). >>>>> -export([sleep_periodical/2]). >>>>> -export([sleep_loop/2]). >>>>> >>>>> sleep_periodical(Num, IntervalMs) -> >>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>> >>>>> sleep_loop(Num, IntervalMs) -> >>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>> >>>>> >>>>> When I run this code, I see the following: >>>>> >>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>> >>>>> Eshell V6.3 (abort with ^G) >>>>> 1> c(test). >>>>> {ok,test} >>>>> 2> test:sleep_periodical(1000, 10). >>>>> Finished in 12257397 microseconds. >>>>> ok >>>>> 3> test:sleep_periodical(2000, 10). >>>>> Finished in 24518070 microseconds. >>>>> ok >>>>> 4> test:sleep_periodical(10000, 1). >>>>> Finished in 20000280 microseconds. >>>>> ok >>>>> 5> test:sleep_periodical(20000, 1). >>>>> Finished in 40000685 microseconds. >>>>> ok >>>>> >>>>> So: >>>>> >>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>> seconds (instead of 10). >>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>> seconds (instead of 20). >>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>> seconds (instead of 10). >>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>> seconds (instead of 20). >>>>> >>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>>> it takes 100% (double). >>>>> >>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>> achieve the desired results? >>>>> >>>>> I'm on OSX, Erlang 17.4. >>>>> >>>>> Help! ^^_ >>>>> r. >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jan 12 14:18:11 2015 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 12 Jan 2015 14:18:11 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: At least put erlang:yield() at the end of the loop. But to me that is just a gigantic waste of CPU. This little c program as an external port does the job very nicely and hardly uses any resources. int main(int argc, char *argv[]) { char c[1]; c[0] = 'a'; for (;;) { usleep(3000); if (write(1, c, 1) <= 0) break; } return 0; } On Mon, Jan 12, 2015 at 2:10 PM, Darach Ennis wrote: > Hi Roberto, > > If you want to avoid clock slewing or you need a timely and periodic event > source you > can achieve this somewhat as follows, although the right solution will of > course depend > on multiple factors. > > -define(INTERVAL, 1000). % One second > > main(_) -> > loop(now_ms(),0). > > trigger(Now,Delta,Count) -> > io:format("At: ~p drift:~p rate:~p/sec~n", [Now, Delta - ?INTERVAL, > Count]). > > loop(Epoch,Count) -> > Now = now_ms(), > Delta = Now - Epoch, > case Delta >= ?INTERVAL of > true -> > trigger(Now,Delta,Count), > loop(Epoch+?INTERVAL,0); > false -> > loop(Epoch,Count+1) > end. > > now_ms() -> > {Mega,Secs,Micros} = os:timestamp(), > (Mega*1000000 + Secs)*1000 + (Micros div 1000). > > This loops at a rate of ~7 million loops/second on my laptop and fires > without > slew or significant drift consistently calling the trigger/3 function > every second. The rub is I > burn a little more core/process for the privilege than with mechanism that > induce slew such > as with timer or erlang:send_after directly. C'est la vie. Accuracy and > timeliness aren't free. > > In a gen_server you could use a combination of erlang:send_after and a > clock based on > os:timestamp and an epoch as above so that you end up with a clock that is > 'accurate enough' > for most use cases, and cheap enough for whatever value you ascribe to > such tradeoffs given > the specifics of your use case. > > Your mileage may vary. > > Cheers, > > Darach. > > On Mon, Jan 12, 2015 at 11:43 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> Hi Roberto, >> >> I'm not sure what you would expect here. timer:sleep/1 has an >> implementation like this: >> >> sleep(T) -> >> receive >> after T -> ok >> end. >> >> which is what other people told you also. In general, such timing >> constructs have no guarantee about when they will fire, only that they will >> fire after the time has elapsed. So what you are seeing is somewhat >> expected behaviour. The more fine-grained and the more timers you have, the >> less precise things will get. In a way, this is one of the properties which >> makes Erlang a soft-realtime system and not a hard-realtime one, although >> the precise definition of soft/hard realtime are subtly different. >> >> I believe there were some effort being made by the OTP team to address >> lock contention around the timer wheel, so the code might change behaviour >> by quite a lot in the future. It might even have to get worse for >> single-process timers in order to become better for multi-process timers. >> >> >> On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli >> wrote: >> >>> Indeed. From very basic try-outs: >>> >>> 1> test:sleep_periodical(6, 1000). >>> Finished in 6022381 microseconds. >>> ok >>> 2> test:sleep_periodical(12, 500). >>> Finished in 6042683 microseconds. >>> ok >>> 3> test:sleep_periodical(24, 250). >>> Finished in 6062096 microseconds. >>> ok >>> 4> test:sleep_periodical(48, 125). >>> Finished in 6133169 microseconds. >>> ok >>> 5> test:sleep_periodical(96, 62). >>> Finished in 6320419 microseconds. >>> ok >>> 6> test:sleep_periodical(192, 31). >>> Finished in 6492321 microseconds. >>> ok >>> 7> test:sleep_periodical(384, 16). >>> Finished in 7327693 microseconds. >>> ok >>> >>> So it somewhat is ok until we get in the 125ms range. >>> >>> >>> >>> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >>> sergej.jurecko@REDACTED> wrote: >>> >>>> timer:sleep is not even remotely as accurate for those low numbers. It >>>> can not go lower than 20ms or so from what i remember. >>>> >>>> I have had this issue once when I needed low interval for timer. I used >>>> an external c app which printed to stdout every 5ms. I dont think there is >>>> any way in erlang to get a timer that low. >>>> >>>> Sergej >>>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>>> wrote: >>>> >>>>> Additionally: >>>>> >>>>> 6> test:sleep_periodical(1000, 1). >>>>> Finished in 1999097 microsecond >>>>> >>>>> So: >>>>> >>>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>>> seconds (instead of 1). >>>>> >>>>> Looks like it's ms related, not total count. >>>>> >>>>> Best, >>>>> r. >>>>> >>>>> >>>>> >>>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>>> roberto@REDACTED> wrote: >>>>> >>>>>> Dear list, >>>>>> I've probably missed out something since I'm not understanding what >>>>>> is going here. >>>>>> >>>>>> I'm basically just trying to send messages periodically, though I >>>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>>> like the timer has some erratic behavior. >>>>>> >>>>>> I've stripped out the code to the bare minimum and it looks like this: >>>>>> >>>>>> -module(test). >>>>>> -export([sleep_periodical/2]). >>>>>> -export([sleep_loop/2]). >>>>>> >>>>>> sleep_periodical(Num, IntervalMs) -> >>>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>>> >>>>>> sleep_loop(Num, IntervalMs) -> >>>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>>> >>>>>> >>>>>> When I run this code, I see the following: >>>>>> >>>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>>> >>>>>> Eshell V6.3 (abort with ^G) >>>>>> 1> c(test). >>>>>> {ok,test} >>>>>> 2> test:sleep_periodical(1000, 10). >>>>>> Finished in 12257397 microseconds. >>>>>> ok >>>>>> 3> test:sleep_periodical(2000, 10). >>>>>> Finished in 24518070 microseconds. >>>>>> ok >>>>>> 4> test:sleep_periodical(10000, 1). >>>>>> Finished in 20000280 microseconds. >>>>>> ok >>>>>> 5> test:sleep_periodical(20000, 1). >>>>>> Finished in 40000685 microseconds. >>>>>> ok >>>>>> >>>>>> So: >>>>>> >>>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>>> seconds (instead of 10). >>>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>>> seconds (instead of 20). >>>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>>> seconds (instead of 10). >>>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>>> seconds (instead of 20). >>>>>> >>>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>>>> it takes 100% (double). >>>>>> >>>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>>> achieve the desired results? >>>>>> >>>>>> I'm on OSX, Erlang 17.4. >>>>>> >>>>>> Help! ^^_ >>>>>> r. >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Mon Jan 12 14:24:11 2015 From: darach@REDACTED (Darach Ennis) Date: Mon, 12 Jan 2015 13:24:11 +0000 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Hi Sergej, I wasn't suggesting using directly. I was illustrating that erlang can issue periodic events with reasonable accuracy and timeliness whilst avoiding slew. Not everyone will want to drop to C to resolve but this is a nice solution for those that do. Cheers, Darach. On Mon, Jan 12, 2015 at 1:18 PM, Sergej Jure?ko wrote: > At least put erlang:yield() at the end of the loop. But to me that is just > a gigantic waste of CPU. > > This little c program as an external port does the job very nicely and > hardly uses any resources. > int main(int argc, char *argv[]) > { > char c[1]; > c[0] = 'a'; > for (;;) > { > usleep(3000); > > if (write(1, c, 1) <= 0) > break; > } > > return 0; > } > > On Mon, Jan 12, 2015 at 2:10 PM, Darach Ennis wrote: > >> Hi Roberto, >> >> If you want to avoid clock slewing or you need a timely and periodic >> event source you >> can achieve this somewhat as follows, although the right solution will of >> course depend >> on multiple factors. >> >> -define(INTERVAL, 1000). % One second >> >> main(_) -> >> loop(now_ms(),0). >> >> trigger(Now,Delta,Count) -> >> io:format("At: ~p drift:~p rate:~p/sec~n", [Now, Delta - ?INTERVAL, >> Count]). >> >> loop(Epoch,Count) -> >> Now = now_ms(), >> Delta = Now - Epoch, >> case Delta >= ?INTERVAL of >> true -> >> trigger(Now,Delta,Count), >> loop(Epoch+?INTERVAL,0); >> false -> >> loop(Epoch,Count+1) >> end. >> >> now_ms() -> >> {Mega,Secs,Micros} = os:timestamp(), >> (Mega*1000000 + Secs)*1000 + (Micros div 1000). >> >> This loops at a rate of ~7 million loops/second on my laptop and fires >> without >> slew or significant drift consistently calling the trigger/3 function >> every second. The rub is I >> burn a little more core/process for the privilege than with mechanism >> that induce slew such >> as with timer or erlang:send_after directly. C'est la vie. Accuracy and >> timeliness aren't free. >> >> In a gen_server you could use a combination of erlang:send_after and a >> clock based on >> os:timestamp and an epoch as above so that you end up with a clock that >> is 'accurate enough' >> for most use cases, and cheap enough for whatever value you ascribe to >> such tradeoffs given >> the specifics of your use case. >> >> Your mileage may vary. >> >> Cheers, >> >> Darach. >> >> On Mon, Jan 12, 2015 at 11:43 AM, Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED> wrote: >> >>> Hi Roberto, >>> >>> I'm not sure what you would expect here. timer:sleep/1 has an >>> implementation like this: >>> >>> sleep(T) -> >>> receive >>> after T -> ok >>> end. >>> >>> which is what other people told you also. In general, such timing >>> constructs have no guarantee about when they will fire, only that they will >>> fire after the time has elapsed. So what you are seeing is somewhat >>> expected behaviour. The more fine-grained and the more timers you have, the >>> less precise things will get. In a way, this is one of the properties which >>> makes Erlang a soft-realtime system and not a hard-realtime one, although >>> the precise definition of soft/hard realtime are subtly different. >>> >>> I believe there were some effort being made by the OTP team to address >>> lock contention around the timer wheel, so the code might change behaviour >>> by quite a lot in the future. It might even have to get worse for >>> single-process timers in order to become better for multi-process timers. >>> >>> >>> On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli >>> wrote: >>> >>>> Indeed. From very basic try-outs: >>>> >>>> 1> test:sleep_periodical(6, 1000). >>>> Finished in 6022381 microseconds. >>>> ok >>>> 2> test:sleep_periodical(12, 500). >>>> Finished in 6042683 microseconds. >>>> ok >>>> 3> test:sleep_periodical(24, 250). >>>> Finished in 6062096 microseconds. >>>> ok >>>> 4> test:sleep_periodical(48, 125). >>>> Finished in 6133169 microseconds. >>>> ok >>>> 5> test:sleep_periodical(96, 62). >>>> Finished in 6320419 microseconds. >>>> ok >>>> 6> test:sleep_periodical(192, 31). >>>> Finished in 6492321 microseconds. >>>> ok >>>> 7> test:sleep_periodical(384, 16). >>>> Finished in 7327693 microseconds. >>>> ok >>>> >>>> So it somewhat is ok until we get in the 125ms range. >>>> >>>> >>>> >>>> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >>>> sergej.jurecko@REDACTED> wrote: >>>> >>>>> timer:sleep is not even remotely as accurate for those low numbers. It >>>>> can not go lower than 20ms or so from what i remember. >>>>> >>>>> I have had this issue once when I needed low interval for timer. I >>>>> used an external c app which printed to stdout every 5ms. I dont think >>>>> there is any way in erlang to get a timer that low. >>>>> >>>>> Sergej >>>>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>>>> wrote: >>>>> >>>>>> Additionally: >>>>>> >>>>>> 6> test:sleep_periodical(1000, 1). >>>>>> Finished in 1999097 microsecond >>>>>> >>>>>> So: >>>>>> >>>>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>>>> seconds (instead of 1). >>>>>> >>>>>> Looks like it's ms related, not total count. >>>>>> >>>>>> Best, >>>>>> r. >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>>>> roberto@REDACTED> wrote: >>>>>> >>>>>>> Dear list, >>>>>>> I've probably missed out something since I'm not understanding what >>>>>>> is going here. >>>>>>> >>>>>>> I'm basically just trying to send messages periodically, though I >>>>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>>>> like the timer has some erratic behavior. >>>>>>> >>>>>>> I've stripped out the code to the bare minimum and it looks like >>>>>>> this: >>>>>>> >>>>>>> -module(test). >>>>>>> -export([sleep_periodical/2]). >>>>>>> -export([sleep_loop/2]). >>>>>>> >>>>>>> sleep_periodical(Num, IntervalMs) -> >>>>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>>>> >>>>>>> sleep_loop(Num, IntervalMs) -> >>>>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>>>> >>>>>>> >>>>>>> When I run this code, I see the following: >>>>>>> >>>>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>>>> >>>>>>> Eshell V6.3 (abort with ^G) >>>>>>> 1> c(test). >>>>>>> {ok,test} >>>>>>> 2> test:sleep_periodical(1000, 10). >>>>>>> Finished in 12257397 microseconds. >>>>>>> ok >>>>>>> 3> test:sleep_periodical(2000, 10). >>>>>>> Finished in 24518070 microseconds. >>>>>>> ok >>>>>>> 4> test:sleep_periodical(10000, 1). >>>>>>> Finished in 20000280 microseconds. >>>>>>> ok >>>>>>> 5> test:sleep_periodical(20000, 1). >>>>>>> Finished in 40000685 microseconds. >>>>>>> ok >>>>>>> >>>>>>> So: >>>>>>> >>>>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>>>> seconds (instead of 10). >>>>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>>>> seconds (instead of 20). >>>>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>>>> seconds (instead of 10). >>>>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>>>> seconds (instead of 20). >>>>>>> >>>>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 >>>>>>> times it takes 100% (double). >>>>>>> >>>>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>>>> achieve the desired results? >>>>>>> >>>>>>> I'm on OSX, Erlang 17.4. >>>>>>> >>>>>>> Help! ^^_ >>>>>>> r. >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 14:50:03 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 14:50:03 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Isn't this using 100% of 1 cpu? On Mon, Jan 12, 2015 at 2:10 PM, Darach Ennis wrote: > Hi Roberto, > > If you want to avoid clock slewing or you need a timely and periodic event > source you > can achieve this somewhat as follows, although the right solution will of > course depend > on multiple factors. > > -define(INTERVAL, 1000). % One second > > main(_) -> > loop(now_ms(),0). > > trigger(Now,Delta,Count) -> > io:format("At: ~p drift:~p rate:~p/sec~n", [Now, Delta - ?INTERVAL, > Count]). > > loop(Epoch,Count) -> > Now = now_ms(), > Delta = Now - Epoch, > case Delta >= ?INTERVAL of > true -> > trigger(Now,Delta,Count), > loop(Epoch+?INTERVAL,0); > false -> > loop(Epoch,Count+1) > end. > > now_ms() -> > {Mega,Secs,Micros} = os:timestamp(), > (Mega*1000000 + Secs)*1000 + (Micros div 1000). > > This loops at a rate of ~7 million loops/second on my laptop and fires > without > slew or significant drift consistently calling the trigger/3 function > every second. The rub is I > burn a little more core/process for the privilege than with mechanism that > induce slew such > as with timer or erlang:send_after directly. C'est la vie. Accuracy and > timeliness aren't free. > > In a gen_server you could use a combination of erlang:send_after and a > clock based on > os:timestamp and an epoch as above so that you end up with a clock that is > 'accurate enough' > for most use cases, and cheap enough for whatever value you ascribe to > such tradeoffs given > the specifics of your use case. > > Your mileage may vary. > > Cheers, > > Darach. > > On Mon, Jan 12, 2015 at 11:43 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> Hi Roberto, >> >> I'm not sure what you would expect here. timer:sleep/1 has an >> implementation like this: >> >> sleep(T) -> >> receive >> after T -> ok >> end. >> >> which is what other people told you also. In general, such timing >> constructs have no guarantee about when they will fire, only that they will >> fire after the time has elapsed. So what you are seeing is somewhat >> expected behaviour. The more fine-grained and the more timers you have, the >> less precise things will get. In a way, this is one of the properties which >> makes Erlang a soft-realtime system and not a hard-realtime one, although >> the precise definition of soft/hard realtime are subtly different. >> >> I believe there were some effort being made by the OTP team to address >> lock contention around the timer wheel, so the code might change behaviour >> by quite a lot in the future. It might even have to get worse for >> single-process timers in order to become better for multi-process timers. >> >> >> On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli >> wrote: >> >>> Indeed. From very basic try-outs: >>> >>> 1> test:sleep_periodical(6, 1000). >>> Finished in 6022381 microseconds. >>> ok >>> 2> test:sleep_periodical(12, 500). >>> Finished in 6042683 microseconds. >>> ok >>> 3> test:sleep_periodical(24, 250). >>> Finished in 6062096 microseconds. >>> ok >>> 4> test:sleep_periodical(48, 125). >>> Finished in 6133169 microseconds. >>> ok >>> 5> test:sleep_periodical(96, 62). >>> Finished in 6320419 microseconds. >>> ok >>> 6> test:sleep_periodical(192, 31). >>> Finished in 6492321 microseconds. >>> ok >>> 7> test:sleep_periodical(384, 16). >>> Finished in 7327693 microseconds. >>> ok >>> >>> So it somewhat is ok until we get in the 125ms range. >>> >>> >>> >>> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >>> sergej.jurecko@REDACTED> wrote: >>> >>>> timer:sleep is not even remotely as accurate for those low numbers. It >>>> can not go lower than 20ms or so from what i remember. >>>> >>>> I have had this issue once when I needed low interval for timer. I used >>>> an external c app which printed to stdout every 5ms. I dont think there is >>>> any way in erlang to get a timer that low. >>>> >>>> Sergej >>>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>>> wrote: >>>> >>>>> Additionally: >>>>> >>>>> 6> test:sleep_periodical(1000, 1). >>>>> Finished in 1999097 microsecond >>>>> >>>>> So: >>>>> >>>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>>> seconds (instead of 1). >>>>> >>>>> Looks like it's ms related, not total count. >>>>> >>>>> Best, >>>>> r. >>>>> >>>>> >>>>> >>>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>>> roberto@REDACTED> wrote: >>>>> >>>>>> Dear list, >>>>>> I've probably missed out something since I'm not understanding what >>>>>> is going here. >>>>>> >>>>>> I'm basically just trying to send messages periodically, though I >>>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>>> like the timer has some erratic behavior. >>>>>> >>>>>> I've stripped out the code to the bare minimum and it looks like this: >>>>>> >>>>>> -module(test). >>>>>> -export([sleep_periodical/2]). >>>>>> -export([sleep_loop/2]). >>>>>> >>>>>> sleep_periodical(Num, IntervalMs) -> >>>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>>> >>>>>> sleep_loop(Num, IntervalMs) -> >>>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>>> >>>>>> >>>>>> When I run this code, I see the following: >>>>>> >>>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>>> >>>>>> Eshell V6.3 (abort with ^G) >>>>>> 1> c(test). >>>>>> {ok,test} >>>>>> 2> test:sleep_periodical(1000, 10). >>>>>> Finished in 12257397 microseconds. >>>>>> ok >>>>>> 3> test:sleep_periodical(2000, 10). >>>>>> Finished in 24518070 microseconds. >>>>>> ok >>>>>> 4> test:sleep_periodical(10000, 1). >>>>>> Finished in 20000280 microseconds. >>>>>> ok >>>>>> 5> test:sleep_periodical(20000, 1). >>>>>> Finished in 40000685 microseconds. >>>>>> ok >>>>>> >>>>>> So: >>>>>> >>>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>>> seconds (instead of 10). >>>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>>> seconds (instead of 20). >>>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>>> seconds (instead of 10). >>>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>>> seconds (instead of 20). >>>>>> >>>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>>>> it takes 100% (double). >>>>>> >>>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>>> achieve the desired results? >>>>>> >>>>>> I'm on OSX, Erlang 17.4. >>>>>> >>>>>> Help! ^^_ >>>>>> r. >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Mon Jan 12 14:59:05 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 12 Jan 2015 14:59:05 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: in the original code, if we replaced sleep_loop with timer:apply_interval/4 , would it give the expected results? From caisonglu@REDACTED Mon Jan 12 16:42:28 2015 From: caisonglu@REDACTED (songlu cai) Date: Mon, 12 Jan 2015 23:42:28 +0800 Subject: [erlang-questions] What's the proper scenario for ecio? Message-ID: Hi, I really appreciate the the ecio feature on OTP 17.4(for lower latency and smoother management of IO), I run the test for two scenes, one for short tcp connection, one for keepalive tcp connection. In order to measure effects of ecio, I make a wrapper for prepare_for_sys_schedule: static ERTS_INLINE *int* prepare_for_sys_schedule_wrapper(ErtsSchedulerData *esdp, *int* non_blocking) { *int* r = prepare_for_sys_schedule(esdp, non_blocking); *int* *index* = non_blocking + erts_eager_check_io*2; *if* (r) { erts_no_atomic32_inc(&(schedule_statistics[*index*])); } erts_no_atomic32_inc(&(schedule_statistics[4])); *if* (schedule_statistics[4] % 10000 == 0) { erts_print(ERTS_PRINT_STDOUT, NULL, "counts:%d %d %d %d %d\n", schedule_statistics[4], schedule_statistics [0], schedule_statistics[1], schedule_statistics[2], schedule_statistics[3]); } *return* r; } ERTS_CIO_EXPORT(erts_check_io)(*int* do_wait) { erts_no_atomic32_inc(&(check_count[do_wait])); erts_no_atomic32_inc(&(check_count[3])); *if* (check_count[3] % 10000 == 0) { erts_print(ERTS_PRINT_STDOUT, NULL, "check:%d %d %d\n", check_count[3], check_count[0], check_count[1]); } } Test A: Test for http (short tcp connection), a http server runs on erl, a client makes stress test with apache ab cmd: ./ab -c 60 -n 100000 http://10.10.10.10:8888/ Test A1: ./bin/erl +K true +h 99999 +P 99999 -secio true -smp enable -s ehttpd Result: counts:30700000 0 0 1465043 2728782 check:17860000 14405477 3454523 prepare_for_sys_schedule=1 ratio (1465043 + 2728782)/30700000 = 13.6% Compared with Test A2: ./bin/erl +K true +h 99999 +P 99999 -secio false -smp enable -s ehttpd counts:36110000 1433811 919525 0 0 check:16070000 12576257 3493743 prepare_for_sys_schedule=1 ratio (1433811 + 919525)/36110000 = 6.5% Check_io_count ratio: A1 check_io / A2 check_io = 17860000/16070000 = 1.11 Time Cost: same (diff < 0.5%) QPS: same (diff < 0.5%) CPU: A1 62% per CPU, A2 64% per CPU Test B: Test for tcp keepalive connection, a proxy server runs on erl, a client make stress test with mysql request. Test B1: beam.smp -secio true -sbt db -sbwt very_short -swt low -sfwi 500 -MBmmsbc 100 -MHmmsbc 100 -MBmmmbc 100 -MHmmmbc 100 -MMscs 1024 -MBsmbcs 10240 -MHsbct 2048 -hmbs 46422 -hms 2586 -zdbbl 8192 -W w -P 1000000 -A 16 -K true -d -e 50000 -Q 1000000 -sct L0T0C0P0N0:L1T1C1P0N0:L2T2C2P0N0:L3T3C3P0N0:L4T4C4P0N0:L5T5C5P0N0:L6T6C0P1N1:L7T7C1P1N1:L8T8C2P1N1:L9T9C3P1N1:L10T10C4P1N1:L11T11C5P1N1:L12T12C0P0N0:L13T13C1P0N0:L14T14C2P0N0:L15T15C3P0N0:L16T16C4P0N0:L17T17C5P0N0:L18T18C0P1N1:L19T19C1P1N1:L20T20C2P1N1:L21T21C3P1N1:L22T22C4P1N1:L23T23C5P1N1 -swct medium -- -root /path/ -progname ** -- -home /path -- -boot /path -mode interactive -config /path -shutdown_time 30000 -setcookie *** -name name@REDACTED ? console Result: *counts*:72090000 0 0 13711145 1499024 *check*:16230000 15777657 452343 prepare_for_sys_schedule=1 ratio = 21 Test B2: beam.smp -secio false -sbt db -sbwt very_short -swt low -sfwi 500 -MBmmsbc 100 -MHmmsbc 100 -MBmmmbc 100 -MHmmmbc 100 -MMscs 1024 -MBsmbcs 10240 -MHsbct 2048 -hmbs 46422 -hms 2586 -zdbbl 8192 -W w -P 1000000 -A 16 -K true -d -e 50000 -Q 1000000 -sct L0T0C0P0N0:L1T1C1P0N0:L2T2C2P0N0:L3T3C3P0N0:L4T4C4P0N0:L5T5C5P0N0:L6T6C0P1N1:L7T7C1P1N1:L8T8C2P1N1:L9T9C3P1N1:L10T10C4P1N1:L11T11C5P1N1:L12T12C0P0N0:L13T13C1P0N0:L14T14C2P0N0:L15T15C3P0N0:L16T16C4P0N0:L17T17C5P0N0:L18T18C0P1N1:L19T19C1P1N1:L20T20C2P1N1:L21T21C3P1N1:L22T22C4P1N1:L23T23C5P1N1 -swct medium -- -root /path/ -progname ** -- -home /path -- -boot /path -mode interactive -config /path -shutdown_time 30000 -setcookie *** -name name@REDACTED ? console Result: *counts*:72170000 13840081 1023183 0 0 *check*:15630000 15303454 326546 prepare_for_sys_schedule=1 ratio = 20.5% Time Cost: B1 with 18 min 17 seconds, B2 with 18 min 12 seconds CPU: B1 with around 1000%, B2 with around 1000% I test scene B for several times, the result is nearly the same, then I review the code and debug the key path for ecio: *else* *if* (!ERTS_SCHEDULER_IS_DIRTY(esdp) && (fcalls > input_reductions && prepare_*for*_sys_schedule(esdp, !0))) { I find when io task is heavy(in Test B), we barely get here, so there is nearly no effect with +secio true. In test A, +secio has a little effect. I presume that +scio true only works with High CPU & Low IO scene, so we can prevent the port task from IO starvation. Is that true? And how do you test ecio? Best Regards, Songlu Cai -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 17:03:54 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 17:03:54 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Hi Jesper, I was expecting a more precise timer, that is all. :) Thank you for your input. So: how can someone execute an action every 1ms? Best, r. On Mon, Jan 12, 2015 at 12:43 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi Roberto, > > I'm not sure what you would expect here. timer:sleep/1 has an > implementation like this: > > sleep(T) -> > receive > after T -> ok > end. > > which is what other people told you also. In general, such timing > constructs have no guarantee about when they will fire, only that they will > fire after the time has elapsed. So what you are seeing is somewhat > expected behaviour. The more fine-grained and the more timers you have, the > less precise things will get. In a way, this is one of the properties which > makes Erlang a soft-realtime system and not a hard-realtime one, although > the precise definition of soft/hard realtime are subtly different. > > I believe there were some effort being made by the OTP team to address > lock contention around the timer wheel, so the code might change behaviour > by quite a lot in the future. It might even have to get worse for > single-process timers in order to become better for multi-process timers. > > > On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli > wrote: > >> Indeed. From very basic try-outs: >> >> 1> test:sleep_periodical(6, 1000). >> Finished in 6022381 microseconds. >> ok >> 2> test:sleep_periodical(12, 500). >> Finished in 6042683 microseconds. >> ok >> 3> test:sleep_periodical(24, 250). >> Finished in 6062096 microseconds. >> ok >> 4> test:sleep_periodical(48, 125). >> Finished in 6133169 microseconds. >> ok >> 5> test:sleep_periodical(96, 62). >> Finished in 6320419 microseconds. >> ok >> 6> test:sleep_periodical(192, 31). >> Finished in 6492321 microseconds. >> ok >> 7> test:sleep_periodical(384, 16). >> Finished in 7327693 microseconds. >> ok >> >> So it somewhat is ok until we get in the 125ms range. >> >> >> >> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >> sergej.jurecko@REDACTED> wrote: >> >>> timer:sleep is not even remotely as accurate for those low numbers. It >>> can not go lower than 20ms or so from what i remember. >>> >>> I have had this issue once when I needed low interval for timer. I used >>> an external c app which printed to stdout every 5ms. I dont think there is >>> any way in erlang to get a timer that low. >>> >>> Sergej >>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>> wrote: >>> >>>> Additionally: >>>> >>>> 6> test:sleep_periodical(1000, 1). >>>> Finished in 1999097 microsecond >>>> >>>> So: >>>> >>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>> seconds (instead of 1). >>>> >>>> Looks like it's ms related, not total count. >>>> >>>> Best, >>>> r. >>>> >>>> >>>> >>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>> roberto@REDACTED> wrote: >>>> >>>>> Dear list, >>>>> I've probably missed out something since I'm not understanding what is >>>>> going here. >>>>> >>>>> I'm basically just trying to send messages periodically, though I >>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>> like the timer has some erratic behavior. >>>>> >>>>> I've stripped out the code to the bare minimum and it looks like this: >>>>> >>>>> -module(test). >>>>> -export([sleep_periodical/2]). >>>>> -export([sleep_loop/2]). >>>>> >>>>> sleep_periodical(Num, IntervalMs) -> >>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>> >>>>> sleep_loop(Num, IntervalMs) -> >>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>> >>>>> >>>>> When I run this code, I see the following: >>>>> >>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>> >>>>> Eshell V6.3 (abort with ^G) >>>>> 1> c(test). >>>>> {ok,test} >>>>> 2> test:sleep_periodical(1000, 10). >>>>> Finished in 12257397 microseconds. >>>>> ok >>>>> 3> test:sleep_periodical(2000, 10). >>>>> Finished in 24518070 microseconds. >>>>> ok >>>>> 4> test:sleep_periodical(10000, 1). >>>>> Finished in 20000280 microseconds. >>>>> ok >>>>> 5> test:sleep_periodical(20000, 1). >>>>> Finished in 40000685 microseconds. >>>>> ok >>>>> >>>>> So: >>>>> >>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>> seconds (instead of 10). >>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>> seconds (instead of 20). >>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>> seconds (instead of 10). >>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>> seconds (instead of 20). >>>>> >>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>>> it takes 100% (double). >>>>> >>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>> achieve the desired results? >>>>> >>>>> I'm on OSX, Erlang 17.4. >>>>> >>>>> Help! ^^_ >>>>> r. >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Jan 12 17:10:24 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 12 Jan 2015 17:10:24 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: Hi Imants, If everything is based on the same timer implementation I'm assuming that it would still yield the same problem. Best, r. On Mon, Jan 12, 2015 at 2:59 PM, Imants Cekusins wrote: > in the original code, if we replaced sleep_loop with > timer:apply_interval/4 > > , would it give the expected results? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Mon Jan 12 17:13:16 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Mon, 12 Jan 2015 17:13:16 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: Message-ID: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> There is no other solution other than the two provided. An endless loop without sleep, or an external program (or driver). Sergej On 12 Jan 2015, at 17:03, Roberto Ostinelli wrote: > Hi Jesper, > I was expecting a more precise timer, that is all. :) > Thank you for your input. > > So: how can someone execute an action every 1ms? > > Best, > r. > > On Mon, Jan 12, 2015 at 12:43 PM, Jesper Louis Andersen wrote: > Hi Roberto, > > I'm not sure what you would expect here. timer:sleep/1 has an implementation like this: > > sleep(T) -> > receive > after T -> ok > end. > > which is what other people told you also. In general, such timing constructs have no guarantee about when they will fire, only that they will fire after the time has elapsed. So what you are seeing is somewhat expected behaviour. The more fine-grained and the more timers you have, the less precise things will get. In a way, this is one of the properties which makes Erlang a soft-realtime system and not a hard-realtime one, although the precise definition of soft/hard realtime are subtly different. > > I believe there were some effort being made by the OTP team to address lock contention around the timer wheel, so the code might change behaviour by quite a lot in the future. It might even have to get worse for single-process timers in order to become better for multi-process timers. > > > On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli wrote: > Indeed. From very basic try-outs: > > 1> test:sleep_periodical(6, 1000). > Finished in 6022381 microseconds. > ok > 2> test:sleep_periodical(12, 500). > Finished in 6042683 microseconds. > ok > 3> test:sleep_periodical(24, 250). > Finished in 6062096 microseconds. > ok > 4> test:sleep_periodical(48, 125). > Finished in 6133169 microseconds. > ok > 5> test:sleep_periodical(96, 62). > Finished in 6320419 microseconds. > ok > 6> test:sleep_periodical(192, 31). > Finished in 6492321 microseconds. > ok > 7> test:sleep_periodical(384, 16). > Finished in 7327693 microseconds. > ok > > So it somewhat is ok until we get in the 125ms range. > > > > On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko wrote: > timer:sleep is not even remotely as accurate for those low numbers. It can not go lower than 20ms or so from what i remember. > > I have had this issue once when I needed low interval for timer. I used an external c app which printed to stdout every 5ms. I dont think there is any way in erlang to get a timer that low. > > Sergej > > On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" wrote: > Additionally: > > 6> test:sleep_periodical(1000, 1). > Finished in 1999097 microsecond > > So: > Looping 1,000 times and sleeping 1 ms each time takes 2.00 seconds (instead of 1). > Looks like it's ms related, not total count. > > Best, > r. > > > > On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli wrote: > Dear list, > I've probably missed out something since I'm not understanding what is going here. > > I'm basically just trying to send messages periodically, though I found out that I wasn't seeing the behavior that I was expecting. It looks like the timer has some erratic behavior. > > I've stripped out the code to the bare minimum and it looks like this: > > -module(test). > -export([sleep_periodical/2]). > -export([sleep_loop/2]). > > sleep_periodical(Num, IntervalMs) -> > {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), > io:format("Finished in ~p microseconds.~n", [Time]). > > sleep_loop(Num, IntervalMs) -> > SleepFun = fun(_) -> timer:sleep(IntervalMs) end, > lists:foreach(SleepFun, lists:seq(1, Num)). > > > When I run this code, I see the following: > > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V6.3 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:sleep_periodical(1000, 10). > Finished in 12257397 microseconds. > ok > 3> test:sleep_periodical(2000, 10). > Finished in 24518070 microseconds. > ok > 4> test:sleep_periodical(10000, 1). > Finished in 20000280 microseconds. > ok > 5> test:sleep_periodical(20000, 1). > Finished in 40000685 microseconds. > ok > > So: > Looping 1,000 times and sleeping 10 ms each time takes 12.26 seconds (instead of 10). > Looping 2,000 times and sleeping 10 ms each time takes 24.52 seconds (instead of 20). > Looping 10,000 times and sleeping 1 ms each time takes 20.00 seconds (instead of 10). > Looping 20,000 times and sleeping 1 ms each time takes 40.00 seconds (instead of 20). > Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times it takes 100% (double). > > Can some kind soul explain what I'm doing wrong here and how to achieve the desired results? > > I'm on OSX, Erlang 17.4. > > Help! ^^_ > r. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Mon Jan 12 17:23:25 2015 From: darach@REDACTED (Darach Ennis) Date: Mon, 12 Jan 2015 16:23:25 +0000 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> References: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> Message-ID: Hi guys, +1. And if you consider a loop consider erlang:yield as Sergej recommended. If your use case can tolerate a little slew or skew then using a timer or erlang:send_after instead might be a pleasant sufficiency. If your use case is deadline critical or hard real-time oriented then perhaps you might be interested in work being done by Peer Strizinger in the embedded space. He has spoken about this at EUC and Codemesh: http://www.codemesh.io/codemesh2014/peer-stritzinger http://www.erlang-factory.com/euc2014/peer-stritzinger Peer has ported Erlang to an RTEMS and this may also be of interest: http://www.grisp.org/ Cheers, Darach. On Mon, Jan 12, 2015 at 4:13 PM, Sergej Jurecko wrote: > There is no other solution other than the two provided. An endless loop > without sleep, or an external program (or driver). > > > Sergej > > > On 12 Jan 2015, at 17:03, Roberto Ostinelli wrote: > > Hi Jesper, > I was expecting a more precise timer, that is all. :) > Thank you for your input. > > So: how can someone execute an action every 1ms? > > Best, > r. > > On Mon, Jan 12, 2015 at 12:43 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> Hi Roberto, >> >> I'm not sure what you would expect here. timer:sleep/1 has an >> implementation like this: >> >> sleep(T) -> >> receive >> after T -> ok >> end. >> >> which is what other people told you also. In general, such timing >> constructs have no guarantee about when they will fire, only that they will >> fire after the time has elapsed. So what you are seeing is somewhat >> expected behaviour. The more fine-grained and the more timers you have, the >> less precise things will get. In a way, this is one of the properties which >> makes Erlang a soft-realtime system and not a hard-realtime one, although >> the precise definition of soft/hard realtime are subtly different. >> >> I believe there were some effort being made by the OTP team to address >> lock contention around the timer wheel, so the code might change behaviour >> by quite a lot in the future. It might even have to get worse for >> single-process timers in order to become better for multi-process timers. >> >> >> On Mon Jan 12 2015 at 12:20:15 PM Roberto Ostinelli >> wrote: >> >>> Indeed. From very basic try-outs: >>> >>> 1> test:sleep_periodical(6, 1000). >>> Finished in 6022381 microseconds. >>> ok >>> 2> test:sleep_periodical(12, 500). >>> Finished in 6042683 microseconds. >>> ok >>> 3> test:sleep_periodical(24, 250). >>> Finished in 6062096 microseconds. >>> ok >>> 4> test:sleep_periodical(48, 125). >>> Finished in 6133169 microseconds. >>> ok >>> 5> test:sleep_periodical(96, 62). >>> Finished in 6320419 microseconds. >>> ok >>> 6> test:sleep_periodical(192, 31). >>> Finished in 6492321 microseconds. >>> ok >>> 7> test:sleep_periodical(384, 16). >>> Finished in 7327693 microseconds. >>> ok >>> >>> So it somewhat is ok until we get in the 125ms range. >>> >>> >>> >>> On Mon, Jan 12, 2015 at 12:12 PM, Sergej Jure?ko < >>> sergej.jurecko@REDACTED> wrote: >>> >>>> timer:sleep is not even remotely as accurate for those low numbers. It >>>> can not go lower than 20ms or so from what i remember. >>>> >>>> I have had this issue once when I needed low interval for timer. I used >>>> an external c app which printed to stdout every 5ms. I dont think there is >>>> any way in erlang to get a timer that low. >>>> >>>> Sergej >>>> On Jan 12, 2015 11:46 AM, "Roberto Ostinelli" >>>> wrote: >>>> >>>>> Additionally: >>>>> >>>>> 6> test:sleep_periodical(1000, 1). >>>>> Finished in 1999097 microsecond >>>>> >>>>> So: >>>>> >>>>> - Looping 1,000 times and sleeping 1 ms each time takes 2.00 >>>>> seconds (instead of 1). >>>>> >>>>> Looks like it's ms related, not total count. >>>>> >>>>> Best, >>>>> r. >>>>> >>>>> >>>>> >>>>> On Mon, Jan 12, 2015 at 11:40 AM, Roberto Ostinelli < >>>>> roberto@REDACTED> wrote: >>>>> >>>>>> Dear list, >>>>>> I've probably missed out something since I'm not understanding what >>>>>> is going here. >>>>>> >>>>>> I'm basically just trying to send messages periodically, though I >>>>>> found out that I wasn't seeing the behavior that I was expecting. It looks >>>>>> like the timer has some erratic behavior. >>>>>> >>>>>> I've stripped out the code to the bare minimum and it looks like this: >>>>>> >>>>>> -module(test). >>>>>> -export([sleep_periodical/2]). >>>>>> -export([sleep_loop/2]). >>>>>> >>>>>> sleep_periodical(Num, IntervalMs) -> >>>>>> {Time, _Value} = timer:tc(?MODULE, sleep_loop, [Num, IntervalMs]), >>>>>> io:format("Finished in ~p microseconds.~n", [Time]). >>>>>> >>>>>> sleep_loop(Num, IntervalMs) -> >>>>>> SleepFun = fun(_) -> timer:sleep(IntervalMs) end, >>>>>> lists:foreach(SleepFun, lists:seq(1, Num)). >>>>>> >>>>>> >>>>>> When I run this code, I see the following: >>>>>> >>>>>> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] >>>>>> [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >>>>>> >>>>>> Eshell V6.3 (abort with ^G) >>>>>> 1> c(test). >>>>>> {ok,test} >>>>>> 2> test:sleep_periodical(1000, 10). >>>>>> Finished in 12257397 microseconds. >>>>>> ok >>>>>> 3> test:sleep_periodical(2000, 10). >>>>>> Finished in 24518070 microseconds. >>>>>> ok >>>>>> 4> test:sleep_periodical(10000, 1). >>>>>> Finished in 20000280 microseconds. >>>>>> ok >>>>>> 5> test:sleep_periodical(20000, 1). >>>>>> Finished in 40000685 microseconds. >>>>>> ok >>>>>> >>>>>> So: >>>>>> >>>>>> - Looping 1,000 times and sleeping 10 ms each time takes 12.26 >>>>>> seconds (instead of 10). >>>>>> - Looping 2,000 times and sleeping 10 ms each time takes 24.52 >>>>>> seconds (instead of 20). >>>>>> - Looping 10,000 times and sleeping 1 ms each time takes 20.00 >>>>>> seconds (instead of 10). >>>>>> - Looping 20,000 times and sleeping 1 ms each time takes 40.00 >>>>>> seconds (instead of 20). >>>>>> >>>>>> Up to 10,000 times it takes 12,16% (1/8) more time, from 10,000 times >>>>>> it takes 100% (double). >>>>>> >>>>>> Can some kind soul explain what I'm doing wrong here and how to >>>>>> achieve the desired results? >>>>>> >>>>>> I'm on OSX, Erlang 17.4. >>>>>> >>>>>> Help! ^^_ >>>>>> r. >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Mon Jan 12 18:11:47 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 12 Jan 2015 18:11:47 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> Message-ID: If the task is to spawn (not complete) a function at specified intervals, timer:apply interval works just fine. Just did a 3-module demo with a gen_server. Interval: 1 ms. Run for 3, then 10 seconds. Result nr of loops =:= expected number. Email me for the code. From imantc@REDACTED Mon Jan 12 18:14:12 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 12 Jan 2015 18:14:12 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> Message-ID: btw CPU looked ok: was spread across cores and did not max out. From max.lapshin@REDACTED Mon Jan 12 18:57:20 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 12 Jan 2015 21:57:20 +0400 Subject: [erlang-questions] luerl stacktraces Message-ID: Hi. luerl (https://github.com/rvirding/luerl ) is a wonderful project, thanks for it. 3 years ago Robert told that it is a big problem to add proper stacktraces to lua errors. Right now it is: {lua_error,{undef_function,nil},{luerl,{array,25,100,undefined,{{{table,{array,0,10,nil,10},{{{{empty,<<"_G">>,{tref,0},empty,<<"_VERSION">>,<<"Lua 5.2">>,empty},<<"assert">>,{function,#Fun},{empty,<<"auth_url">>,<<" http://go.commpass.tv/flussonic/auth ">>,empty,<<"bit32">>,{tref,11},empty},<<"collectgarbage">>,{function,#Fun},{empty,<<"comet">>,{tref,19},empty,<<"converter">>,{tref,18},empty}},<<"dofile">>,{function,#Fun},{{empty,<<"eprint">>,{function,#Fun},empty},<<"error">>,..... For something like: mytable.unexisting_function("Hello world") Are there any severe problems to add stacktraces? It would be very good at least to have file and line of the error to show user where he has error. Perhaps some library function like: luerl_lib:unpack_stacktrace(Error) -> ... ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jan 12 19:00:09 2015 From: rvirding@REDACTED (Robert Virding) Date: Mon, 12 Jan 2015 19:00:09 +0100 Subject: [erlang-questions] Ring Benchmark Problem In-Reply-To: References: Message-ID: A much better way than sending the path around is for every process to keep track for itself the next process in the ring so you just need to send the message between the processes. At least if you are after speed. If you do do need to be able to dynamically restructure the ring it would be easy to add an extra message where you tell a process its new next process. Robert On 11 January 2015 at 20:48, Harit Himanshu wrote: > Hello there, > > I need help with code review on my attempt to following problem > > Write a ring benchmark. Create N processes in a ring. Send a message > round the ring M times so that a total of N * M messages get sent. Time > how long this takes for different values of N and M. > > I have not timed this yet, and guidance on recommended ways to time is > very much appreciated. > > For now, following is the code for ring that I wrote. Please let me know > if there are any shortcomings or code could be written in an much idiomatic > way. > > Thanks a lot > + Harit > > *Code* > > -module(ring). > -author("harith"). > > %% API > -export([message/2]). > > % create ring of N processes and > % send M messages between them > message(N, M) when is_integer(N), is_integer(M), N > 0, M > 0 -> > Ring = create_ring(N), > [Start | T] = Ring, > Start ! {T, Ring, 1, M}. > > create_ring(N) -> > Processes = [spawn(fun() -> loop() end) || _ <- lists:seq(1, N)], > [H | _] = Processes, > lists:append(Processes, [H]). > > loop() -> > receive > {[H | T], _L, CurrentMessage, M} -> > io:format("~p received ~p~n", [self(), CurrentMessage]), > H ! {T, _L, CurrentMessage, M}, > loop(); > {[], Ring, CurrentMessage, M} -> > io:format("~p received ~p with empty list~n", [self(), > CurrentMessage]), > case CurrentMessage < M of > true -> > [_ | [Next | T]] = Ring, > NewMessage = CurrentMessage + 1, > io:format("sending message ~p to ~p~n", [NewMessage, Next]), > Next ! {T, Ring, NewMessage, M}; > false -> io:format("done sending ~p messages in ~p ring, taking > rest now.~n",[M, Ring]) > end, > loop() > end. > > *Output* > 1> ring:message(4, 3). > <0.33.0> received 1 > {[<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], > [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], > 1,3} > <0.34.0> received 1 > <0.35.0> received 1 > 2> <0.36.0> received 1 > 2> <0.33.0> received 1 with empty list > 2> sending message 2 to <0.34.0> > 2> <0.34.0> received 2 > 2> <0.35.0> received 2 > 2> <0.36.0> received 2 > 2> <0.33.0> received 2 with empty list > 2> sending message 3 to <0.34.0> > 2> <0.34.0> received 3 > 2> <0.35.0> received 3 > 2> <0.36.0> received 3 > 2> <0.33.0> received 3 with empty list > 2> done sending 3 messages in > [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>] ring, taking rest now. > 2> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jan 12 19:13:59 2015 From: rvirding@REDACTED (Robert Virding) Date: Mon, 12 Jan 2015 19:13:59 +0100 Subject: [erlang-questions] luerl stacktraces In-Reply-To: References: Message-ID: Yes, I know. But I am doing some work on the Luerl VM and I am going to try to add more Lua call stack information. Function names can be a bit difficult but line numbers should be easier. Robert On 12 January 2015 at 18:57, Max Lapshin wrote: > Hi. > > luerl (https://github.com/rvirding/luerl ) is a wonderful project, thanks > for it. > > > 3 years ago Robert told that it is a big problem to add proper stacktraces > to lua errors. > > Right now it is: > > {lua_error,{undef_function,nil},{luerl,{array,25,100,undefined,{{{table,{array,0,10,nil,10},{{{{empty,<<"_G">>,{tref,0},empty,<<"_VERSION">>,<<"Lua > 5.2">>,empty},<<"assert">>,{function,#Fun},{empty,<<"auth_url">>,<<" > http://go.commpass.tv/flussonic/auth > ">>,empty,<<"bit32">>,{tref,11},empty},<<"collectgarbage">>,{function,#Fun},{empty,<<"comet">>,{tref,19},empty,<<"converter">>,{tref,18},empty}},<<"dofile">>,{function,#Fun},{{empty,<<"eprint">>,{function,#Fun},empty},<<"error">>,..... > > > For something like: > > mytable.unexisting_function("Hello world") > > > Are there any severe problems to add stacktraces? > > It would be very good at least to have file and line of the error to show > user where he has error. > > > Perhaps some library function like: luerl_lib:unpack_stacktrace(Error) -> > ... ? > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 12 19:25:34 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 12 Jan 2015 10:25:34 -0800 Subject: [erlang-questions] Ring Benchmark Problem In-Reply-To: References: Message-ID: That sounds like a good idea Robert, could you please guide me how to achieve this? Thanks On Mon, Jan 12, 2015 at 10:00 AM, Robert Virding wrote: > A much better way than sending the path around is for every process to > keep track for itself the next process in the ring so you just need to send > the message between the processes. At least if you are after speed. If you > do do need to be able to dynamically restructure the ring it would be easy > to add an extra message where you tell a process its new next process. > > Robert > > > On 11 January 2015 at 20:48, Harit Himanshu > wrote: > >> Hello there, >> >> I need help with code review on my attempt to following problem >> >> Write a ring benchmark. Create N processes in a ring. Send a message >> round the ring M times so that a total of N * M messages get sent. Time >> how long this takes for different values of N and M. >> >> I have not timed this yet, and guidance on recommended ways to time is >> very much appreciated. >> >> For now, following is the code for ring that I wrote. Please let me know >> if there are any shortcomings or code could be written in an much idiomatic >> way. >> >> Thanks a lot >> + Harit >> >> *Code* >> >> -module(ring). >> -author("harith"). >> >> %% API >> -export([message/2]). >> >> % create ring of N processes and >> % send M messages between them >> message(N, M) when is_integer(N), is_integer(M), N > 0, M > 0 -> >> Ring = create_ring(N), >> [Start | T] = Ring, >> Start ! {T, Ring, 1, M}. >> >> create_ring(N) -> >> Processes = [spawn(fun() -> loop() end) || _ <- lists:seq(1, N)], >> [H | _] = Processes, >> lists:append(Processes, [H]). >> >> loop() -> >> receive >> {[H | T], _L, CurrentMessage, M} -> >> io:format("~p received ~p~n", [self(), CurrentMessage]), >> H ! {T, _L, CurrentMessage, M}, >> loop(); >> {[], Ring, CurrentMessage, M} -> >> io:format("~p received ~p with empty list~n", [self(), >> CurrentMessage]), >> case CurrentMessage < M of >> true -> >> [_ | [Next | T]] = Ring, >> NewMessage = CurrentMessage + 1, >> io:format("sending message ~p to ~p~n", [NewMessage, Next]), >> Next ! {T, Ring, NewMessage, M}; >> false -> io:format("done sending ~p messages in ~p ring, taking >> rest now.~n",[M, Ring]) >> end, >> loop() >> end. >> >> *Output* >> 1> ring:message(4, 3). >> <0.33.0> received 1 >> {[<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], >> [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], >> 1,3} >> <0.34.0> received 1 >> <0.35.0> received 1 >> 2> <0.36.0> received 1 >> 2> <0.33.0> received 1 with empty list >> 2> sending message 2 to <0.34.0> >> 2> <0.34.0> received 2 >> 2> <0.35.0> received 2 >> 2> <0.36.0> received 2 >> 2> <0.33.0> received 2 with empty list >> 2> sending message 3 to <0.34.0> >> 2> <0.34.0> received 3 >> 2> <0.35.0> received 3 >> 2> <0.36.0> received 3 >> 2> <0.33.0> received 3 with empty list >> 2> done sending 3 messages in >> [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>] ring, taking rest now. >> 2> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ryankbrown@REDACTED Mon Jan 12 20:09:39 2015 From: ryankbrown@REDACTED (Ryan Brown) Date: Mon, 12 Jan 2015 12:09:39 -0700 Subject: [erlang-questions] Erlang omac error (R15B03) Message-ID: I know I have seen this error in the past and have resolved it. However, I am back in this situation and cannot recall the cause or resolution. Any help would be greatly appreciated. =ERROR REPORT==== 9-Jan-2015::14:57:09 === Unable to complete Request: badarg Request Dump: {mochiweb_request, [{method,'POST'}, {version,{1,1}}, {raw_path,"/v1/message"}, {headers, [{'Connection',"Keep-Alive"}, {'Content-Length',"277"}, {'Content-Type', "application/x-www-form-urlencoded; charset=US-ASCII"}, {'Host',"10.180.121.59:4778"}, {'User-Agent',"Apache-HttpClient/4.2.6 (java 1.5)"}]}]} Stack Trace: [{omac1,pad_binary_128,1,[{file,"src/omac1.erl"},{line,73}]}, {omac1,cmac_aes_cbc_128,6,[{file,"src/omac1.erl"},{line,66}]}, {pe_util,create_signature,3,[{file,"src/pe_util.erl"},{line,105}]}, {pe_util,verify_signature,4,[{file,"src/pe_util.erl"},{line,109}]}, {pe_auth,authenticate,5,[{file,"src/pe_auth.erl"},{line,50}]}, {pe_rest,intake_post,12,[{file,"src/pe_rest.erl"},{line,603}]}, {pe_rest,handle_request,2,[{file,"src/pe_rest.erl"},{line,136}]}, {pe_rest,'-start_link/0-fun-0-',1,[{file,"src/pe_rest.erl"},{line,23}]}] Best regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Mon Jan 12 20:54:15 2015 From: bryan@REDACTED (Bryan) Date: Mon, 12 Jan 2015 11:54:15 -0800 Subject: [erlang-questions] erlang:send_after/3 questions In-Reply-To: References: Message-ID: Hi Gleb, Hope you don?t mind me digging up this thread, but I was wondering if you could ask you a few questions based on your experience as we are looking at a similar situation where we have 500k (or more) processes active on a single node. Each process kicks off a couple of long running timers using erlang:send_after with dest being self(). We haven?t hit any slowdowns yet, but I am curious to understand your experience better. Were you using erlang:send_after or the timer module? How did you eventually resolve your problem with the locking? Did you have to redesign how you were using the 3 to 5 timers per process so the total number of timers was reduced, or did you do something else, such as stagger the timers so they were not all firing at the same time? Thanks in advance! Cheers, Bryan ---- Bryan Hughes Go Factory http://www.go-factory.net > On Sep 22, 2012, at 1:06 AM, Gleb Peregud wrote: > > In my experience, using excessive timers once bit me hard due to lock contention on global timers lock in Erts. Roughly it was about 3-5 timers created/expired for each of around 200k-500k processes at roughly the same moment (a big broadcast to all processes). Fortunately lcnt showed the problem which was slowing down the broadcast tenfold - lock conflicts was at 95% level on timers lock. > > On Sep 22, 2012 8:07 AM, "Rapsey" > wrote: > > 3. Is spawning a lot of send_after's or timers a dangerous idea? In case of using timer module, we'll just lose some started timers due to port restart. But something tells me that crash of erlang internal timers is a somewhat more dangerous thing that might even crash the VM. > > How much is a lot of send_afters? My server runs a lot of timers, but very rarely in more than one per process. But there can easily be thousands of timers running in different processes. > I would advise against using timer module. Using the timer module in a few places turned out to be a big mistake. It sometimes just stops working completely. It may be a mistake on my part, but erlang:send_after generally does everything you need and is completely reliable. > > > Sergej > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Mon Jan 12 21:53:08 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 12 Jan 2015 21:53:08 +0100 Subject: [erlang-questions] erlang:send_after/3 questions In-Reply-To: References: Message-ID: Maybe a timer pool is needed. It could handle timed calls for a few subscribers and stagger them if it's ok for subscribers. Is there an app which does this already? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Mon Jan 12 22:05:22 2015 From: rvirding@REDACTED (Robert Virding) Date: Mon, 12 Jan 2015 22:05:22 +0100 Subject: [erlang-questions] Ring Benchmark Problem In-Reply-To: References: Message-ID: Your ring loop becomes something like: ring(Next) -> receive {_From,Msg} -> Next ! {self(),Msg}, loop(Next); stop -> Next ! stop, ok end. where we include the sender in the message (might be useful) and a stop message which it sends to the next process and then dies. Starting the processes can be done in a number of ways, for example "backwards" around the ring or each process spawning its successor. Robert On 12 January 2015 at 19:25, Harit Himanshu wrote: > That sounds like a good idea Robert, could you please guide me how to > achieve this? > > Thanks > > On Mon, Jan 12, 2015 at 10:00 AM, Robert Virding > wrote: > >> A much better way than sending the path around is for every process to >> keep track for itself the next process in the ring so you just need to send >> the message between the processes. At least if you are after speed. If you >> do do need to be able to dynamically restructure the ring it would be easy >> to add an extra message where you tell a process its new next process. >> >> Robert >> >> >> On 11 January 2015 at 20:48, Harit Himanshu < >> harit.subscriptions@REDACTED> wrote: >> >>> Hello there, >>> >>> I need help with code review on my attempt to following problem >>> >>> Write a ring benchmark. Create N processes in a ring. Send a message >>> round the ring M times so that a total of N * M messages get sent. Time >>> how long this takes for different values of N and M. >>> >>> I have not timed this yet, and guidance on recommended ways to time is >>> very much appreciated. >>> >>> For now, following is the code for ring that I wrote. Please let me know >>> if there are any shortcomings or code could be written in an much idiomatic >>> way. >>> >>> Thanks a lot >>> + Harit >>> >>> *Code* >>> >>> -module(ring). >>> -author("harith"). >>> >>> %% API >>> -export([message/2]). >>> >>> % create ring of N processes and >>> % send M messages between them >>> message(N, M) when is_integer(N), is_integer(M), N > 0, M > 0 -> >>> Ring = create_ring(N), >>> [Start | T] = Ring, >>> Start ! {T, Ring, 1, M}. >>> >>> create_ring(N) -> >>> Processes = [spawn(fun() -> loop() end) || _ <- lists:seq(1, N)], >>> [H | _] = Processes, >>> lists:append(Processes, [H]). >>> >>> loop() -> >>> receive >>> {[H | T], _L, CurrentMessage, M} -> >>> io:format("~p received ~p~n", [self(), CurrentMessage]), >>> H ! {T, _L, CurrentMessage, M}, >>> loop(); >>> {[], Ring, CurrentMessage, M} -> >>> io:format("~p received ~p with empty list~n", [self(), >>> CurrentMessage]), >>> case CurrentMessage < M of >>> true -> >>> [_ | [Next | T]] = Ring, >>> NewMessage = CurrentMessage + 1, >>> io:format("sending message ~p to ~p~n", [NewMessage, Next]), >>> Next ! {T, Ring, NewMessage, M}; >>> false -> io:format("done sending ~p messages in ~p ring, taking >>> rest now.~n",[M, Ring]) >>> end, >>> loop() >>> end. >>> >>> *Output* >>> 1> ring:message(4, 3). >>> <0.33.0> received 1 >>> {[<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], >>> [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>], >>> 1,3} >>> <0.34.0> received 1 >>> <0.35.0> received 1 >>> 2> <0.36.0> received 1 >>> 2> <0.33.0> received 1 with empty list >>> 2> sending message 2 to <0.34.0> >>> 2> <0.34.0> received 2 >>> 2> <0.35.0> received 2 >>> 2> <0.36.0> received 2 >>> 2> <0.33.0> received 2 with empty list >>> 2> sending message 3 to <0.34.0> >>> 2> <0.34.0> received 3 >>> 2> <0.35.0> received 3 >>> 2> <0.36.0> received 3 >>> 2> <0.33.0> received 3 with empty list >>> 2> done sending 3 messages in >>> [<0.33.0>,<0.34.0>,<0.35.0>,<0.36.0>,<0.33.0>] ring, taking rest now. >>> 2> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Tue Jan 13 00:14:07 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 12 Jan 2015 15:14:07 -0800 Subject: [erlang-questions] Print a message when a process dies Message-ID: I need to make my own function my_spawn(M,F,A) which work similar to spawn(M,F,A) but if a spawned process dies, I need to print a message telling how long a process lived. My attempt looks like -export([my_spawn/3]). my_spawn(Mod, Func, Args) -> Pid = spawn(Mod, Func, Args), Ref = monitor(process, Pid), Start = os:timestamp(), receive {'DOWN', Ref, process, Pid, _Why} -> io:format("process died after ~p microseconds.~n", [timer:now_diff(os:timestamp(), Start)]) end, Pid. so when I run this, I see 1> Pid = error_handling:my_spawn(area_server1, area, []). process died after 363 microseconds. =ERROR REPORT==== 12-Jan-2015::15:06:53 === Error in process <0.33.0> with exit value: {undef,[{area_server1,area,[],[]}]} <0.33.0> 2> *Problems**?* - I am learning both Erlang and concurrency for the first time, so I see following potential problems with this approach a.) What if spawned process failed in the first place, so I get no Pid b.) What if command fails executing either "Ref = .." or "Start = .."? In such scenarios the program will not work effectively, what are the recommended ways to solve such thing? Also, do you see any potential race conditions here? I take a guess saying no because even if multiple process trying to execute the same code, they will return different Pid anyway, is that understanding correct? Thanks a lot + Harit -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Tue Jan 13 00:50:16 2015 From: felixgallo@REDACTED (Felix Gallo) Date: Mon, 12 Jan 2015 15:50:16 -0800 Subject: [erlang-questions] Print a message when a process dies In-Reply-To: References: Message-ID: a) spawn() never fails. You'll get a Pid back, or the BEAM will crash in which case mere erlang programs are the least of your problems. The Pid will then either apply M:F to A, if M:F exists and is of arity befitting A, or apply error_handler:undefined_function to [M,F,A]), which in turns prints out that message you saw unless you've redefined the error handler, which you should definitely not do. b) monitor() also will never fail. If monitor() is called on a non-existent Pid, you will get a DOWN message immediately. monitor() will still return a Ref. (note: this applies to erlang released within the last few years; before, you would get a badarg exception). os:timestamp() will also never fail. In general your approach should be to assume that everything will work out fine and only program, at first, the 'happy path' for your code. Then fix them up as you find valid inputs which cause them to crash. http://erldocs.com/ and http://www.erlang.org/doc/ are really good, and can help you answer a lot of these questions; the functions you list are well documented, for example. If you're using a Mac, I can recommend getting 'Dash' (a documentation-at-your-hotkey app with a search bar) as well. F. On Mon, Jan 12, 2015 at 3:14 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > I need to make my own function my_spawn(M,F,A) which work similar to > spawn(M,F,A) but if a spawned process dies, I need to print a message > telling how long a process lived. > > > My attempt looks like > > -export([my_spawn/3]). > > my_spawn(Mod, Func, Args) -> > Pid = spawn(Mod, Func, Args), > Ref = monitor(process, Pid), > Start = os:timestamp(), > receive > {'DOWN', Ref, process, Pid, _Why} -> > io:format("process died after ~p microseconds.~n", > [timer:now_diff(os:timestamp(), Start)]) > end, > Pid. > > > so when I run this, I see > > 1> Pid = error_handling:my_spawn(area_server1, area, []). > process died after 363 microseconds. > > =ERROR REPORT==== 12-Jan-2015::15:06:53 === > Error in process <0.33.0> with exit value: > {undef,[{area_server1,area,[],[]}]} > > <0.33.0> > 2> > > *Problems**?* > - I am learning both Erlang and concurrency for the first time, so I see > following potential problems with this approach > > a.) What if spawned process failed in the first place, so I get no Pid > b.) What if command fails executing either "Ref = .." or "Start = .."? > > In such scenarios the program will not work effectively, what are the > recommended ways to solve such thing? > > Also, do you see any potential race conditions here? I take a guess saying > no because even if multiple process trying to execute the same code, they > will return different Pid anyway, is that understanding correct? > > Thanks a lot > + Harit > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elixmoon@REDACTED Tue Jan 13 01:29:27 2015 From: elixmoon@REDACTED (Dmitry Aleksandrov) Date: Tue, 13 Jan 2015 01:29:27 +0100 Subject: [erlang-questions] maps with records Message-ID: <54B466E7.6070508@gmail.com> Hi! We found strange behaviour from (As I can judge) erlang compiler, when dealing with records and maps in the same function. Here is example: -module(test_ebug). -compile([export_all]). -record(session, {args, test}). init(#session{args = #{test := Test}}=S) -> io:format(user, "info~n", []), {ok, [], S#session{test = Test}}. test() -> init(#session{args = #{test => 1}}). After compiling and test: 1> test_ebug:test(). info ** exception error: bad argument in function test_ebug:init/1 (test_ebug.erl, line 6) It shows, that info(which happen in line 7), but stacktrace shows line 6, what is wrong. And, this syntax should be correct, because if we make assignment in another function - it works without problem. For me, it seems, that core is generated not right on that place(or at least, I do not understand, what is <_fol3> here): let <_fol3> = ~{~<'test',Test>}~ in let <_cor3> = %% Line 8 call 'erlang':'setelement' (3, S, Test) in %% Line 8 {'ok',[],_cor3} If I replace record syntax, with 'erlang:setelement(3, S, Test)', than core is: let <_cor1> = %% Line 8 call 'erlang':'setelement' (3, S, Test) in %% Line 8 {'ok',[],_cor1} And its working. Should we send it to erlang-bugs? Tested on erlang 17.4. From harit.subscriptions@REDACTED Tue Jan 13 04:52:56 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 12 Jan 2015 19:52:56 -0800 Subject: [erlang-questions] Print a message when a process dies In-Reply-To: References: Message-ID: Thank you very much Felix. really appreciate your time and effort On Mon, Jan 12, 2015 at 3:50 PM, Felix Gallo wrote: > a) spawn() never fails. You'll get a Pid back, or the BEAM will crash in > which case mere erlang programs are the least of your problems. The Pid > will then either apply M:F to A, if M:F exists and is of arity befitting A, > or apply error_handler:undefined_function to [M,F,A]), which in turns > prints out that message you saw unless you've redefined the error handler, > which you should definitely not do. > > b) monitor() also will never fail. If monitor() is called on a > non-existent Pid, you will get a DOWN message immediately. monitor() will > still return a Ref. (note: this applies to erlang released within the last > few years; before, you would get a badarg exception). > > os:timestamp() will also never fail. > > In general your approach should be to assume that everything will work out > fine and only program, at first, the 'happy path' for your code. Then fix > them up as you find valid inputs which cause them to crash. > > http://erldocs.com/ and http://www.erlang.org/doc/ are really good, and > can help you answer a lot of these questions; the functions you list are > well documented, for example. If you're using a Mac, I can recommend > getting 'Dash' (a documentation-at-your-hotkey app with a search bar) as > well. > > F. > > On Mon, Jan 12, 2015 at 3:14 PM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> I need to make my own function my_spawn(M,F,A) which work similar to >> spawn(M,F,A) but if a spawned process dies, I need to print a message >> telling how long a process lived. >> >> >> My attempt looks like >> >> -export([my_spawn/3]). >> >> my_spawn(Mod, Func, Args) -> >> Pid = spawn(Mod, Func, Args), >> Ref = monitor(process, Pid), >> Start = os:timestamp(), >> receive >> {'DOWN', Ref, process, Pid, _Why} -> >> io:format("process died after ~p microseconds.~n", >> [timer:now_diff(os:timestamp(), Start)]) >> end, >> Pid. >> >> >> so when I run this, I see >> >> 1> Pid = error_handling:my_spawn(area_server1, area, []). >> process died after 363 microseconds. >> >> =ERROR REPORT==== 12-Jan-2015::15:06:53 === >> Error in process <0.33.0> with exit value: >> {undef,[{area_server1,area,[],[]}]} >> >> <0.33.0> >> 2> >> >> *Problems**?* >> - I am learning both Erlang and concurrency for the first time, so I see >> following potential problems with this approach >> >> a.) What if spawned process failed in the first place, so I get no Pid >> b.) What if command fails executing either "Ref = .." or "Start = .."? >> >> In such scenarios the program will not work effectively, what are the >> recommended ways to solve such thing? >> >> Also, do you see any potential race conditions here? I take a guess >> saying no because even if multiple process trying to execute the same code, >> they will return different Pid anyway, is that understanding correct? >> >> Thanks a lot >> + Harit >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mloftis@REDACTED Tue Jan 13 07:27:23 2015 From: mloftis@REDACTED (Michael Loftis) Date: Mon, 12 Jan 2015 22:27:23 -0800 Subject: [erlang-questions] Print a message when a process dies In-Reply-To: References: Message-ID: You could also do a spawn_monitor (or possibly spawn_link depending on how you want to handle things) to avoid racing a potentially crashing process. On Mon, Jan 12, 2015 at 7:52 PM, Harit Himanshu wrote: > Thank you very much Felix. really appreciate your time and effort > > On Mon, Jan 12, 2015 at 3:50 PM, Felix Gallo wrote: >> >> a) spawn() never fails. You'll get a Pid back, or the BEAM will crash in >> which case mere erlang programs are the least of your problems. The Pid >> will then either apply M:F to A, if M:F exists and is of arity befitting A, >> or apply error_handler:undefined_function to [M,F,A]), which in turns prints >> out that message you saw unless you've redefined the error handler, which >> you should definitely not do. >> >> b) monitor() also will never fail. If monitor() is called on a >> non-existent Pid, you will get a DOWN message immediately. monitor() will >> still return a Ref. (note: this applies to erlang released within the last >> few years; before, you would get a badarg exception). >> >> os:timestamp() will also never fail. >> >> In general your approach should be to assume that everything will work out >> fine and only program, at first, the 'happy path' for your code. Then fix >> them up as you find valid inputs which cause them to crash. >> >> http://erldocs.com/ and http://www.erlang.org/doc/ are really good, and >> can help you answer a lot of these questions; the functions you list are >> well documented, for example. If you're using a Mac, I can recommend >> getting 'Dash' (a documentation-at-your-hotkey app with a search bar) as >> well. >> >> F. >> >> On Mon, Jan 12, 2015 at 3:14 PM, Harit Himanshu >> wrote: >>> >>> I need to make my own function my_spawn(M,F,A) which work similar to >>> spawn(M,F,A) but if a spawned process dies, I need to print a message >>> telling how long a process lived. >>> >>> >>> My attempt looks like >>> >>> -export([my_spawn/3]). >>> >>> my_spawn(Mod, Func, Args) -> >>> Pid = spawn(Mod, Func, Args), >>> Ref = monitor(process, Pid), >>> Start = os:timestamp(), >>> receive >>> {'DOWN', Ref, process, Pid, _Why} -> >>> io:format("process died after ~p microseconds.~n", >>> [timer:now_diff(os:timestamp(), Start)]) >>> end, >>> Pid. >>> >>> >>> so when I run this, I see >>> >>> 1> Pid = error_handling:my_spawn(area_server1, area, []). >>> process died after 363 microseconds. >>> >>> =ERROR REPORT==== 12-Jan-2015::15:06:53 === >>> Error in process <0.33.0> with exit value: >>> {undef,[{area_server1,area,[],[]}]} >>> >>> <0.33.0> >>> 2> >>> >>> Problems? >>> - I am learning both Erlang and concurrency for the first time, so I see >>> following potential problems with this approach >>> >>> a.) What if spawned process failed in the first place, so I get no Pid >>> b.) What if command fails executing either "Ref = .." or "Start = .."? >>> >>> In such scenarios the program will not work effectively, what are the >>> recommended ways to solve such thing? >>> >>> Also, do you see any potential race conditions here? I take a guess >>> saying no because even if multiple process trying to execute the same code, >>> they will return different Pid anyway, is that understanding correct? >>> >>> Thanks a lot >>> + Harit >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From wallentin.dahlberg@REDACTED Tue Jan 13 10:08:09 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Tue, 13 Jan 2015 10:08:09 +0100 Subject: [erlang-questions] maps with records In-Reply-To: <54B466E7.6070508@gmail.com> References: <54B466E7.6070508@gmail.com> Message-ID: Compiling with erlc +no_copt test_ebug.erl generates no such error. So it looks like an error in core optimization pass. 2015-01-13 1:29 GMT+01:00 Dmitry Aleksandrov : > Hi! > > We found strange behaviour from (As I can judge) erlang compiler, when > dealing with records and maps in the same function. > > Here is example: > > -module(test_ebug). > -compile([export_all]). > > -record(session, {args, test}). > > init(#session{args = #{test := Test}}=S) -> > io:format(user, "info~n", []), > {ok, [], S#session{test = Test}}. > > test() -> init(#session{args = #{test => 1}}). > > After compiling and test: > > 1> test_ebug:test(). > info > ** exception error: bad argument > in function test_ebug:init/1 (test_ebug.erl, line 6) > > It shows, that info(which happen in line 7), but stacktrace shows line 6, > what is wrong. And, this syntax should be correct, because if we make > assignment in another function - it works without problem. > > For me, it seems, that core is generated not right on that place(or at > least, I do not understand, what is <_fol3> here): > > let <_fol3> = > ~{~<'test',Test>}~ > in let <_cor3> = > %% Line 8 > call 'erlang':'setelement' > (3, S, Test) > in %% Line 8 > {'ok',[],_cor3} > > If I replace record syntax, with 'erlang:setelement(3, S, Test)', than > core is: > > let <_cor1> = > %% Line 8 > call 'erlang':'setelement' > (3, S, Test) > in %% Line 8 > {'ok',[],_cor1} > > And its working. > > Should we send it to erlang-bugs? > > Tested on erlang 17.4. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Jan 13 10:28:10 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 13 Jan 2015 10:28:10 +0100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> Message-ID: Thank you Imants. Indeed, timer:apply_interval/4 actually works perfectly, no timer weirdness. I guess the implementation is not depending on the normal receive..after..end loop. For others interested, here's a working code example: -module(test). -export([call_periodical/2]). -export([periodical_fun/1]). call_periodical(Num, IntervalMs) -> Start = epoch_time_ms(), {ok, TRef} = timer:apply_interval(IntervalMs, ?MODULE, periodical_fun, [self()]), wait(Num), io:format("Finished in ~p microseconds.~n", [epoch_time_ms() - Start]), timer:cancel(TRef). wait(0) -> ok; wait(Num) -> receive ok -> wait(Num - 1); _ -> ok end. periodical_fun(Pid) -> Pid ! ok. epoch_time_ms() -> {Mega, Sec, Micro} = os:timestamp(), (Mega * 1000000 + Sec) * 1000 + round(Micro / 1000). Running it: Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V6.3 (abort with ^G) 1> c(test). {ok,test} 2> test:call_periodical(1000, 10). Finished in 10004 microseconds. {ok,cancel} 3> test:call_periodical(2000, 10). Finished in 20002 microseconds. {ok,cancel} 4> test:call_periodical(10000, 1). Finished in 10002 microseconds. {ok,cancel} 5> test:call_periodical(20000, 1). Finished in 20000 microseconds. {ok,cancel} 6> test:call_periodical(1000, 1). Finished in 1002 microseconds. {ok,cancel} Where everything works as expected. Thank you everyone! Best, r. On Mon, Jan 12, 2015 at 6:14 PM, Imants Cekusins wrote: > btw CPU looked ok: was spread across cores and did not max out. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Jan 13 10:44:09 2015 From: bob@REDACTED (Bob Ippolito) Date: Tue, 13 Jan 2015 20:44:09 +1100 Subject: [erlang-questions] I'm having a SNAFU moment: timer:sleep/1 In-Reply-To: References: <3CA79191-8B6D-4C38-8C14-65BFE60288DA@gmail.com> Message-ID: The implementation of the timer module does rely on the same sleeping machinery as receive (gen_server timeout) but it doesn't suffer drift by using the technique that Darach suggested. It keeps track of timers in terms of absolute time, and when it wakes up it samples the clock to see which timers need to fire. Nothing special about the timer module, it just uses a better algorithm... although it is a single gen_server process per node, so having a lot of timers can be a bottleneck. https://github.com/erlang/otp/blob/maint/lib/stdlib/src/timer.erl#L345-L375 On Tue, Jan 13, 2015 at 8:28 PM, Roberto Ostinelli wrote: > Thank you Imants. > Indeed, timer:apply_interval/4 actually works perfectly, no timer > weirdness. I guess the implementation is not depending on the normal > receive..after..end loop. > > For others interested, here's a working code example: > > -module(test). > -export([call_periodical/2]). > -export([periodical_fun/1]). > > call_periodical(Num, IntervalMs) -> > Start = epoch_time_ms(), > {ok, TRef} = timer:apply_interval(IntervalMs, ?MODULE, periodical_fun, > [self()]), > wait(Num), > io:format("Finished in ~p microseconds.~n", [epoch_time_ms() - Start]), > timer:cancel(TRef). > > wait(0) -> ok; > wait(Num) -> > receive > ok -> wait(Num - 1); > _ -> ok > end. > > periodical_fun(Pid) -> > Pid ! ok. > > epoch_time_ms() -> > {Mega, Sec, Micro} = os:timestamp(), > (Mega * 1000000 + Sec) * 1000 + round(Micro / 1000). > > > Running it: > > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V6.3 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:call_periodical(1000, 10). > Finished in 10004 microseconds. > {ok,cancel} > 3> test:call_periodical(2000, 10). > Finished in 20002 microseconds. > {ok,cancel} > 4> test:call_periodical(10000, 1). > Finished in 10002 microseconds. > {ok,cancel} > 5> test:call_periodical(20000, 1). > Finished in 20000 microseconds. > {ok,cancel} > 6> test:call_periodical(1000, 1). > Finished in 1002 microseconds. > {ok,cancel} > > Where everything works as expected. > > Thank you everyone! > > Best, > r. > > > On Mon, Jan 12, 2015 at 6:14 PM, Imants Cekusins wrote: > >> btw CPU looked ok: was spread across cores and did not max out. >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rabbe.fogelholm@REDACTED Tue Jan 13 12:16:03 2015 From: rabbe.fogelholm@REDACTED (Rabbe Fogelholm) Date: Tue, 13 Jan 2015 12:16:03 +0100 Subject: [erlang-questions] Multiple SFTP subsystem_spec() when invoking ssh:daemon/2, is it possible and how would it work? In-Reply-To: References: <54B466E7.6070508@gmail.com> Message-ID: <54B4FE73.4060405@ericsson.com> The subsystems option in ssh:daemon/2 and ssh:daemon/3 is list-valued, which indicates that there can be more than one value. It is mentioned that the option value can be the empty list, but what about multiple values? Each list value would be a tuple created with the ssh_sftpd:subsystem_spec/1 function. But what if the 'cwd' options are different? And what if there are multiple and contradictory 'root' options? And would it be possible a separate file_handler option for each subsystem spec? (this is what I am after in fact). Rabbe Fogelholm, Ericsson, Stockholm From vladdu55@REDACTED Tue Jan 13 13:36:25 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 13:36:25 +0100 Subject: [erlang-questions] Tracing and debugging Message-ID: Hi all! I just got a crazy idea and I think it's not that crazy, but I'm willing to get feedback on that :-) The main reason I use the debugger is to check that the intermediary values in a computation are the expected ones. The alternative (which works without messing up timeouts) is to print out values at points of interest, but it is messy (there's a lot of boilerplate to type and the interesting code becomes hard to read). So, I thought, what if, instead of interpreting a module in order to debug it, we compile it with a special parse transform that inserts tracing calls after each expression in the code, automatically keeping track of the variables visible in the scope and their values? The output can be via io:format, et:report_event, or something else (configurable). Could this "trogging" or "logtracing" be useful, or am I delusional? Maybe there already is such a thing, the Erlang ecosystem is getting difficult to keep in the working memory :-) best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Tue Jan 13 13:48:30 2015 From: elbrujohalcon@REDACTED (Fernando 'Brujo' Benavides) Date: Tue, 13 Jan 2015 09:48:30 -0300 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: <7F838479-447E-40C7-BB21-81CF2F9E15EF@inaka.net> I think that?s pretty close to what redbug does. I use redbug for debugging purposes in almost all of my projects. In fact, eper is one of my ?default dependencies?, together with lager and sync :) Fernando "Brujo" Benavides about.me/elbrujohalcon > On Jan 13, 2015, at 09:36, Vlad Dumitrescu wrote: > > Hi all! > > I just got a crazy idea and I think it's not that crazy, but I'm willing to get feedback on that :-) > > The main reason I use the debugger is to check that the intermediary values in a computation are the expected ones. The alternative (which works without messing up timeouts) is to print out values at points of interest, but it is messy (there's a lot of boilerplate to type and the interesting code becomes hard to read). > > So, I thought, what if, instead of interpreting a module in order to debug it, we compile it with a special parse transform that inserts tracing calls after each expression in the code, automatically keeping track of the variables visible in the scope and their values? > > The output can be via io:format, et:report_event, or something else (configurable). > > Could this "trogging" or "logtracing" be useful, or am I delusional? Maybe there already is such a thing, the Erlang ecosystem is getting difficult to keep in the working memory :-) > > best regards, > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Tue Jan 13 13:51:46 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 13 Jan 2015 13:51:46 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: <7F838479-447E-40C7-BB21-81CF2F9E15EF@inaka.net> References: <7F838479-447E-40C7-BB21-81CF2F9E15EF@inaka.net> Message-ID: Does redbug log variables inside functions, or only calls to functions, args & return values? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Jan 13 13:54:17 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 13 Jan 2015 07:54:17 -0500 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: <20150113125416.GB7939@ferdair.local> On 01/13, Vlad Dumitrescu wrote: > So, I thought, what if, instead of interpreting a module in order to debug > it, we compile it with a special parse transform that inserts tracing calls > after each expression in the code, automatically keeping track of the > variables visible in the scope and their values? > > The output can be via io:format, et:report_event, or something else > (configurable). > > Could this "trogging" or "logtracing" be useful, or am I delusional? Maybe > there already is such a thing, the Erlang ecosystem is getting difficult to > keep in the working memory :-) > How would this be different from tracing BIFs and the libraries built on top? These even let you specify things if you want to look at values put in a function, and returned from it. I'm guessing the part that's too annoying could be selecting the functions that require tracing, maybe? Or that adding debug id functions (`trace(X) -> X.', then trace on ?MODULE:trace/1) is too cumbersome? You could probably use stuff like xref to find all calls that take place in a module, and then fill the input into trace BIFs to get an equivalent result, though there's probably no great scaffolding in place for this today. From elbrujohalcon@REDACTED Tue Jan 13 13:54:27 2015 From: elbrujohalcon@REDACTED (Fernando 'Brujo' Benavides) Date: Tue, 13 Jan 2015 09:54:27 -0300 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: <7F838479-447E-40C7-BB21-81CF2F9E15EF@inaka.net> Message-ID: <7B76B91F-0A77-4090-AB59-60E864A587CF@inaka.net> I think only calls to functions, args & return values. Maybe Mats is around and can shed some more light on the subject ;) Fernando "Brujo" Benavides about.me/elbrujohalcon > On Jan 13, 2015, at 09:51, Imants Cekusins wrote: > > Does redbug log variables inside functions, or only calls to functions, args & return values? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladimir.ralev@REDACTED Tue Jan 13 13:56:00 2015 From: vladimir.ralev@REDACTED (Vladimir Ralev) Date: Tue, 13 Jan 2015 14:56:00 +0200 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: Further it will be awesome if there is a UI debug tool where you can step through the code on a timeline based on the generated logs without having to read them. Java has Chronon debugger which works sort of this way but records more metadata due to the nature of Java. In Erlang it will be easier. On Tue, Jan 13, 2015 at 2:36 PM, Vlad Dumitrescu wrote: > Hi all! > > I just got a crazy idea and I think it's not that crazy, but I'm willing to > get feedback on that :-) > > The main reason I use the debugger is to check that the intermediary values > in a computation are the expected ones. The alternative (which works without > messing up timeouts) is to print out values at points of interest, but it is > messy (there's a lot of boilerplate to type and the interesting code becomes > hard to read). > > So, I thought, what if, instead of interpreting a module in order to debug > it, we compile it with a special parse transform that inserts tracing calls > after each expression in the code, automatically keeping track of the > variables visible in the scope and their values? > > The output can be via io:format, et:report_event, or something else > (configurable). > > Could this "trogging" or "logtracing" be useful, or am I delusional? Maybe > there already is such a thing, the Erlang ecosystem is getting difficult to > keep in the working memory :-) > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From imantc@REDACTED Tue Jan 13 14:11:59 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 13 Jan 2015 14:11:59 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: Yep, pt can be written to insert hooks to call trace function defined in a custom (configurable) module: trace(Module, Line, Var_name, Var_value) -> ... From lostcolony@REDACTED Tue Jan 13 14:31:32 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Tue, 13 Jan 2015 08:31:32 -0500 Subject: [erlang-questions] erlang-questions Digest, Vol 200, Issue 4 In-Reply-To: References: Message-ID: Not to keep beating on this, but, one thing I noticed when testing timers in Erlang a couple years ago was that many -closely clumped- timers would lead to skew. That is, apply_interval works cleanly because it only ever has a single timer running, and just calculates how long to sleep for in between each call. The problem is trying to create many processes, with timers all having very close together stop times leads to very noticeable drift. The closer they are together, the worse it is (as you noticed in your original question). Having just as many timers, but with greater space between their firing, leads to less drift, and at large enough spacing essentially no drift. > ------------------------------ > > Date: Tue, 13 Jan 2015 20:44:09 +1100 > From: Bob Ippolito > To: Roberto Ostinelli > Cc: Erlang Questions > Subject: Re: [erlang-questions] I'm having a SNAFU moment: > timer:sleep/1 > Message-ID: > < > CACwMPm-sZx9a5ZcF+kh8SE1S3E4r6WYCT8Ch82QHUEFUwt2Y6Q@REDACTED> > Content-Type: text/plain; charset="utf-8" > > The implementation of the timer module does rely on the same sleeping > machinery as receive (gen_server timeout) but it doesn't suffer drift by > using the technique that Darach suggested. It keeps track of timers in > terms of absolute time, and when it wakes up it samples the clock to see > which timers need to fire. Nothing special about the timer module, it just > uses a better algorithm... although it is a single gen_server process per > node, so having a lot of timers can be a bottleneck. > > https://github.com/erlang/otp/blob/maint/lib/stdlib/src/timer.erl#L345-L375 > > On Tue, Jan 13, 2015 at 8:28 PM, Roberto Ostinelli > wrote: > > > Thank you Imants. > > Indeed, timer:apply_interval/4 actually works perfectly, no timer > > weirdness. I guess the implementation is not depending on the normal > > receive..after..end loop. > > > > For others interested, here's a working code example: > > > > -module(test). > > -export([call_periodical/2]). > > -export([periodical_fun/1]). > > > > call_periodical(Num, IntervalMs) -> > > Start = epoch_time_ms(), > > {ok, TRef} = timer:apply_interval(IntervalMs, ?MODULE, periodical_fun, > > [self()]), > > wait(Num), > > io:format("Finished in ~p microseconds.~n", [epoch_time_ms() - Start]), > > timer:cancel(TRef). > > > > wait(0) -> ok; > > wait(Num) -> > > receive > > ok -> wait(Num - 1); > > _ -> ok > > end. > > > > periodical_fun(Pid) -> > > Pid ! ok. > > > > epoch_time_ms() -> > > {Mega, Sec, Micro} = os:timestamp(), > > (Mega * 1000000 + Sec) * 1000 + round(Micro / 1000). > > > > > > Running it: > > > > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] > > [hipe] [kernel-poll:false] [dtrace] > > > > Eshell V6.3 (abort with ^G) > > 1> c(test). > > {ok,test} > > 2> test:call_periodical(1000, 10). > > Finished in 10004 microseconds. > > {ok,cancel} > > 3> test:call_periodical(2000, 10). > > Finished in 20002 microseconds. > > {ok,cancel} > > 4> test:call_periodical(10000, 1). > > Finished in 10002 microseconds. > > {ok,cancel} > > 5> test:call_periodical(20000, 1). > > Finished in 20000 microseconds. > > {ok,cancel} > > 6> test:call_periodical(1000, 1). > > Finished in 1002 microseconds. > > {ok,cancel} > > > > Where everything works as expected. > > > > Thank you everyone! > > > > Best, > > r. > > > > > > On Mon, Jan 12, 2015 at 6:14 PM, Imants Cekusins > wrote: > > > >> btw CPU looked ok: was spread across cores and did not max out. > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://erlang.org/pipermail/erlang-questions/attachments/20150113/dc386403/attachment-0001.html > > > > ------------------------------ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > End of erlang-questions Digest, Vol 200, Issue 4 > ************************************************ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jan 13 14:57:38 2015 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 13 Jan 2015 14:57:38 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: What really should be done is that the current interpreting debugger should be destroyed with fire and a real bytecode-level debugger should be implemented (with low level functionality done in C), which would be way faster, only require updating when new Beam operations are added, and not require the abstract code to be present in the beam files. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Jan 13 15:11:10 2015 From: mihai@REDACTED (Mihai Balea) Date: Tue, 13 Jan 2015 09:11:10 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: > On Jan 12, 2015, at 4:10 AM, T Ty wrote: > > The LMAX architecture is an easy fit for Erlang and one which an Erlanger would naturally arrive to. I beg to differ. LMAX uses a single threaded process to implement its entire business logic, which makes it highly sensitive to sequential performance. This is not one of Erlang?s strong points. Also, the disruptors - which are essentially fancy circular buffers - depend on preallocated memory and destructive updates to attain their performance. Finally, the communication between various stages in the pipeline is done through shared memory, and they do some clever tricks to avoid locking. Yes, you could emulate this architecture in Erlang, but you won?t get anywhere near LMAX?s performance. Mihai From vladdu55@REDACTED Tue Jan 13 15:18:46 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 15:18:46 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: <20150113125416.GB7939@ferdair.local> References: <20150113125416.GB7939@ferdair.local> Message-ID: The major difference is that you can trace intermediary results without polluting the code with dummy ?trace calls. Having the variables selected automatically is nice too, no way to forget some or refer to the wrong one. Some more advanced stuff can be done too, like only tracing variables whose values have changed. In other words, why do things manually if it was possible and efficient to have them don for you by the compiler? regards, Vlad On Tue, Jan 13, 2015 at 1:54 PM, Fred Hebert wrote: > On 01/13, Vlad Dumitrescu wrote: > > So, I thought, what if, instead of interpreting a module in order to > debug > > it, we compile it with a special parse transform that inserts tracing > calls > > after each expression in the code, automatically keeping track of the > > variables visible in the scope and their values? > > > > The output can be via io:format, et:report_event, or something else > > (configurable). > > > > Could this "trogging" or "logtracing" be useful, or am I delusional? > Maybe > > there already is such a thing, the Erlang ecosystem is getting difficult > to > > keep in the working memory :-) > > > > How would this be different from tracing BIFs and the libraries built on > top? These even let you specify things if you want to look at values put > in a function, and returned from it. > > I'm guessing the part that's too annoying could be selecting the > functions that require tracing, maybe? Or that adding debug id functions > (`trace(X) -> X.', then trace on ?MODULE:trace/1) is too cumbersome? > > You could probably use stuff like xref to find all calls that take place > in a module, and then fill the input into trace BIFs to get an > equivalent result, though there's probably no great scaffolding in place > for this today. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jan 13 15:22:42 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 13 Jan 2015 15:22:42 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: <20150113125416.GB7939@ferdair.local> Message-ID: <54B52A32.5060402@ninenines.eu> On 01/13/2015 03:18 PM, Vlad Dumitrescu wrote: > Some more advanced stuff can be done too, like only > tracing variables whose values have changed. Curious what you mean by that? -- Lo?c Hoguin http://ninenines.eu From vladdu55@REDACTED Tue Jan 13 15:22:50 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 15:22:50 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: Yes, I was a bit inspired by Chronon too. I thought I'd introduce it at a later stage, as it's not exactly the same thing. Chronon saves the program's execution in a very efficient way and lets the user go back and forth, inspecting all variables and parameters and results. One can even make changes to the data and see what would have happened. regards, Vlad On Tue, Jan 13, 2015 at 1:56 PM, Vladimir Ralev wrote: > Further it will be awesome if there is a UI debug tool where you can > step through the code on a timeline based on the generated logs > without having to read them. Java has Chronon debugger which works > sort of this way but records more metadata due to the nature of Java. > In Erlang it will be easier. > > On Tue, Jan 13, 2015 at 2:36 PM, Vlad Dumitrescu > wrote: > > Hi all! > > > > I just got a crazy idea and I think it's not that crazy, but I'm willing > to > > get feedback on that :-) > > > > The main reason I use the debugger is to check that the intermediary > values > > in a computation are the expected ones. The alternative (which works > without > > messing up timeouts) is to print out values at points of interest, but > it is > > messy (there's a lot of boilerplate to type and the interesting code > becomes > > hard to read). > > > > So, I thought, what if, instead of interpreting a module in order to > debug > > it, we compile it with a special parse transform that inserts tracing > calls > > after each expression in the code, automatically keeping track of the > > variables visible in the scope and their values? > > > > The output can be via io:format, et:report_event, or something else > > (configurable). > > > > Could this "trogging" or "logtracing" be useful, or am I delusional? > Maybe > > there already is such a thing, the Erlang ecosystem is getting difficult > to > > keep in the working memory :-) > > > > best regards, > > Vlad > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Jan 13 15:31:33 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 15:31:33 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: <54B52A32.5060402@ninenines.eu> References: <20150113125416.GB7939@ferdair.local> <54B52A32.5060402@ninenines.eu> Message-ID: On Tue, Jan 13, 2015 at 3:22 PM, Lo?c Hoguin wrote: > On 01/13/2015 03:18 PM, Vlad Dumitrescu wrote: > >> Some more advanced stuff can be done too, like only >> tracing variables whose values have changed. >> > > Curious what you mean by that? > > I mean that for the following code {A, B} = {a, b}, {B, C} = {b, c} the simplistic approach would trace A and B after the first expression and B and C after the other. But B has the same value and it's useless to output it again. If the implementation could be done efficiently enough, this could even detect "chain updates" and present them as changes to a single meta-variable. So S = #rec{}, S1= S#rec{a=1}, S2=S1#rec{b=2} might not output the potentially large record each time, but only the changed values. This is probably something better done offline, as it's a presentation thing. I am thinking out loud, basically, so thanks for challenging me. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Jan 13 15:34:31 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 15:34:31 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: On Tue, Jan 13, 2015 at 2:57 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > What really should be done is that the current interpreting debugger > should be destroyed with fire and a real bytecode-level debugger should be > implemented (with low level functionality done in C), which would be way > faster, only require updating when new Beam operations are added, and not > require the abstract code to be present in the beam files. > A new bytecode-level debugger is not something I can implement myself, but the other one is. Even so, I think that tracing would still have its place, as stopping on a breakpoint will still trigger all sorts of timeouts in a live system. regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Tue Jan 13 15:39:03 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Tue, 13 Jan 2015 09:39:03 -0500 Subject: [erlang-questions] Trouble with JInterface Message-ID: Folks, We are having a problem with Jinterface and Erlang 17.2. Essentially we have a Java/Clojure application that is using a home-brewed, Erlang, two-node, in-memory cache. We think the problem is either a bug with Jinterface or a misunderstanding on how to use it. On the Java side, we create a node and 1 to 12 unnamed mailboxes. We have a receive loop on each mailbox: receiving messages from Erlang, doing the right things with those messages, catching any errors and looping back to the receive. In general, everything works fine. When one of the Erlang cache-nodes goes down, the application recreates its mailboxes, connects to the ?other? Erlang node via the new mailboxes and attempts to carry on. Here is where we have the problem: The Failover Protocol: For each mailbox 1. Receive loop gets OtpErlangExit 2. Existing mailbox is closed 3. New unnamed mailbox created 4. Application appropriate message is sent to the failover node via this new mailbox 5. Loop to the receive Symptom: If the receive loop has a timeout, no message is received by the new mailbox until AFTER the timeout fires and the loop iterates back to the receive. If there is no timeout specified the receive hangs (seemingly) forever. We have confirmed that the failover Erlang node received the message (step 4) sent via the new mailbox, and that the failover Erlang node is sending messages to the PID representing the new mailbox ? inspecting my server using Observer, I can see that it is sending the expected message to the correct PID. Our workaround is to use a short timeout in our mailbox receive loop so that we don?t hang for too long, but we are troubled by the thought that we might not understand JInterface best-practices as we are skeptical that the JInterface code is anything but perfect :) Is this a known issue, or does someone have any advice on how to proceed? Thanks, Jr0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Tue Jan 13 15:41:10 2015 From: davidnwelton@REDACTED (David Welton) Date: Tue, 13 Jan 2015 15:41:10 +0100 Subject: [erlang-questions] Graphical display of messages Message-ID: Hi, I'm wondering if something like this exists: I'd like to create various graphics to show the relationships between different processes and/or applications. I could see a couple of different displays: * The more messages go back and forth, the stronger a link is shown. I think graphviz probably has a way to generate this. * A "swim-lane" graphic that tracks who sent what when. Useful for getting an idea of how a system works. Not sure how to generate that one. I envision it working like so: 1) Gather a list of pids for an application. 2) Trace those pids send/receive operations, and write those to a file. 3) Process the file and generate the desired graph... Is there something like that out there already? This is pretty cool, and similar to what I'm interested in, but not exactly: https://www.youtube.com/watch?v=lHoWfeNuAN8 - https://github.com/krestenkrab/erlubi Thank you -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From vladdu55@REDACTED Tue Jan 13 16:01:00 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Jan 2015 16:01:00 +0100 Subject: [erlang-questions] Trouble with JInterface In-Reply-To: References: Message-ID: Hi Jim! Does this happen every time, for all mailboxes? Did you try to trace the Java code, to see if there is more information to get? Use -DOtpConnection.trace=3 and you will get all the activity on the connection from the Java side (it's global, so it will be for _everything_, you might want to try starting with a value of 1 at first and increase if it's not enough). regards, Vlad On Tue, Jan 13, 2015 at 3:39 PM, jim rosenblum wrote: > Folks, > > We are having a problem with Jinterface and Erlang 17.2. Essentially we > have a Java/Clojure application that is using a home-brewed, Erlang, > two-node, in-memory cache. We think the problem is either a bug with > Jinterface or a misunderstanding on how to use it. > > On the Java side, we create a node and 1 to 12 unnamed mailboxes. We have > a receive loop on each mailbox: receiving messages from Erlang, doing the > right things with those messages, catching any errors and looping back to > the receive. In general, everything works fine. > > When one of the Erlang cache-nodes goes down, the application recreates > its mailboxes, connects to the ?other? Erlang node via the new mailboxes > and attempts to carry on. Here is where we have the problem: > > The Failover Protocol: > For each mailbox > 1. Receive loop gets OtpErlangExit > 2. Existing mailbox is closed > 3. New unnamed mailbox created > 4. Application appropriate message is sent to the failover node via this > new mailbox > 5. Loop to the receive > > Symptom: > If the receive loop has a timeout, no message is received by the new > mailbox until AFTER the timeout fires and the loop iterates back to the > receive. If there is no timeout specified the receive hangs (seemingly) > forever. We have confirmed that the failover Erlang node received the > message (step 4) sent via the new mailbox, and that the failover Erlang > node is sending messages to the PID representing the new mailbox ? > inspecting my server using Observer, I can see that it is sending the > expected message to the correct PID. > > Our workaround is to use a short timeout in our mailbox receive loop so > that we don?t hang for too long, but we are troubled by the thought that we > might not understand JInterface best-practices as we are skeptical that the > JInterface code is anything but perfect :) > > Is this a known issue, or does someone have any advice on how to proceed? > > > Thanks, > > Jr0 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Tue Jan 13 16:10:48 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Tue, 13 Jan 2015 16:10:48 +0100 Subject: [erlang-questions] Graphical display of messages In-Reply-To: References: Message-ID: I started something like that ages ago: https://github.com/psyeugenic/fgraph using wx But that needs to be productified, or at the very least .. some love. 2015-01-13 15:41 GMT+01:00 David Welton : > Hi, > > I'm wondering if something like this exists: > > I'd like to create various graphics to show the relationships between > different processes and/or applications. I could see a couple of > different displays: > > * The more messages go back and forth, the stronger a link is shown. > I think graphviz probably has a way to generate this. > > * A "swim-lane" graphic that tracks who sent what when. Useful for > getting an idea of how a system works. Not sure how to generate that > one. > > I envision it working like so: > > 1) Gather a list of pids for an application. > > 2) Trace those pids send/receive operations, and write those to a file. > > 3) Process the file and generate the desired graph... > > Is there something like that out there already? > > This is pretty cool, and similar to what I'm interested in, but not > exactly: https://www.youtube.com/watch?v=lHoWfeNuAN8 - > https://github.com/krestenkrab/erlubi > > Thank you > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tty.erlang@REDACTED Tue Jan 13 16:36:59 2015 From: tty.erlang@REDACTED (T Ty) Date: Tue, 13 Jan 2015 15:36:59 +0000 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: Your comment is quite interesting. My take-away from reading the LMAX article wasn't the single threaded business logic process nor the shared memory communication. My take-aways were: 1. pipelining. Do all input validation and transformation before/after main business logic processing. This makes coding the 'happy-path' easier and keeps the business logic core small. 2. the business logic process is a master process that reacts to events and generate events. This allows scaling of the business logic process as it makes it easier to spawn processes to deal with the events. (gen_event/gen_fsm triggering supervisor in simple_one_for_one that starts other processes) 3. Event Sourcing. One added benefit is compliance. It is easier to demonstrate compliance when there are events to show how the system changed from one state to another. Compliance here can mean security compliance or regulatory compliance. 4. business logic processor deals with events in-memory. mnesia ram_tables fit this criteria easily as do ets tables. 5. Business Logic State Snapshot. Not only does this allow resilience between node restarts it also allows migrating the partial process to a different node entirely. 6. Avoiding external service calls in the Business Logic process. Transition to an event based model for interacting with external services. Have a middle layer of gen_fsm to determine where in the logic the current process is in. 7. Mention about session data being transient and can be discarded. Aka let process die and supervisor restart. The Disruptor sounds like Riak. Cheers On Tue, Jan 13, 2015 at 2:11 PM, Mihai Balea wrote: > > > On Jan 12, 2015, at 4:10 AM, T Ty wrote: > > > > The LMAX architecture is an easy fit for Erlang and one which an > Erlanger would naturally arrive to. > > I beg to differ. LMAX uses a single threaded process to implement its > entire business logic, which makes it highly sensitive to sequential > performance. This is not one of Erlang?s strong points. Also, the > disruptors - which are essentially fancy circular buffers - depend on > preallocated memory and destructive updates to attain their performance. > Finally, the communication between various stages in the pipeline is done > through shared memory, and they do some clever tricks to avoid locking. > > Yes, you could emulate this architecture in Erlang, but you won?t get > anywhere near LMAX?s performance. > > Mihai > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Tue Jan 13 17:20:12 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Tue, 13 Jan 2015 11:20:12 -0500 Subject: [erlang-questions] Trouble with JInterface In-Reply-To: References: Message-ID: It happens for all mailboxes. Outside of the failover scenario, everything functions perfectly. So it happens for all mailboxes that are in their receive loop when the OtpErlangExit is received. We will try the tracing with the -DOtpConnection.trace parameter as you suggested - we haven't done that yet. thanks On Tue, Jan 13, 2015 at 10:01 AM, Vlad Dumitrescu wrote: > Hi Jim! > > Does this happen every time, for all mailboxes? > > Did you try to trace the Java code, to see if there is more information to > get? > Use -DOtpConnection.trace=3 and you will get all the activity on the > connection from the Java side (it's global, so it will be for _everything_, > you might want to try starting with a value of 1 at first and increase if > it's not enough). > > regards, > Vlad > > > On Tue, Jan 13, 2015 at 3:39 PM, jim rosenblum > wrote: > >> Folks, >> >> We are having a problem with Jinterface and Erlang 17.2. Essentially we >> have a Java/Clojure application that is using a home-brewed, Erlang, >> two-node, in-memory cache. We think the problem is either a bug with >> Jinterface or a misunderstanding on how to use it. >> >> On the Java side, we create a node and 1 to 12 unnamed mailboxes. We have >> a receive loop on each mailbox: receiving messages from Erlang, doing the >> right things with those messages, catching any errors and looping back to >> the receive. In general, everything works fine. >> >> When one of the Erlang cache-nodes goes down, the application recreates >> its mailboxes, connects to the ?other? Erlang node via the new mailboxes >> and attempts to carry on. Here is where we have the problem: >> >> The Failover Protocol: >> For each mailbox >> 1. Receive loop gets OtpErlangExit >> 2. Existing mailbox is closed >> 3. New unnamed mailbox created >> 4. Application appropriate message is sent to the failover node via this >> new mailbox >> 5. Loop to the receive >> >> Symptom: >> If the receive loop has a timeout, no message is received by the new >> mailbox until AFTER the timeout fires and the loop iterates back to the >> receive. If there is no timeout specified the receive hangs (seemingly) >> forever. We have confirmed that the failover Erlang node received the >> message (step 4) sent via the new mailbox, and that the failover Erlang >> node is sending messages to the PID representing the new mailbox ? >> inspecting my server using Observer, I can see that it is sending the >> expected message to the correct PID. >> >> Our workaround is to use a short timeout in our mailbox receive loop so >> that we don?t hang for too long, but we are troubled by the thought that we >> might not understand JInterface best-practices as we are skeptical that the >> JInterface code is anything but perfect :) >> >> Is this a known issue, or does someone have any advice on how to proceed? >> >> >> Thanks, >> >> Jr0 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.hoffmann@REDACTED Tue Jan 13 16:34:03 2015 From: torben.hoffmann@REDACTED (Torben Hoffmann) Date: Tue, 13 Jan 2015 16:34:03 +0100 Subject: [erlang-questions] Graphical display of messages In-Reply-To: References: Message-ID: Hi David, For the "swim-lane" aka MSC view you can consider using the et application: http://www.erlang.org/doc/man/et.html The et_viewer can show a really nice MSC of what is going on. I used it with QuickCheck to get an MSC of a failing test case as it was easier to figure out what was going on that way. Cheers, Torben David Welton writes: > Hi, > > I'm wondering if something like this exists: > > I'd like to create various graphics to show the relationships between > different processes and/or applications. I could see a couple of > different displays: > > * The more messages go back and forth, the stronger a link is shown. > I think graphviz probably has a way to generate this. > > * A "swim-lane" graphic that tracks who sent what when. Useful for > getting an idea of how a system works. Not sure how to generate that > one. > > I envision it working like so: > > 1) Gather a list of pids for an application. > > 2) Trace those pids send/receive operations, and write those to a file. > > 3) Process the file and generate the desired graph... > > Is there something like that out there already? > > This is pretty cool, and similar to what I'm interested in, but not > exactly: https://www.youtube.com/watch?v=lHoWfeNuAN8 - > https://github.com/krestenkrab/erlubi > > Thank you -- Torben Hoffmann CTO Erlang Solutions Ltd. Tel: +45 25 14 05 38 http://www.erlang-solutions.com From harit.subscriptions@REDACTED Tue Jan 13 17:25:23 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 08:25:23 -0800 Subject: [erlang-questions] Print a message when a process dies In-Reply-To: References: Message-ID: Thanks Michael for pointing to spawn_monitor/3, that is what I was looking for + Harit On Mon, Jan 12, 2015 at 10:27 PM, Michael Loftis wrote: > You could also do a spawn_monitor (or possibly spawn_link depending on > how you want to handle things) to avoid racing a potentially crashing > process. > > On Mon, Jan 12, 2015 at 7:52 PM, Harit Himanshu > wrote: > > Thank you very much Felix. really appreciate your time and effort > > > > On Mon, Jan 12, 2015 at 3:50 PM, Felix Gallo > wrote: > >> > >> a) spawn() never fails. You'll get a Pid back, or the BEAM will crash > in > >> which case mere erlang programs are the least of your problems. The Pid > >> will then either apply M:F to A, if M:F exists and is of arity > befitting A, > >> or apply error_handler:undefined_function to [M,F,A]), which in turns > prints > >> out that message you saw unless you've redefined the error handler, > which > >> you should definitely not do. > >> > >> b) monitor() also will never fail. If monitor() is called on a > >> non-existent Pid, you will get a DOWN message immediately. monitor() > will > >> still return a Ref. (note: this applies to erlang released within the > last > >> few years; before, you would get a badarg exception). > >> > >> os:timestamp() will also never fail. > >> > >> In general your approach should be to assume that everything will work > out > >> fine and only program, at first, the 'happy path' for your code. Then > fix > >> them up as you find valid inputs which cause them to crash. > >> > >> http://erldocs.com/ and http://www.erlang.org/doc/ are really good, and > >> can help you answer a lot of these questions; the functions you list are > >> well documented, for example. If you're using a Mac, I can recommend > >> getting 'Dash' (a documentation-at-your-hotkey app with a search bar) as > >> well. > >> > >> F. > >> > >> On Mon, Jan 12, 2015 at 3:14 PM, Harit Himanshu > >> wrote: > >>> > >>> I need to make my own function my_spawn(M,F,A) which work similar to > >>> spawn(M,F,A) but if a spawned process dies, I need to print a message > >>> telling how long a process lived. > >>> > >>> > >>> My attempt looks like > >>> > >>> -export([my_spawn/3]). > >>> > >>> my_spawn(Mod, Func, Args) -> > >>> Pid = spawn(Mod, Func, Args), > >>> Ref = monitor(process, Pid), > >>> Start = os:timestamp(), > >>> receive > >>> {'DOWN', Ref, process, Pid, _Why} -> > >>> io:format("process died after ~p microseconds.~n", > >>> [timer:now_diff(os:timestamp(), Start)]) > >>> end, > >>> Pid. > >>> > >>> > >>> so when I run this, I see > >>> > >>> 1> Pid = error_handling:my_spawn(area_server1, area, []). > >>> process died after 363 microseconds. > >>> > >>> =ERROR REPORT==== 12-Jan-2015::15:06:53 === > >>> Error in process <0.33.0> with exit value: > >>> {undef,[{area_server1,area,[],[]}]} > >>> > >>> <0.33.0> > >>> 2> > >>> > >>> Problems? > >>> - I am learning both Erlang and concurrency for the first time, so I > see > >>> following potential problems with this approach > >>> > >>> a.) What if spawned process failed in the first place, so I get no Pid > >>> b.) What if command fails executing either "Ref = .." or "Start = .."? > >>> > >>> In such scenarios the program will not work effectively, what are the > >>> recommended ways to solve such thing? > >>> > >>> Also, do you see any potential race conditions here? I take a guess > >>> saying no because even if multiple process trying to execute the same > code, > >>> they will return different Pid anyway, is that understanding correct? > >>> > >>> Thanks a lot > >>> + Harit > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > > "Genius might be described as a supreme capacity for getting its possessors > into trouble of all kinds." > -- Samuel Butler > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Tue Jan 13 18:01:45 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Tue, 13 Jan 2015 12:01:45 -0500 Subject: [erlang-questions] Windows compiled Jiffy library Message-ID: Anyone have a version of Jiffy that they have compiled for windows 7 64 bit and / or Windows 8 64 bit? Alternatively, good instructions on how to do it... much appreciated... -------------- next part -------------- An HTML attachment was scrubbed... URL: From andra.dinu@REDACTED Tue Jan 13 18:23:34 2015 From: andra.dinu@REDACTED (Andra Dinu) Date: Tue, 13 Jan 2015 18:23:34 +0100 (CET) Subject: [erlang-questions] Elang Factory SF Bay Area- Most speakers up, Early Bird on Message-ID: <1902289416.2904304.1421169814044.JavaMail.zimbra@erlang-solutions.com> Hi all, The Erlang Factory SF Bay area will take place on 26-27 March, with training courses on 23-25 March and 28 March-1 April. Early Bird Registration is now open and most speakers are up. Inventor of Smalltalk and Turing Award Winner Alan Kay, Elixir Creator Jos? Valim, Erlang inventors Joe Armstrong, Robert Virding and Mike Williams will all be there! Other speakers include Haskell and QuickCheck co-inventor John Hughes, 'Seven Languages in Seven weeks' author Bruce Tate, O'Reilly Author and CS professor Simon Thompson and Andreas Olofsson, creator of the Parallella board. Most speakers are up, take a look here: http://www.erlang-factory.com/sfbay2015/#speakers This year Training Sessions will be 23-25 March and 30 March-1 April: Erlan Express, OTP Express, Kazoo, Advanced Erlang Techniques, QuickCheck http://www.erlang-factory.com/sfbay2015/#home Cheers, Andra -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From harit.subscriptions@REDACTED Tue Jan 13 19:55:54 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 10:55:54 -0800 Subject: [erlang-questions] Kill a process after T seconds Message-ID: The recent problem I worked on is Write a function my_spawn(Mod, Func, Args, Time) that behaves like spawn(Mod, Func, Args) but with one difference. If the spawned process lives for more than Time seconds, it should be killed. My attempt looks like my_spawn(Mod, Func, Args, Time) -> Pid = spawn(Mod, Func, Args), io:format("created new process: ~p.~n", [Pid]), timer:kill_after(Time * 1000), Pid. and when I run I see 1> c(error_handling). {ok,error_handling} 2> Pid1 = error_handling:my_spawn(area_server0, loop, [], 30). created new process: <0.39.0>. <0.39.0> 3> Pid1 ! {self(), {square, 10}}. {<0.32.0>,{square,10}} 4> receive X->X end. 100 5> Pid1 ! {self(), {square, 20}}. {<0.32.0>,{square,20}} 6> receive Y->Y end. 400 ** exception error: killed 7> Looking for suggestions on the code or ways to write it better Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jan 13 20:02:08 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 13 Jan 2015 19:02:08 +0000 Subject: [erlang-questions] Kill a process after T seconds References: Message-ID: Hi Harit, I don't think there is any way you could do that better, without knowing what semantics you are striving for in the first place. The code looks fine to me, and depending in which context it runs, it could be perfect or have a problem. It is really hard to tell without any more information. For a larger system, you may gain some by using either the 'lager' application (github.com/basho/lager) or the 'error_logger' to report progress. That way, you can later route logging messages in a favorable fashion. Another thing you could look into is to use 'proc_lib' to turn the framework you have built into an OTP-compliant framework. But that may be a bit out on the horizon, depending on how much time you have spent with Erlang to this date. On Tue Jan 13 2015 at 7:56:03 PM Harit Himanshu < harit.subscriptions@REDACTED> wrote: > The recent problem I worked on is > > Write a function my_spawn(Mod, Func, Args, Time) that behaves like > spawn(Mod, Func, Args) but with one difference. If the spawned process > lives for more than Time seconds, it should be killed. > > My attempt looks like > > my_spawn(Mod, Func, Args, Time) -> > Pid = spawn(Mod, Func, Args), > io:format("created new process: ~p.~n", [Pid]), > timer:kill_after(Time * 1000), > Pid. > > and when I run I see > > 1> c(error_handling). > > {ok,error_handling} > > 2> Pid1 = error_handling:my_spawn(area_server0, loop, [], 30). > > created new process: <0.39.0>. > > <0.39.0> > > 3> Pid1 ! {self(), {square, 10}}. > > {<0.32.0>,{square,10}} > > 4> receive X->X end. > > 100 > > 5> Pid1 ! {self(), {square, 20}}. > > {<0.32.0>,{square,20}} > > 6> receive Y->Y end. > > 400 > > ** exception error: killed > > 7> > > Looking for suggestions on the code or ways to write it better > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Tue Jan 13 20:22:51 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 11:22:51 -0800 Subject: [erlang-questions] Kill a process after T seconds In-Reply-To: References: Message-ID: Thanks Jesper, this is my 13th Day with Erlang (Its new year resolution for me to learn Erlang this year and make at least 1 project on concurrency :)) I will take a look at lager btw Thank you again On Tue, Jan 13, 2015 at 11:02 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi Harit, > > I don't think there is any way you could do that better, without knowing > what semantics you are striving for in the first place. The code looks fine > to me, and depending in which context it runs, it could be perfect or have > a problem. It is really hard to tell without any more information. > > For a larger system, you may gain some by using either the 'lager' > application (github.com/basho/lager) or the 'error_logger' to report > progress. That way, you can later route logging messages in a favorable > fashion. > > Another thing you could look into is to use 'proc_lib' to turn the > framework you have built into an OTP-compliant framework. But that may be a > bit out on the horizon, depending on how much time you have spent with > Erlang to this date. > > On Tue Jan 13 2015 at 7:56:03 PM Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> The recent problem I worked on is >> >> Write a function my_spawn(Mod, Func, Args, Time) that behaves like >> spawn(Mod, Func, Args) but with one difference. If the spawned process >> lives for more than Time seconds, it should be killed. >> >> My attempt looks like >> >> my_spawn(Mod, Func, Args, Time) -> >> Pid = spawn(Mod, Func, Args), >> io:format("created new process: ~p.~n", [Pid]), >> timer:kill_after(Time * 1000), >> Pid. >> >> and when I run I see >> >> 1> c(error_handling). >> >> {ok,error_handling} >> >> 2> Pid1 = error_handling:my_spawn(area_server0, loop, [], 30). >> >> created new process: <0.39.0>. >> >> <0.39.0> >> >> 3> Pid1 ! {self(), {square, 10}}. >> >> {<0.32.0>,{square,10}} >> >> 4> receive X->X end. >> >> 100 >> >> 5> Pid1 ! {self(), {square, 20}}. >> >> {<0.32.0>,{square,20}} >> >> 6> receive Y->Y end. >> >> 400 >> >> ** exception error: killed >> >> 7> >> >> Looking for suggestions on the code or ways to write it better >> >> Thanks >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Tue Jan 13 21:01:22 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 13 Jan 2015 21:01:22 +0100 Subject: [erlang-questions] erlang-questions Digest, Vol 200, Issue 4 In-Reply-To: References: Message-ID: .."trying to create many processes, with timers all having very close together stop times leads to very noticeable drift. The closer they are together, the worse it is." Would this be a fair summary: - timer works for pausing for at least a while; - when exact interval is critical, another solution may be necessary; - there are technical limits to timer, much like to the vm. The fact that both timer and vm can take a lot of wear does not mean that things should be piled up until vm breaks. Spreading the load may help. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jan 13 21:14:54 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 13 Jan 2015 21:14:54 +0100 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: On Tue, Jan 13, 2015 at 6:01 PM, jim rosenblum wrote: > Anyone have a version of Jiffy that they have compiled for windows 7 > 64 bit and / or Windows 8 64 bit? Alternatively, good instructions > on how to do it... I don't have access to a Windows env, but from what I can see in the jiffy repo, it should build out of the box on Windows via rebar. That is, if the C code is not incompatible with msvc. If it is, and there's a requirement for gcc or clang, it's possible to modify rebar.config accordingly. So, assuming you're in a command prompt with the msvc toolchain in %PATH%, have you tried 'rebar compile'? From paul.joseph.davis@REDACTED Tue Jan 13 21:26:21 2015 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 13 Jan 2015 14:26:21 -0600 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: I don't know of any prebuilt packages off the top of my head. Dave Cottlehuber did most of the Windows support but I'm not sure how recently he's worked on Windows build stuff. On Tue, Jan 13, 2015 at 2:14 PM, Tuncer Ayaz wrote: > On Tue, Jan 13, 2015 at 6:01 PM, jim rosenblum wrote: >> Anyone have a version of Jiffy that they have compiled for windows 7 >> 64 bit and / or Windows 8 64 bit? Alternatively, good instructions >> on how to do it... > > I don't have access to a Windows env, but from what I can see in the > jiffy repo, it should build out of the box on Windows via rebar. That > is, if the C code is not incompatible with msvc. If it is, and there's > a requirement for gcc or clang, it's possible to modify rebar.config > accordingly. > > So, assuming you're in a command prompt with the msvc toolchain in > %PATH%, have you tried 'rebar compile'? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jim.rosenblum@REDACTED Tue Jan 13 22:32:54 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Tue, 13 Jan 2015 16:32:54 -0500 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: Doing a rebar compile results in an error -- complaining that it cannot find the c include files - the path to those include files is part of my Windows path and I can see no obvious way to add the /I to the argument string On Tue, Jan 13, 2015 at 3:26 PM, Paul Davis wrote: > I don't know of any prebuilt packages off the top of my head. Dave > Cottlehuber did most of the Windows support but I'm not sure how > recently he's worked on Windows build stuff. > > > On Tue, Jan 13, 2015 at 2:14 PM, Tuncer Ayaz > wrote: > > On Tue, Jan 13, 2015 at 6:01 PM, jim rosenblum wrote: > >> Anyone have a version of Jiffy that they have compiled for windows 7 > >> 64 bit and / or Windows 8 64 bit? Alternatively, good instructions > >> on how to do it... > > > > I don't have access to a Windows env, but from what I can see in the > > jiffy repo, it should build out of the box on Windows via rebar. That > > is, if the C code is not incompatible with msvc. If it is, and there's > > a requirement for gcc or clang, it's possible to modify rebar.config > > accordingly. > > > > So, assuming you're in a command prompt with the msvc toolchain in > > %PATH%, have you tried 'rebar compile'? > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Tue Jan 13 22:56:45 2015 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 13 Jan 2015 15:56:45 -0600 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: Not entirely sure which include files you mean, but if you need to affect the compiler invocation you can change this line in rebar.config: https://github.com/davisp/jiffy/blob/master/rebar.config#L25 On Tue, Jan 13, 2015 at 3:32 PM, jim rosenblum wrote: > Doing a rebar compile results in an error -- complaining that it cannot find > the c include files - the path to those include files is part of my Windows > path and I can see no obvious way to add the /I to the argument string > > On Tue, Jan 13, 2015 at 3:26 PM, Paul Davis > wrote: >> >> I don't know of any prebuilt packages off the top of my head. Dave >> Cottlehuber did most of the Windows support but I'm not sure how >> recently he's worked on Windows build stuff. >> >> >> On Tue, Jan 13, 2015 at 2:14 PM, Tuncer Ayaz >> wrote: >> > On Tue, Jan 13, 2015 at 6:01 PM, jim rosenblum wrote: >> >> Anyone have a version of Jiffy that they have compiled for windows 7 >> >> 64 bit and / or Windows 8 64 bit? Alternatively, good instructions >> >> on how to do it... >> > >> > I don't have access to a Windows env, but from what I can see in the >> > jiffy repo, it should build out of the box on Windows via rebar. That >> > is, if the C code is not incompatible with msvc. If it is, and there's >> > a requirement for gcc or clang, it's possible to modify rebar.config >> > accordingly. >> > >> > So, assuming you're in a command prompt with the msvc toolchain in >> > %PATH%, have you tried 'rebar compile'? >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > > From lostcolony@REDACTED Tue Jan 13 23:00:44 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Tue, 13 Jan 2015 17:00:44 -0500 Subject: [erlang-questions] erlang-questions Digest, Vol 200, Issue 4 In-Reply-To: References: Message-ID: Heh, I see I forgot to change the title when replying. Whoops. Sort of. I'd flow it like this - if fixed interval -> apply_interval(); varied interval andalso not multiple timers firing nearly at the same time -> timer:sleep(), spawn(fun() -> process() end), timer:sleep(); %One timer at a time _Otherwise -> send_after() %Scales better when multiple timers are in use, may still drift end Basically, assume that one timer is extremely exact, but that two timers set to wake up at the same time will cause them to drift. Assume also that the amount of drift tapers off with the amount of distance in between the two timers' wake time (interrupts and lock contention? I don't know the reason). Optimize and architect accordingly; in testing with a few thousand send_after calls, set to go off in 100 group batches, the drift I saw was still < 40ms (probably far less, but it was related to video, so my tolerance was 'frame accurate'). Again, this is based on my own experiences, my hardware, YMMV, etc. On Tue, Jan 13, 2015 at 3:01 PM, Imants Cekusins wrote: > .."trying to create many processes, with timers all having very close > together stop times leads to very noticeable drift. The closer they are > together, the worse it is." > > Would this be a fair summary: > > - timer works for pausing for at least a while; > > - when exact interval is critical, another solution may be necessary; > > - there are technical limits to timer, much like to the vm. > > The fact that both timer and vm can take a lot of wear does not mean that > things should be piled up until vm breaks. Spreading the load may help. > > ? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Tue Jan 13 23:03:14 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Tue, 13 Jan 2015 17:03:14 -0500 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: I was able to use the INCLUDE environment variable to point to the include directories - C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include - and that got me past this complaint. Now it is complaining that it canno open LIBCMT.lib. I have both the path and the INCLUDE variables pointing to the correct path to where that lib lives, but that doesn't seem to work. I tried changing the line that you indicated, but it did not seem to do anything On Tue, Jan 13, 2015 at 4:56 PM, Paul Davis wrote: > Not entirely sure which include files you mean, but if you need to > affect the compiler invocation you can change this line in > rebar.config: > > https://github.com/davisp/jiffy/blob/master/rebar.config#L25 > > On Tue, Jan 13, 2015 at 3:32 PM, jim rosenblum > wrote: > > Doing a rebar compile results in an error -- complaining that it cannot > find > > the c include files - the path to those include files is part of my > Windows > > path and I can see no obvious way to add the /I to the argument string > > > > On Tue, Jan 13, 2015 at 3:26 PM, Paul Davis > > > wrote: > >> > >> I don't know of any prebuilt packages off the top of my head. Dave > >> Cottlehuber did most of the Windows support but I'm not sure how > >> recently he's worked on Windows build stuff. > >> > >> > >> On Tue, Jan 13, 2015 at 2:14 PM, Tuncer Ayaz > >> wrote: > >> > On Tue, Jan 13, 2015 at 6:01 PM, jim rosenblum wrote: > >> >> Anyone have a version of Jiffy that they have compiled for windows 7 > >> >> 64 bit and / or Windows 8 64 bit? Alternatively, good instructions > >> >> on how to do it... > >> > > >> > I don't have access to a Windows env, but from what I can see in the > >> > jiffy repo, it should build out of the box on Windows via rebar. That > >> > is, if the C code is not incompatible with msvc. If it is, and there's > >> > a requirement for gcc or clang, it's possible to modify rebar.config > >> > accordingly. > >> > > >> > So, assuming you're in a command prompt with the msvc toolchain in > >> > %PATH%, have you tried 'rebar compile'? > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jan 13 23:17:12 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 13 Jan 2015 23:17:12 +0100 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: On Tue, Jan 13, 2015 at 11:03 PM, jim rosenblum wrote: > I was able to use the INCLUDE environment variable to point to the > include directories - C:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\include - and that got me past this complaint. Now it is > complaining that it canno open LIBCMT.lib. I have both the path and > the INCLUDE variables pointing to the correct path to where that lib > lives, but that doesn't seem to work. I tried changing the line that > you indicated, but it did not seem to do anything I can only speculate, but are you sure you've started a command prompt that initially executes the Visual Studio env batch file? There should be a shortcut to start that in Visual Studio's start menu folder. From harit.subscriptions@REDACTED Tue Jan 13 23:23:51 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 14:23:51 -0800 Subject: [erlang-questions] Can not kill the registered process Message-ID: The problem I am trying while learning from exercises from Joe's book is Write a function that creates a registered process that writes out "I?m still running" every five seconds. Write a function that monitors this process and restarts it if it dies. Start the global process and the monitor process. Kill the global process and check that it has been restarted by the monitor process. So I created the runner and monitor_runner as runner() -> receive after 5 * 1000 -> io:format("I am running.~n"), runner() end. monitor_runner() -> spawn(fun() -> {Pid, Ref} = spawn_monitor(error_handling, runner, []), io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), register(runner, Pid), receive {'DOWN', Ref, process, Pid, _Why} -> io:format("runner got killed, restarting"), monitor_runner(); X -> X end end). When I run this, I get 1> c(error_handling). {ok,error_handling} 2> Pid = error_handling:monitor_runner(). started new process <0.40.0> and reference #Ref<0.0.0.77>. <0.39.0> I am running. 3> 3> I am running. 3> Pid. <0.39.0> I am running. I am running. I am running. I am running. I am running. 4> whereis(runner). <0.40.0> I am running. 6> exit(whereis(runner)). ** exception exit: <0.40.0> I am running. I am running. I am running. I am running. I am running. I am running. I am running. 7> *Problem?* The runner is still running. 1.) Why is the process not getting killed? 2.) Why is it not caught by {'DOWN'....} What am I missing here? Thanks + Harit -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Tue Jan 13 23:31:23 2015 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 13 Jan 2015 14:31:23 -0800 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: References: Message-ID: exit/1 doesn't do what you think it does. Maybe try exit/2? On Tue, Jan 13, 2015 at 2:23 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > The problem I am trying while learning from exercises from Joe's book is > > Write a function that creates a registered process that writes out "I?m > still running" every five seconds. Write a function that monitors this > process and restarts it if it dies. Start the global process and the > monitor process. Kill the global process and check that it has been > restarted by the monitor process. > > So I created the runner and monitor_runner as > > runner() -> > receive > after 5 * 1000 -> > io:format("I am running.~n"), > runner() > end. > > monitor_runner() -> > spawn(fun() -> > {Pid, Ref} = spawn_monitor(error_handling, runner, []), > io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), > register(runner, Pid), > receive > {'DOWN', Ref, process, Pid, _Why} -> > io:format("runner got killed, restarting"), > monitor_runner(); > X -> X > end > end). > > > When I run this, I get > > 1> c(error_handling). > > {ok,error_handling} > > 2> Pid = error_handling:monitor_runner(). > > started new process <0.40.0> and reference #Ref<0.0.0.77>. > > <0.39.0> > > I am running. > > 3> > > 3> > > I am running. > > 3> Pid. > > <0.39.0> > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > 4> whereis(runner). > > <0.40.0> > > I am running. > > 6> exit(whereis(runner)). > > ** exception exit: <0.40.0> > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > 7> > > *Problem?* > The runner is still running. > 1.) Why is the process not getting killed? > 2.) Why is it not caught by {'DOWN'....} > > What am I missing here? > > Thanks > + Harit > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Tue Jan 13 23:33:39 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 14:33:39 -0800 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: References: Message-ID: you are right Felix, exit/1 did not help. Instead I tried timer:kill_after() and this seems to work as expected. I made no changes to code, only changed how to kill the registered process and it started to work 1> c(error_handling). {ok,error_handling} 2> error_handling:monitor_runner(). started new process <0.40.0> and reference #Ref<0.0.0.77>. <0.39.0> I am running. I am running. I am running. I am running. 3> timer:kill_after(3000, whereis(runner)). {ok,{1421188290386972,#Ref<0.0.0.88>}} I am running. runner got killed, restarting. started new process <0.45.0> and reference #Ref<0.0.0.95>. I am running. I am running. I am running. 4> timer:kill_after(3000, whereis(runner)). {ok,{1421188310002128,#Ref<0.0.0.101>}} runner got killed, restarting. started new process <0.48.0> and reference #Ref<0.0.0.107>. I am running. I am running. 5> Thank you + Harit Himanshu On Tue, Jan 13, 2015 at 2:31 PM, Felix Gallo wrote: > exit/1 doesn't do what you think it does. Maybe try exit/2? > > On Tue, Jan 13, 2015 at 2:23 PM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> The problem I am trying while learning from exercises from Joe's book is >> >> Write a function that creates a registered process that writes out "I?m >> still running" every five seconds. Write a function that monitors this >> process and restarts it if it dies. Start the global process and the >> monitor process. Kill the global process and check that it has been >> restarted by the monitor process. >> >> So I created the runner and monitor_runner as >> >> runner() -> >> receive >> after 5 * 1000 -> >> io:format("I am running.~n"), >> runner() >> end. >> >> monitor_runner() -> >> spawn(fun() -> >> {Pid, Ref} = spawn_monitor(error_handling, runner, []), >> io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), >> register(runner, Pid), >> receive >> {'DOWN', Ref, process, Pid, _Why} -> >> io:format("runner got killed, restarting"), >> monitor_runner(); >> X -> X >> end >> end). >> >> >> When I run this, I get >> >> 1> c(error_handling). >> >> {ok,error_handling} >> >> 2> Pid = error_handling:monitor_runner(). >> >> started new process <0.40.0> and reference #Ref<0.0.0.77>. >> >> <0.39.0> >> >> I am running. >> >> 3> >> >> 3> >> >> I am running. >> >> 3> Pid. >> >> <0.39.0> >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> 4> whereis(runner). >> >> <0.40.0> >> >> I am running. >> >> 6> exit(whereis(runner)). >> >> ** exception exit: <0.40.0> >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> I am running. >> >> 7> >> >> *Problem?* >> The runner is still running. >> 1.) Why is the process not getting killed? >> 2.) Why is it not caught by {'DOWN'....} >> >> What am I missing here? >> >> Thanks >> + Harit >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Tue Jan 13 23:38:32 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 14:38:32 -0800 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: References: Message-ID: and as per your suggestion Felix, I tried exit/2 as well, which seems to work pretty well 1> c(error_handling). {ok,error_handling} 2> error_handling:monitor_runner(). started new process <0.40.0> and reference #Ref<0.0.0.77>. <0.39.0> I am running. I am running. 3> exit(whereis(runner), kill). runner got killed, restarting. started new process <0.43.0> and reference #Ref<0.0.0.87>. true I am running. I am running. 4> exit(whereis(runner), kill). runner got killed, restarting. started new process <0.46.0> and reference #Ref<0.0.0.96>. true I am running. I am running. I am running. I am running. I am running. I am running. I am running. I am running. I am running. 5> I am running. I am running. 5> exit(whereis(runner), "kill you runner"). runner got killed, restarting. started new process <0.49.0> and reference #Ref<0.0.0.114>. true I am running. I am running. Thank you for your help + Harit Himanshu On Tue, Jan 13, 2015 at 2:33 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > you are right Felix, exit/1 did not help. > > Instead I tried timer:kill_after() and this seems to work as expected. I > made no changes to code, only changed how to kill the registered process > and it started to work > > > 1> c(error_handling). > > {ok,error_handling} > > 2> error_handling:monitor_runner(). > > started new process <0.40.0> and reference #Ref<0.0.0.77>. > > <0.39.0> > > I am running. > > I am running. > > I am running. > > I am running. > > 3> timer:kill_after(3000, whereis(runner)). > > {ok,{1421188290386972,#Ref<0.0.0.88>}} > > I am running. > > runner got killed, restarting. > > started new process <0.45.0> and reference #Ref<0.0.0.95>. > > I am running. > > I am running. > > I am running. > > 4> timer:kill_after(3000, whereis(runner)). > > {ok,{1421188310002128,#Ref<0.0.0.101>}} > > runner got killed, restarting. > > started new process <0.48.0> and reference #Ref<0.0.0.107>. > > I am running. > > I am running. > > 5> > > Thank you > + Harit Himanshu > > On Tue, Jan 13, 2015 at 2:31 PM, Felix Gallo wrote: > >> exit/1 doesn't do what you think it does. Maybe try exit/2? >> >> On Tue, Jan 13, 2015 at 2:23 PM, Harit Himanshu < >> harit.subscriptions@REDACTED> wrote: >> >>> The problem I am trying while learning from exercises from Joe's book is >>> >>> Write a function that creates a registered process that writes out "I?m >>> still running" every five seconds. Write a function that monitors this >>> process and restarts it if it dies. Start the global process and the >>> monitor process. Kill the global process and check that it has been >>> restarted by the monitor process. >>> >>> So I created the runner and monitor_runner as >>> >>> runner() -> >>> receive >>> after 5 * 1000 -> >>> io:format("I am running.~n"), >>> runner() >>> end. >>> >>> monitor_runner() -> >>> spawn(fun() -> >>> {Pid, Ref} = spawn_monitor(error_handling, runner, []), >>> io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), >>> register(runner, Pid), >>> receive >>> {'DOWN', Ref, process, Pid, _Why} -> >>> io:format("runner got killed, restarting"), >>> monitor_runner(); >>> X -> X >>> end >>> end). >>> >>> >>> When I run this, I get >>> >>> 1> c(error_handling). >>> >>> {ok,error_handling} >>> >>> 2> Pid = error_handling:monitor_runner(). >>> >>> started new process <0.40.0> and reference #Ref<0.0.0.77>. >>> >>> <0.39.0> >>> >>> I am running. >>> >>> 3> >>> >>> 3> >>> >>> I am running. >>> >>> 3> Pid. >>> >>> <0.39.0> >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> 4> whereis(runner). >>> >>> <0.40.0> >>> >>> I am running. >>> >>> 6> exit(whereis(runner)). >>> >>> ** exception exit: <0.40.0> >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> I am running. >>> >>> 7> >>> >>> *Problem?* >>> The runner is still running. >>> 1.) Why is the process not getting killed? >>> 2.) Why is it not caught by {'DOWN'....} >>> >>> What am I missing here? >>> >>> Thanks >>> + Harit >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Wed Jan 14 00:17:06 2015 From: rpettit@REDACTED (Rick Pettit) Date: Tue, 13 Jan 2015 17:17:06 -0600 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: References: Message-ID: <55B67441-8A17-4F29-BE69-E76D5F315DA1@vailsys.com> I think you want to replace your call to exit/1 with an appropriate call to exit/2. The way you have it written your shell is committing suicide (as opposed to murder :-) -Rick On Jan 13, 2015, at 4:23 PM, Harit Himanshu wrote: > The problem I am trying while learning from exercises from Joe's book is > > Write a function that creates a registered process that writes out "I?m still running" every five seconds. Write a function that monitors this process and restarts it if it dies. Start the global process and the monitor process. Kill the global process and check that it has been restarted by the monitor process. > > So I created the runner and monitor_runner as > > runner() -> > receive > after 5 * 1000 -> > io:format("I am running.~n"), > runner() > end. > > monitor_runner() -> > spawn(fun() -> > {Pid, Ref} = spawn_monitor(error_handling, runner, []), > io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), > register(runner, Pid), > receive > {'DOWN', Ref, process, Pid, _Why} -> > io:format("runner got killed, restarting"), > monitor_runner(); > X -> X > end > end). > > > When I run this, I get > 1> c(error_handling). > {ok,error_handling} > 2> Pid = error_handling:monitor_runner(). > started new process <0.40.0> and reference #Ref<0.0.0.77>. > <0.39.0> > I am running. > 3> > 3> > I am running. > 3> Pid. > <0.39.0> > I am running. > I am running. > I am running. > I am running. > I am running. > 4> whereis(runner). > <0.40.0> > I am running. > 6> exit(whereis(runner)). > ** exception exit: <0.40.0> > I am running. > I am running. > I am running. > I am running. > I am running. > I am running. > I am running. > 7> > > Problem? > The runner is still running. > 1.) Why is the process not getting killed? > 2.) Why is it not caught by {'DOWN'....} > > What am I missing here? > > Thanks > + Harit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From harit.subscriptions@REDACTED Wed Jan 14 00:24:09 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 15:24:09 -0800 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: <55B67441-8A17-4F29-BE69-E76D5F315DA1@vailsys.com> References: <55B67441-8A17-4F29-BE69-E76D5F315DA1@vailsys.com> Message-ID: ha ha, good analogy, thanks Rick! + Harit On Tue, Jan 13, 2015 at 3:17 PM, Rick Pettit wrote: > I think you want to replace your call to exit/1 with an appropriate call > to exit/2. > > The way you have it written your shell is committing suicide (as opposed > to murder :-) > > -Rick > > On Jan 13, 2015, at 4:23 PM, Harit Himanshu > wrote: > > > The problem I am trying while learning from exercises from Joe's book is > > > > Write a function that creates a registered process that writes out "I?m > still running" every five seconds. Write a function that monitors this > process and restarts it if it dies. Start the global process and the > monitor process. Kill the global process and check that it has been > restarted by the monitor process. > > > > So I created the runner and monitor_runner as > > > > runner() -> > > receive > > after 5 * 1000 -> > > io:format("I am running.~n"), > > runner() > > end. > > > > monitor_runner() -> > > spawn(fun() -> > > {Pid, Ref} = spawn_monitor(error_handling, runner, []), > > io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), > > register(runner, Pid), > > receive > > {'DOWN', Ref, process, Pid, _Why} -> > > io:format("runner got killed, restarting"), > > monitor_runner(); > > X -> X > > end > > end). > > > > > > When I run this, I get > > 1> c(error_handling). > > {ok,error_handling} > > 2> Pid = error_handling:monitor_runner(). > > started new process <0.40.0> and reference #Ref<0.0.0.77>. > > <0.39.0> > > I am running. > > 3> > > 3> > > I am running. > > 3> Pid. > > <0.39.0> > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > 4> whereis(runner). > > <0.40.0> > > I am running. > > 6> exit(whereis(runner)). > > ** exception exit: <0.40.0> > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > I am running. > > 7> > > > > Problem? > > The runner is still running. > > 1.) Why is the process not getting killed? > > 2.) Why is it not caught by {'DOWN'....} > > > > What am I missing here? > > > > Thanks > > + Harit > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Wed Jan 14 00:25:34 2015 From: rpettit@REDACTED (Rick Pettit) Date: Tue, 13 Jan 2015 17:25:34 -0600 Subject: [erlang-questions] Can not kill the registered process In-Reply-To: <55B67441-8A17-4F29-BE69-E76D5F315DA1@vailsys.com> References: <55B67441-8A17-4F29-BE69-E76D5F315DA1@vailsys.com> Message-ID: <60178540-380F-46A8-AAE0-F9A20F16170E@vailsys.com> Doh! I didn?t realize I had my wifi disabled when I replied to your original email (and so didn?t see the replies others had already sent you indicating the problem was your calling of exit/1 vs. exit/2). Sorry for the redundant reply. -Rick On Jan 13, 2015, at 5:17 PM, Rick Pettit wrote: > I think you want to replace your call to exit/1 with an appropriate call to exit/2. > > The way you have it written your shell is committing suicide (as opposed to murder :-) > > -Rick > > On Jan 13, 2015, at 4:23 PM, Harit Himanshu wrote: > >> The problem I am trying while learning from exercises from Joe's book is >> >> Write a function that creates a registered process that writes out "I?m still running" every five seconds. Write a function that monitors this process and restarts it if it dies. Start the global process and the monitor process. Kill the global process and check that it has been restarted by the monitor process. >> >> So I created the runner and monitor_runner as >> >> runner() -> >> receive >> after 5 * 1000 -> >> io:format("I am running.~n"), >> runner() >> end. >> >> monitor_runner() -> >> spawn(fun() -> >> {Pid, Ref} = spawn_monitor(error_handling, runner, []), >> io:format("started new process ~p and reference ~p.~n", [Pid, Ref]), >> register(runner, Pid), >> receive >> {'DOWN', Ref, process, Pid, _Why} -> >> io:format("runner got killed, restarting"), >> monitor_runner(); >> X -> X >> end >> end). >> >> >> When I run this, I get >> 1> c(error_handling). >> {ok,error_handling} >> 2> Pid = error_handling:monitor_runner(). >> started new process <0.40.0> and reference #Ref<0.0.0.77>. >> <0.39.0> >> I am running. >> 3> >> 3> >> I am running. >> 3> Pid. >> <0.39.0> >> I am running. >> I am running. >> I am running. >> I am running. >> I am running. >> 4> whereis(runner). >> <0.40.0> >> I am running. >> 6> exit(whereis(runner)). >> ** exception exit: <0.40.0> >> I am running. >> I am running. >> I am running. >> I am running. >> I am running. >> I am running. >> I am running. >> 7> >> >> Problem? >> The runner is still running. >> 1.) Why is the process not getting killed? >> 2.) Why is it not caught by {'DOWN'....} >> >> What am I missing here? >> >> Thanks >> + Harit >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From harit.subscriptions@REDACTED Wed Jan 14 04:52:36 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 13 Jan 2015 19:52:36 -0800 Subject: [erlang-questions] How to identify the process which died and restart it? Message-ID: I was working on the following problem and wrote the code that works, but I have conceptual questions which are still bothering me. So I thought to put them in community and seek recommendations and guidance *Question* Write a function that starts and monitors several worker processes. If any of the worker processes dies abnormally, restart it. My attempt looks like monitor_workers(F, NumberOfWorkers) -> spawn(error_handling, monitor_workers_loop, [F, NumberOfWorkers]). monitor_workers_loop(F, NumberOfWorkers) -> Workers = [spawn_monitor(F) || _Worker <- lists:seq(1, NumberOfWorkers)], io:format("started workers: ~p.~n", [Workers]), receive {'DOWN', _, process, _, _} -> io:format("one of workers killed, starting one.~n"), monitor_workers_loop(F, 1) end. and when I run, I see 1> c(error_handling). {ok,error_handling} 2> 2> error_handling:monitor_workers(fun()-> area_server0:loop() end, 2). started workers: [{<0.40.0>,#Ref<0.0.0.81>},{<0.41.0>,#Ref<0.0.0.82>}]. <0.39.0> 3> 3> Worker1 = list_to_pid("<0.40.0>"). <0.40.0> 4> 4> Worker2 = list_to_pid("<0.41.0>"). <0.41.0> 5> 5> exit(Worker1, kill). one of workers killed, starting one. started workers: [{<0.45.0>,#Ref<0.0.0.98>}]. true 6> exit(Worker2, kill). one of workers killed, starting one. started workers: [{<0.47.0>,#Ref<0.0.0.105>}]. true 7> *Problem(s)?* 1. A better way to solve this would be pass in different functions(than same, what I did) and let workers work on different problems. In such a scenario all we know if Pid and Ref to the workers. 2. When any process fails, it matches {'DOWN', Ref, process, Pid, Why}, and given this context how could I identify what function this process was running? I am very new to Erlang so have not much knowledge to solve such problems. Thank you + Harit Himanshu -------------- next part -------------- An HTML attachment was scrubbed... URL: From eng.carlos.suarez@REDACTED Wed Jan 14 06:06:08 2015 From: eng.carlos.suarez@REDACTED (Carlos Suarez Fontalvo) Date: Wed, 14 Jan 2015 00:06:08 -0500 Subject: [erlang-questions] Lists Comprehension issue? Message-ID: Hi. I've reading this documentation: http://www.erlang.org/doc/programming_examples/list_comprehensions.html And it says that 3.5 Simplifications with List Comprehensions As an example, list comprehensions can be used to simplify some of the functions in lists.erl: append(L) -> [X || L1 <- L, X <- L1]. map(Fun, L) -> [Fun(X) || X <- L]. filter(Pred, L) -> [X || X <- L, Pred(X)]. However when I tried to test this first example "append" doing something simple like this: 73> [X || L1 <- 20, X <- L1]. ** exception error: bad generator 20 I got a "exception error" message. Any idea why?. Am I not considering something? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Wed Jan 14 06:50:58 2015 From: bob@REDACTED (Bob Ippolito) Date: Wed, 14 Jan 2015 16:50:58 +1100 Subject: [erlang-questions] Lists Comprehension issue? In-Reply-To: References: Message-ID: 20 is not a list of lists. On Wednesday, January 14, 2015, Carlos Suarez Fontalvo < eng.carlos.suarez@REDACTED> wrote: > Hi. I've reading this documentation: > http://www.erlang.org/doc/programming_examples/list_comprehensions.html > > And it says that > > 3.5 Simplifications with List Comprehensions > > As an example, list comprehensions can be used to simplify some of the > functions in lists.erl: > > append(L) -> [X || L1 <- L, X <- L1]. > map(Fun, L) -> [Fun(X) || X <- L]. > filter(Pred, L) -> [X || X <- L, Pred(X)]. > > However when I tried to test this first example "append" doing something simple like this: > > 73> [X || L1 <- 20, X <- L1]. > ** exception error: bad generator 20 > > I got a "exception error" message. Any idea why?. Am I not considering something? > > Thanks in advance > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xiande.duan@REDACTED Wed Jan 14 07:08:51 2015 From: xiande.duan@REDACTED (Duan, Xiande) Date: Wed, 14 Jan 2015 06:08:51 +0000 Subject: [erlang-questions] Lists Comprehension issue? In-Reply-To: References: Message-ID: <5A0DB9EC001ECD4DA29B9F40F5A35F401B0A2D@SHSMSX103.ccr.corp.intel.com> The ?L? must be a list of list. Like: 1> L = [[1,2,3],[a,b,c]]. [[1,2,3],[a,b,c]] 2> [X || L1 <- L, X <- L1]. [1,2,3,a,b,c] 3> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Carlos Suarez Fontalvo Sent: Wednesday, January 14, 2015 1:06 PM To: erlang-questions@REDACTED Subject: [erlang-questions] Lists Comprehension issue? Hi. I've reading this documentation: http://www.erlang.org/doc/programming_examples/list_comprehensions.html And it says that 3.5 Simplifications with List Comprehensions As an example, list comprehensions can be used to simplify some of the functions in lists.erl: append(L) -> [X || L1 <- L, X <- L1]. map(Fun, L) -> [Fun(X) || X <- L]. filter(Pred, L) -> [X || X <- L, Pred(X)]. However when I tried to test this first example "append" doing something simple like this: 73> [X || L1 <- 20, X <- L1]. ** exception error: bad generator 20 I got a "exception error" message. Any idea why?. Am I not considering something? Thanks in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvidal@REDACTED Wed Jan 14 07:25:39 2015 From: gvidal@REDACTED (German Vidal) Date: Wed, 14 Jan 2015 10:25:39 +0400 Subject: [erlang-questions] Lists Comprehension issue? In-Reply-To: References: Message-ID: <0D134F9E-A22E-4499-B552-F5243DBD9A7C@dsic.upv.es> Hi, L should be a list of lists, e.g., append([[1,2],[3,4]]) will return [1,2,3,4]. g-- On 14 Jan 2015, at 09:06, Carlos Suarez Fontalvo wrote: > Hi. I've reading this documentation: http://www.erlang.org/doc/programming_examples/list_comprehensions.html > > And it says that > > 3.5 Simplifications with List Comprehensions > > As an example, list comprehensions can be used to simplify some of the functions in lists.erl: > > append(L) -> [X || L1 <- L, X <- L1]. > map(Fun, L) -> [Fun(X) || X <- L]. > filter(Pred, L) -> [X || X <- L, Pred(X)]. > > > However when I tried to test this first example "append" doing something simple like this: > > 73> [X || L1 <- 20, X <- L1]. > ** exception error: bad generator 20 > > I got a "exception error" message. Any idea why?. Am I not considering something? > > Thanks in advance > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto@REDACTED Wed Jan 14 10:02:00 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 10:02:00 +0100 Subject: [erlang-questions] How to identify the process which died and restart it? In-Reply-To: References: Message-ID: Is there any reason why you're not considering a supervisor [1]? Best, r. [1] http://erlang.org/doc/man/supervisor.html On Wed, Jan 14, 2015 at 4:52 AM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > I was working on the following problem and wrote the code that works, but > I have conceptual questions which are still bothering me. So I thought to > put them in community and seek recommendations and guidance > > *Question* > Write a function that starts and monitors several worker processes. If any > of the worker processes dies abnormally, restart it. > > My attempt looks like > monitor_workers(F, NumberOfWorkers) -> > spawn(error_handling, monitor_workers_loop, [F, NumberOfWorkers]). > > > monitor_workers_loop(F, NumberOfWorkers) -> > Workers = [spawn_monitor(F) || _Worker <- lists:seq(1, NumberOfWorkers)], > io:format("started workers: ~p.~n", [Workers]), > receive > {'DOWN', _, process, _, _} -> > io:format("one of workers killed, starting one.~n"), > monitor_workers_loop(F, 1) > end. > > and when I run, I see > > 1> c(error_handling). > {ok,error_handling} > 2> > 2> error_handling:monitor_workers(fun()-> area_server0:loop() end, 2). > started workers: [{<0.40.0>,#Ref<0.0.0.81>},{<0.41.0>,#Ref<0.0.0.82>}]. > <0.39.0> > 3> > 3> Worker1 = list_to_pid("<0.40.0>"). > <0.40.0> > 4> > 4> Worker2 = list_to_pid("<0.41.0>"). > <0.41.0> > 5> > 5> exit(Worker1, kill). > one of workers killed, starting one. > started workers: [{<0.45.0>,#Ref<0.0.0.98>}]. > true > 6> exit(Worker2, kill). > one of workers killed, starting one. > started workers: [{<0.47.0>,#Ref<0.0.0.105>}]. > true > 7> > > *Problem(s)?* > > 1. A better way to solve this would be pass in different > functions(than same, what I did) and let workers work on different > problems. In such a scenario all we know if Pid and Ref to the workers. > 2. When any process fails, it matches {'DOWN', Ref, process, Pid, > Why}, and given this context how could I identify what function this > process was running? > > I am very new to Erlang so have not much knowledge to solve such problems. > > Thank you > + Harit Himanshu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.vershilov@REDACTED Wed Jan 14 11:57:15 2015 From: alexander.vershilov@REDACTED (Alexander V Vershilov) Date: Wed, 14 Jan 2015 14:57:15 +0400 Subject: [erlang-questions] Fwd: Question about registry semantics for the remote processes. In-Reply-To: References: Message-ID: Hello. I want to understand the reasons for the current semantics for saving remote processes in local registry. According to the specification for register function [1] it's impossible to save remote process in local registry: Failure: badarg if PidOrPort is not an existing, local process or port, if RegName is already in use, if the process or port is already registered (already has a name), or if RegName is the atom undefined. As a result it's not possible to construct expression Name ! Message, where Name is evaluated to a atom stored in registry that points to the remote process. Another approach is described in unified semantics paper (Section 2.3) However, for uniformity, in this semantics names can be registered for remote processes (i.e., register(name,pid) does not fail if pid is a remote process), and registering a local process at a remote node is supported too (using the operation register(node,name,pid)). As a consequence, when a message is sent to a remote node using the syntax {atom,node}!msg there is no guarantee that the process that should receive the mes- sage is located at node; thus it may be necessary to relay the message to a process on yet another node So I have two questions: 1. Were there any technical reasons for forbidding remote pids in local registry as we have in current erlang, or it's a historical reasons? 2. If there are technical reasons for that, then why uniformed specification introduces such behaviour, or it provides some mechanisms to solve those problems? One can argue that the reason is monitoring, and case when we are sending to the remote process from registry that already dead, but I don't see any difference with sending to remote Pid that already died, semantics of all the functions and guarantees are the same. Thanks [1] http://www.erlang.org/doc/man/erlang.html#register-2 [2] http://happy-testing.com/hans/papers/EW2010-UnifiedSemantics.pdf -- Alexander From harit.subscriptions@REDACTED Wed Jan 14 14:52:41 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Wed, 14 Jan 2015 05:52:41 -0800 Subject: [erlang-questions] How to identify the process which died and restart it? In-Reply-To: References: Message-ID: Yes, because I am not on the chapter which teaches about it yet. My understanding is that the same semantics could be reaching without using it as well. Having said that, I do believe supervisor would be the best solution, but this problem has to be used without Supervisor. I guess the exercise is given to understand the semantics of Supervisor works Thank you + Harit Himanshu On Wed, Jan 14, 2015 at 1:02 AM, Roberto Ostinelli wrote: > Is there any reason why you're not considering a supervisor [1]? > > Best, > r. > > [1] http://erlang.org/doc/man/supervisor.html > > On Wed, Jan 14, 2015 at 4:52 AM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> I was working on the following problem and wrote the code that works, but >> I have conceptual questions which are still bothering me. So I thought to >> put them in community and seek recommendations and guidance >> >> *Question* >> Write a function that starts and monitors several worker processes. If >> any of the worker processes dies abnormally, restart it. >> >> My attempt looks like >> monitor_workers(F, NumberOfWorkers) -> >> spawn(error_handling, monitor_workers_loop, [F, NumberOfWorkers]). >> >> >> monitor_workers_loop(F, NumberOfWorkers) -> >> Workers = [spawn_monitor(F) || _Worker <- lists:seq(1, >> NumberOfWorkers)], >> io:format("started workers: ~p.~n", [Workers]), >> receive >> {'DOWN', _, process, _, _} -> >> io:format("one of workers killed, starting one.~n"), >> monitor_workers_loop(F, 1) >> end. >> >> and when I run, I see >> >> 1> c(error_handling). >> {ok,error_handling} >> 2> >> 2> error_handling:monitor_workers(fun()-> area_server0:loop() end, 2). >> started workers: [{<0.40.0>,#Ref<0.0.0.81>},{<0.41.0>,#Ref<0.0.0.82>}]. >> <0.39.0> >> 3> >> 3> Worker1 = list_to_pid("<0.40.0>"). >> <0.40.0> >> 4> >> 4> Worker2 = list_to_pid("<0.41.0>"). >> <0.41.0> >> 5> >> 5> exit(Worker1, kill). >> one of workers killed, starting one. >> started workers: [{<0.45.0>,#Ref<0.0.0.98>}]. >> true >> 6> exit(Worker2, kill). >> one of workers killed, starting one. >> started workers: [{<0.47.0>,#Ref<0.0.0.105>}]. >> true >> 7> >> >> *Problem(s)?* >> >> 1. A better way to solve this would be pass in different >> functions(than same, what I did) and let workers work on different >> problems. In such a scenario all we know if Pid and Ref to the workers. >> 2. When any process fails, it matches {'DOWN', Ref, process, Pid, >> Why}, and given this context how could I identify what function this >> process was running? >> >> I am very new to Erlang so have not much knowledge to solve such problems. >> >> Thank you >> + Harit Himanshu >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.rosenblum@REDACTED Wed Jan 14 14:54:06 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Wed, 14 Jan 2015 08:54:06 -0500 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: I reinstalled visual Studio 2013 and now the rebar comand executed to completion , but I have a ton of warnings and a message about unknown options (-g and -03) and the result doesn't pass the eunit tests due to the NIF not being loadable. I assume that I am using the wrong tool chain, but this is over my head (I am not a Windows or C/C++ guy). Thanks for you help, but I think I'm close to giving up on this. On Tue, Jan 13, 2015 at 5:17 PM, Tuncer Ayaz wrote: > On Tue, Jan 13, 2015 at 11:03 PM, jim rosenblum wrote: > > I was able to use the INCLUDE environment variable to point to the > > include directories - C:\Program Files (x86)\Microsoft Visual Studio > > 12.0\VC\include - and that got me past this complaint. Now it is > > complaining that it canno open LIBCMT.lib. I have both the path and > > the INCLUDE variables pointing to the correct path to where that lib > > lives, but that doesn't seem to work. I tried changing the line that > > you indicated, but it did not seem to do anything > > I can only speculate, but are you sure you've started a command prompt > that initially executes the Visual Studio env batch file? There should be > a shortcut to start that in Visual Studio's start menu folder. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Jan 14 15:01:45 2015 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 14 Jan 2015 15:01:45 +0100 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: On Wed, Jan 14, 2015 at 2:54 PM, jim rosenblum wrote: > I reinstalled visual Studio 2013 and now the rebar comand executed to > completion , but I have a ton of warnings and a message about unknown > options (-g and -03) and the result doesn't pass the eunit tests due to the > NIF not being loadable. I assume that I am using the wrong tool chain, but > this is over my head (I am not a Windows or C/C++ guy). > > Thanks for you help, but I think I'm close to giving up on this. If this is something you are creating yourself (rather than a requirement for your dependency), you might want to consider a pure Erlang JSON parser, like jsx[1]/jsxn[2]. If it is the only C program in your chain, it might save/solve you a bunch of troubles. Not exactly an answer to your question, but exactly this advise helped someone before. Motiejus [1]: https://github.com/talentdeficit/jsx [2]: https://github.com/talentdeficit/jsxn From xiut.xu@REDACTED Wed Jan 14 15:04:09 2015 From: xiut.xu@REDACTED (xu xiut) Date: Wed, 14 Jan 2015 09:04:09 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: https://github.com/LMAX-Exchange/disruptor/wiki/Introduction Does the disruptor still sound like Riak? On Tue, Jan 13, 2015 at 10:36 AM, T Ty wrote: > Your comment is quite interesting. My take-away from reading the LMAX > article wasn't the single threaded business logic process nor the shared > memory communication. > > My take-aways were: > > 1. pipelining. Do all input validation and transformation before/after > main business logic processing. This makes coding the 'happy-path' easier > and keeps the business logic core small. > > 2. the business logic process is a master process that reacts to events > and generate events. This allows scaling of the business logic process as > it makes it easier to spawn processes to deal with the events. > (gen_event/gen_fsm triggering supervisor in simple_one_for_one that starts > other processes) > > 3. Event Sourcing. One added benefit is compliance. It is easier to > demonstrate compliance when there are events to show how the system changed > from one state to another. Compliance here can mean security compliance or > regulatory compliance. > > 4. business logic processor deals with events in-memory. mnesia ram_tables > fit this criteria easily as do ets tables. > > 5. Business Logic State Snapshot. Not only does this allow resilience > between node restarts it also allows migrating the partial process to a > different node entirely. > > 6. Avoiding external service calls in the Business Logic process. > Transition to an event based model for interacting with external services. > Have a middle layer of gen_fsm to determine where in the logic the current > process is in. > > 7. Mention about session data being transient and can be discarded. Aka > let process die and supervisor restart. > > The Disruptor sounds like Riak. > > Cheers > > On Tue, Jan 13, 2015 at 2:11 PM, Mihai Balea wrote: > >> >> > On Jan 12, 2015, at 4:10 AM, T Ty wrote: >> > >> > The LMAX architecture is an easy fit for Erlang and one which an >> Erlanger would naturally arrive to. >> >> I beg to differ. LMAX uses a single threaded process to implement its >> entire business logic, which makes it highly sensitive to sequential >> performance. This is not one of Erlang?s strong points. Also, the >> disruptors - which are essentially fancy circular buffers - depend on >> preallocated memory and destructive updates to attain their performance. >> Finally, the communication between various stages in the pipeline is done >> through shared memory, and they do some clever tricks to avoid locking. >> >> Yes, you could emulate this architecture in Erlang, but you won?t get >> anywhere near LMAX?s performance. >> >> Mihai >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From william@REDACTED Wed Jan 14 12:45:53 2015 From: william@REDACTED (William B) Date: Wed, 14 Jan 2015 22:15:53 +1030 Subject: [erlang-questions] Eldap and start_tls issues Message-ID: <20150114221553.26a07fbb@franky.ipa.blackhats.net.au> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, When I attempt to use the start_tls function in my code as below I am recieving the error: **error:{case_clause,{error,ssl_not_started}} The cert is the correct CA cert, and I have verified that TLS and the cert work with ldap search. The Hosts list is also correct. When connected to ldap without start_tls, the connection works and can be quiered. How can I proceed debugging this, or am I just missing parameters needed by start_tls? Any advice is appreciated. ldap_starttls(Conn) -> case eldap:start_tls(Conn, [ {cacertfile, "/etc/pki/tls/certs/ca.crt"}, {verify, verify_none} ]) of ok -> Conn end. ldap_open(Hosts) -> case eldap:open(Hosts, [{port, 389}]) of {ok, Conn} -> ldap_starttls(Conn); {error, Reason} -> io:format("Error ~p", [Reason]); {_, _} -> io:format("Error UNKNOWN", []) end. - -- Sincerely, William Brown -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJUtlbxAAoJEO/EFteBqAmaP7AP/2iMmqZrimEISWUaY6sZz7nx plYY40ITJ/ec2mFVe5uoWrR/lRySfhKGbtkV6i4cFk/Pvs0prO1P5xCtqGl4dT09 0XJEaPfM4eiqCaEfiCteqwibgPtLMURMiHFS5FGUyyOPf+frsdJMk7kiNuencn2L 7XKZ9tfO12qSgBAvAtdpPrTkw3U+UYj2qnsVC/oYoUH1TnS6UYHnNUUXLrZrcFqw 2qyiXUzVVwl1nlUHPE/4oKCVQp4fMi32fqlO0VRIm3OjMTIUl7GRNtS4/k+g9rdO 7xk9AduNdVSMgF9GAfKNJm+49OAyngiuDv/JLTjscpcmzACPAHHE/RUMGK9skks2 3MwtYzyEDOCqCmrB/gunkex6CCz24NphB2PoxTedmA8xeEiemINhy+27Br8gFQ8p GVnCfxQML3vh34O7sLamtX4t1hPJWbVAeiP6xEI9uw7ufMAOAj/TB5C2qLfihQ0o OV3wXNISyba33ffaaUBa6/tgVysAuDVWbq+T4kj0K4q6+bLE4cxBePkNk45uEX16 /GxTgyFx7cTOMybk25JDry9Lxj7zSjaOmAKQaNFv7vWoeranen+t1zcXaAZolFTl g9zBKhGYXQS2LklMb6exvKoG7lKfRdUaRwZ+K2oxhbKxOUisKbSs4xM5dpioGwTL FiA4DAtkGHwFr4vhfqzY =poEY -----END PGP SIGNATURE----- From eng.carlos.suarez@REDACTED Wed Jan 14 13:50:25 2015 From: eng.carlos.suarez@REDACTED (Carlos Suarez Fontalvo) Date: Wed, 14 Jan 2015 07:50:25 -0500 Subject: [erlang-questions] Lists Comprehension issue? In-Reply-To: References: Message-ID: Thank you. On Wednesday, January 14, 2015, Bob Ippolito wrote: > 20 is not a list of lists. > > On Wednesday, January 14, 2015, Carlos Suarez Fontalvo < > eng.carlos.suarez@REDACTED > > wrote: > >> Hi. I've reading this documentation: >> http://www.erlang.org/doc/programming_examples/list_comprehensions.html >> >> And it says that >> >> 3.5 Simplifications with List Comprehensions >> >> As an example, list comprehensions can be used to simplify some of the >> functions in lists.erl: >> >> append(L) -> [X || L1 <- L, X <- L1]. >> map(Fun, L) -> [Fun(X) || X <- L]. >> filter(Pred, L) -> [X || X <- L, Pred(X)]. >> >> However when I tried to test this first example "append" doing something simple like this: >> >> 73> [X || L1 <- 20, X <- L1]. >> ** exception error: bad generator 20 >> >> I got a "exception error" message. Any idea why?. Am I not considering something? >> >> Thanks in advance >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Wed Jan 14 15:32:18 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 14 Jan 2015 15:32:18 +0100 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: On Wed, Jan 14, 2015 at 2:54 PM, jim rosenblum wrote: > I reinstalled visual Studio 2013 and now the rebar comand executed > to completion , but I have a ton of warnings and a message about > unknown options (-g and -03) and the result doesn't pass the eunit > tests due to the NIF not being loadable. I assume that I am using > the wrong tool chain, but this is over my head (I am not a Windows > or C/C++ guy). Looking at jiffy's rebar.config, it seems like this is caused by lines 10 and 11, which seem to be safe to omit, so delete the lines and try again. Or you could replace ".*" with the regex from line 13. I'll send you a modified rebar.config. > Thanks for you help, but I think I'm close to giving up on this. Don't give up, this is the technology we have to deal with, and it's usually because people decided to disregard/forget good solutions from the past in favor of inferior reinventions. > On Tue, Jan 13, 2015 at 5:17 PM, Tuncer Ayaz wrote: > > > > On Tue, Jan 13, 2015 at 11:03 PM, jim rosenblum wrote: > > > I was able to use the INCLUDE environment variable to point to > > > the include directories - C:\Program Files (x86)\Microsoft > > > Visual Studio 12.0\VC\include - and that got me past this > > > complaint. Now it is complaining that it canno open LIBCMT.lib. > > > I have both the path and the INCLUDE variables pointing to the > > > correct path to where that lib lives, but that doesn't seem to > > > work. I tried changing the line that you indicated, but it did > > > not seem to do anything > > > > I can only speculate, but are you sure you've started a command > > prompt that initially executes the Visual Studio env batch file? > > There should be a shortcut to start that in Visual Studio's start > > menu folder. From a.shneyderman@REDACTED Wed Jan 14 15:51:07 2015 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Wed, 14 Jan 2015 09:51:07 -0500 Subject: [erlang-questions] Eldap and start_tls issues In-Reply-To: <20150114221553.26a07fbb@franky.ipa.blackhats.net.au> References: <20150114221553.26a07fbb@franky.ipa.blackhats.net.au> Message-ID: you need to start ssl application. Run this application:ensure_all_started(ssl) before eldap calls. On Wed, Jan 14, 2015 at 6:45 AM, William B wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > When I attempt to use the start_tls function in my code as below I am > recieving the error: > > **error:{case_clause,{error,ssl_not_started}} > > The cert is the correct CA cert, and I have verified that TLS and the > cert work with ldap search. The Hosts list is also correct. When > connected to ldap without start_tls, the connection works and can be > quiered. > > How can I proceed debugging this, or am I just missing parameters > needed by start_tls? > > Any advice is appreciated. > > > ldap_starttls(Conn) -> > case eldap:start_tls(Conn, [ > {cacertfile, "/etc/pki/tls/certs/ca.crt"}, > {verify, verify_none} > ]) of > ok -> Conn > end. > > ldap_open(Hosts) -> > case eldap:open(Hosts, [{port, 389}]) of > {ok, Conn} -> ldap_starttls(Conn); > {error, Reason} -> io:format("Error ~p", [Reason]); > {_, _} -> io:format("Error UNKNOWN", []) > end. > > - -- > Sincerely, > > William Brown > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iQIcBAEBAgAGBQJUtlbxAAoJEO/EFteBqAmaP7AP/2iMmqZrimEISWUaY6sZz7nx > plYY40ITJ/ec2mFVe5uoWrR/lRySfhKGbtkV6i4cFk/Pvs0prO1P5xCtqGl4dT09 > 0XJEaPfM4eiqCaEfiCteqwibgPtLMURMiHFS5FGUyyOPf+frsdJMk7kiNuencn2L > 7XKZ9tfO12qSgBAvAtdpPrTkw3U+UYj2qnsVC/oYoUH1TnS6UYHnNUUXLrZrcFqw > 2qyiXUzVVwl1nlUHPE/4oKCVQp4fMi32fqlO0VRIm3OjMTIUl7GRNtS4/k+g9rdO > 7xk9AduNdVSMgF9GAfKNJm+49OAyngiuDv/JLTjscpcmzACPAHHE/RUMGK9skks2 > 3MwtYzyEDOCqCmrB/gunkex6CCz24NphB2PoxTedmA8xeEiemINhy+27Br8gFQ8p > GVnCfxQML3vh34O7sLamtX4t1hPJWbVAeiP6xEI9uw7ufMAOAj/TB5C2qLfihQ0o > OV3wXNISyba33ffaaUBa6/tgVysAuDVWbq+T4kj0K4q6+bLE4cxBePkNk45uEX16 > /GxTgyFx7cTOMybk25JDry9Lxj7zSjaOmAKQaNFv7vWoeranen+t1zcXaAZolFTl > g9zBKhGYXQS2LklMb6exvKoG7lKfRdUaRwZ+K2oxhbKxOUisKbSs4xM5dpioGwTL > FiA4DAtkGHwFr4vhfqzY > =poEY > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Wed Jan 14 15:52:57 2015 From: imantc@REDACTED (Imants Cekusins) Date: Wed, 14 Jan 2015 15:52:57 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: Here is v1: https://github.com/aminishiki/kvas Works with simple code. Try, anyone? From rpettit@REDACTED Wed Jan 14 16:12:13 2015 From: rpettit@REDACTED (Rick Pettit) Date: Wed, 14 Jan 2015 09:12:13 -0600 Subject: [erlang-questions] How to identify the process which died and restart it? In-Reply-To: References: Message-ID: <2EB82E0A-FDC7-4BED-9378-04F5C4738C4E@vailsys.com> Harit, Sorry for going off topic here, but in case someone hasn?t already suggested it when you are done with your current reading you might want to check out Fred Hebert?s book _Learn You Some Erlang For Great Good_. Not to take anything away from your current read by any means, but I?ve been using Erlang for years and have developed a number of successful production systems with it and am now working on teaching others within the company by way of going through LYSEFGG with them. I am finding it to be an excellent read, one I *really* wish was available back when I first started learning. Check it out if you haven?t already, you won?t be disappointed. -Rick On Jan 14, 2015, at 7:52 AM, Harit Himanshu wrote: > Yes, because I am not on the chapter which teaches about it yet. My understanding is that the same semantics could be reaching without using it as well. > > Having said that, I do believe supervisor would be the best solution, but this problem has to be used without Supervisor. I guess the exercise is given to understand the semantics of Supervisor works > > Thank you > + Harit Himanshu > > On Wed, Jan 14, 2015 at 1:02 AM, Roberto Ostinelli wrote: > Is there any reason why you're not considering a supervisor [1]? > > Best, > r. > > [1] http://erlang.org/doc/man/supervisor.html > > On Wed, Jan 14, 2015 at 4:52 AM, Harit Himanshu wrote: > I was working on the following problem and wrote the code that works, but I have conceptual questions which are still bothering me. So I thought to put them in community and seek recommendations and guidance > > Question > Write a function that starts and monitors several worker processes. If any of the worker processes dies abnormally, restart it. > > My attempt looks like > monitor_workers(F, NumberOfWorkers) -> > spawn(error_handling, monitor_workers_loop, [F, NumberOfWorkers]). > > > monitor_workers_loop(F, NumberOfWorkers) -> > Workers = [spawn_monitor(F) || _Worker <- lists:seq(1, NumberOfWorkers)], > io:format("started workers: ~p.~n", [Workers]), > receive > {'DOWN', _, process, _, _} -> > io:format("one of workers killed, starting one.~n"), > monitor_workers_loop(F, 1) > end. > > and when I run, I see > > 1> c(error_handling). > {ok,error_handling} > 2> > 2> error_handling:monitor_workers(fun()-> area_server0:loop() end, 2). > started workers: [{<0.40.0>,#Ref<0.0.0.81>},{<0.41.0>,#Ref<0.0.0.82>}]. > <0.39.0> > 3> > 3> Worker1 = list_to_pid("<0.40.0>"). > <0.40.0> > 4> > 4> Worker2 = list_to_pid("<0.41.0>"). > <0.41.0> > 5> > 5> exit(Worker1, kill). > one of workers killed, starting one. > started workers: [{<0.45.0>,#Ref<0.0.0.98>}]. > true > 6> exit(Worker2, kill). > one of workers killed, starting one. > started workers: [{<0.47.0>,#Ref<0.0.0.105>}]. > true > 7> > > Problem(s)? > ? A better way to solve this would be pass in different functions(than same, what I did) and let workers work on different problems. In such a scenario all we know if Pid and Ref to the workers. > ? When any process fails, it matches {'DOWN', Ref, process, Pid, Why}, and given this context how could I identify what function this process was running? > I am very new to Erlang so have not much knowledge to solve such problems. > > Thank you > + Harit Himanshu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tuncer.ayaz@REDACTED Wed Jan 14 16:12:52 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 14 Jan 2015 16:12:52 +0100 Subject: [erlang-questions] Windows compiled Jiffy library In-Reply-To: References: Message-ID: On Wed, Jan 14, 2015 at 3:32 PM, Tuncer Ayaz wrote: > On Wed, Jan 14, 2015 at 2:54 PM, jim rosenblum wrote: > > I reinstalled visual Studio 2013 and now the rebar comand executed > > to completion , but I have a ton of warnings and a message about > > unknown options (-g and -03) and the result doesn't pass the eunit > > tests due to the NIF not being loadable. I assume that I am using > > the wrong tool chain, but this is over my head (I am not a Windows > > or C/C++ guy). > > Looking at jiffy's rebar.config, it seems like this is caused by > lines 10 and 11, which seem to be safe to omit, so delete the lines > and try again. Or you could replace ".*" with the regex from line > 13. > > I'll send you a modified rebar.config. rebar.config fix: https://github.com/davisp/jiffy/pull/78 jim's build log: https://github.com/davisp/jiffy/issues/79 From xiut.xu@REDACTED Wed Jan 14 16:38:33 2015 From: xiut.xu@REDACTED (xu xiut) Date: Wed, 14 Jan 2015 10:38:33 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: T Ty, Sean Cribbs thinks Riak and Disruptor have very little in common: http://riak-users.197444.n3.nabble.com/Disruptor-like-behaviour-td4032394.html On Wed, Jan 14, 2015 at 9:04 AM, xu xiut wrote: > https://github.com/LMAX-Exchange/disruptor/wiki/Introduction > > Does the disruptor still sound like Riak? > > On Tue, Jan 13, 2015 at 10:36 AM, T Ty wrote: > >> Your comment is quite interesting. My take-away from reading the LMAX >> article wasn't the single threaded business logic process nor the shared >> memory communication. >> >> My take-aways were: >> >> 1. pipelining. Do all input validation and transformation before/after >> main business logic processing. This makes coding the 'happy-path' easier >> and keeps the business logic core small. >> >> 2. the business logic process is a master process that reacts to events >> and generate events. This allows scaling of the business logic process as >> it makes it easier to spawn processes to deal with the events. >> (gen_event/gen_fsm triggering supervisor in simple_one_for_one that starts >> other processes) >> >> 3. Event Sourcing. One added benefit is compliance. It is easier to >> demonstrate compliance when there are events to show how the system changed >> from one state to another. Compliance here can mean security compliance or >> regulatory compliance. >> >> 4. business logic processor deals with events in-memory. mnesia >> ram_tables fit this criteria easily as do ets tables. >> >> 5. Business Logic State Snapshot. Not only does this allow resilience >> between node restarts it also allows migrating the partial process to a >> different node entirely. >> >> 6. Avoiding external service calls in the Business Logic process. >> Transition to an event based model for interacting with external services. >> Have a middle layer of gen_fsm to determine where in the logic the current >> process is in. >> >> 7. Mention about session data being transient and can be discarded. Aka >> let process die and supervisor restart. >> >> The Disruptor sounds like Riak. >> >> Cheers >> >> On Tue, Jan 13, 2015 at 2:11 PM, Mihai Balea wrote: >> >>> >>> > On Jan 12, 2015, at 4:10 AM, T Ty wrote: >>> > >>> > The LMAX architecture is an easy fit for Erlang and one which an >>> Erlanger would naturally arrive to. >>> >>> I beg to differ. LMAX uses a single threaded process to implement its >>> entire business logic, which makes it highly sensitive to sequential >>> performance. This is not one of Erlang?s strong points. Also, the >>> disruptors - which are essentially fancy circular buffers - depend on >>> preallocated memory and destructive updates to attain their performance. >>> Finally, the communication between various stages in the pipeline is done >>> through shared memory, and they do some clever tricks to avoid locking. >>> >>> Yes, you could emulate this architecture in Erlang, but you won?t get >>> anywhere near LMAX?s performance. >>> >>> Mihai >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Wed Jan 14 17:33:09 2015 From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=) Date: Wed, 14 Jan 2015 17:33:09 +0100 Subject: [erlang-questions] How to identify the process which died and restart it? In-Reply-To: References: Message-ID: You need to keep a table of either pid-> task (function or whatever) or mref-> task. There are several representation choices for the table. A {Key, Value} list would be the simplest. Some of the key functions in the lists module might be of help - or you could do the manipulations directly. /Erik Den 14/01/2015 04.52 skrev "Harit Himanshu" : > I was working on the following problem and wrote the code that works, but > I have conceptual questions which are still bothering me. So I thought to > put them in community and seek recommendations and guidance > > *Question* > Write a function that starts and monitors several worker processes. If any > of the worker processes dies abnormally, restart it. > > My attempt looks like > monitor_workers(F, NumberOfWorkers) -> > spawn(error_handling, monitor_workers_loop, [F, NumberOfWorkers]). > > > monitor_workers_loop(F, NumberOfWorkers) -> > Workers = [spawn_monitor(F) || _Worker <- lists:seq(1, NumberOfWorkers)], > io:format("started workers: ~p.~n", [Workers]), > receive > {'DOWN', _, process, _, _} -> > io:format("one of workers killed, starting one.~n"), > monitor_workers_loop(F, 1) > end. > > and when I run, I see > > 1> c(error_handling). > {ok,error_handling} > 2> > 2> error_handling:monitor_workers(fun()-> area_server0:loop() end, 2). > started workers: [{<0.40.0>,#Ref<0.0.0.81>},{<0.41.0>,#Ref<0.0.0.82>}]. > <0.39.0> > 3> > 3> Worker1 = list_to_pid("<0.40.0>"). > <0.40.0> > 4> > 4> Worker2 = list_to_pid("<0.41.0>"). > <0.41.0> > 5> > 5> exit(Worker1, kill). > one of workers killed, starting one. > started workers: [{<0.45.0>,#Ref<0.0.0.98>}]. > true > 6> exit(Worker2, kill). > one of workers killed, starting one. > started workers: [{<0.47.0>,#Ref<0.0.0.105>}]. > true > 7> > > *Problem(s)?* > > 1. A better way to solve this would be pass in different > functions(than same, what I did) and let workers work on different > problems. In such a scenario all we know if Pid and Ref to the workers. > 2. When any process fails, it matches {'DOWN', Ref, process, Pid, > Why}, and given this context how could I identify what function this > process was running? > > I am very new to Erlang so have not much knowledge to solve such problems. > > Thank you > + Harit Himanshu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Wed Jan 14 17:38:36 2015 From: mihai@REDACTED (Mihai Balea) Date: Wed, 14 Jan 2015 11:38:36 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> Message-ID: <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> > On Jan 14, 2015, at 10:38 AM, xu xiut wrote: > > T Ty, Sean Cribbs thinks Riak and Disruptor have very little in common: http://riak-users.197444.n3.nabble.com/Disruptor-like-behaviour-td4032394.html They do have very little in common? in fact pretty much the only thing they do have in common is that people usually use circles when discussing their respective architectures. Mihai -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Wed Jan 14 18:03:52 2015 From: egil@REDACTED (=?windows-1252?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 14 Jan 2015 18:03:52 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: <54B6A178.2040205@erlang.org> On 2015-01-13 14:57, Richard Carlsson wrote: > What really should be done is that the current interpreting debugger > should be destroyed with fire and a real bytecode-level debugger > should be implemented (with low level functionality done in C), which > would be way faster, only require updating when new Beam operations > are added, and not require the abstract code to be present in the beam > files. Tony Rogvall has written the beginnings of a bytecode interpreter. See, https://github.com/tonyrog/beam If someone wanted to build a bytecode interpreter I think that is good place to start. It does not handle all the opcodes, but a fair amount. // Bj?rn-Egil > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jan 14 18:05:32 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 18:05:32 +0100 Subject: [erlang-questions] Gun process gone Message-ID: Dear list, I'm using gun as a HTTP client. I've chosen gun because I want a HTTP library that does not retry to reconnect or to issue a call on error. I was previously using hackney, but I couldn't set it to not retry automatically. Si, in gun I set {retry, 0} in the Options of the gun:open/3 call. I'm issuing consecutive calls, but after 3 or 4 calls (randomly) the gun process exits, and all I get is: crashed with reason: gone in gun:retry_loop/2 line 449 I don't have any problems with the same server and another library. Has anyone seen this behavior with gun? Thanks, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Jan 14 19:17:28 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 14 Jan 2015 18:17:28 +0000 Subject: [erlang-questions] Gun process gone References: Message-ID: hrmmmm hackney doesn't retry or it's a bug. do you have any code i could use to reproduce and maybe fix? On Wed 14 Jan 2015 at 18:05 Roberto Ostinelli wrote: > Dear list, > I'm using gun as a HTTP client. I've chosen gun because I want a HTTP > library that does not retry to reconnect or to issue a call on error. I was > previously using hackney, but I couldn't set it to not retry automatically. > > Si, in gun I set {retry, 0} in the Options of the gun:open/3 call. > > I'm issuing consecutive calls, but after 3 or 4 calls (randomly) the gun > process exits, and all I get is: > > crashed with reason: gone in gun:retry_loop/2 line 449 > > I don't have any problems with the same server and another library. Has > anyone seen this behavior with gun? > > Thanks, > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Wed Jan 14 19:22:12 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 19:22:12 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: References: Message-ID: <2A4A9C51-612F-4582-8E6A-3D7E78EC8237@widetag.com> > On 14/gen/2015, at 19:17, Benoit Chesneau wrote: > > hrmmmm hackney doesn't retry or it's a bug. do you have any code i could use to reproduce and maybe fix? From roberto.ostinelli@REDACTED Wed Jan 14 19:23:55 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 19:23:55 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: References: Message-ID: Hi Benoit, Hackney is not automatically reconnecting upon disconnection? If not, I will need to investigate. > On 14/gen/2015, at 19:17, Benoit Chesneau wrote: > > hrmmmm hackney doesn't retry or it's a bug. do you have any code i could use to reproduce and maybe fix? From roberto.ostinelli@REDACTED Wed Jan 14 19:26:02 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 19:26:02 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: References: Message-ID: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> > 1. Are you sure the server is not just closing the connection? Then since you set retry to 0, gun doesn't try to reconnect. Have you checked last server response headers? I'm sure. I do not have this problem with other libraries. > 2. How consecutive are these calls? gun's default keepalive setting is 5 seconds. I've set the keepalive to 6000. > Also, what server are you using? What difference does this make? I have no control on the server. It's a standard API server. From unix1@REDACTED Wed Jan 14 18:52:13 2015 From: unix1@REDACTED (Unix One) Date: Wed, 14 Jan 2015 09:52:13 -0800 Subject: [erlang-questions] Gun process gone In-Reply-To: References: Message-ID: Hi Roberto, Some suggestions: 1. Are you sure the server is not just closing the connection? Then since you set retry to 0, gun doesn't try to reconnect. Have you checked last server response headers? 2. How consecutive are these calls? gun's default keepalive setting is 5 seconds. Also, what server are you using? Date: Wed, 14 Jan 2015 18:05:32 +0100 From: roberto@REDACTED To: erlang-questions@REDACTED Subject: [erlang-questions] Gun process gone Dear list,I'm using gun as a HTTP client. I've chosen gun because I want a HTTP library that does not retry to reconnect or to issue a call on error. I was previously using hackney, but I couldn't set it to not retry automatically. Si, in gun I set {retry, 0} in the Options of the gun:open/3 call. I'm issuing consecutive calls, but after 3 or 4 calls (randomly) the gun process exits, and all I get is: crashed with reason: gone in gun:retry_loop/2 line 449 I don't have any problems with the same server and another library. Has anyone seen this behavior with gun? Thanks,r. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jan 14 19:37:08 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 14 Jan 2015 19:37:08 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> References: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> Message-ID: <54B6B754.6080400@ninenines.eu> On 01/14/2015 07:26 PM, Roberto Ostinelli wrote: >> 2. How consecutive are these calls? gun's default keepalive setting is 5 seconds. > > I've set the keepalive to 6000. 6000 is 6 seconds, that's not much more. Maybe you meant 60000? Anyway it's very possible that Gun has a bug, it's still fairly early code and it's only tested against Cowboy at this point. 'gone' means the connection is gone and all retries have failed (in this case, all 0 of them). Would be interesting to trace the process and see what it's doing. -- Lo?c Hoguin http://ninenines.eu From tuncer.ayaz@REDACTED Wed Jan 14 21:34:07 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 14 Jan 2015 21:34:07 +0100 Subject: [erlang-questions] travis-ci erlang support broken? In-Reply-To: References: Message-ID: On Sun, Jan 4, 2015 at 12:38 PM, Tuncer Ayaz wrote: > On Fri, Jan 2, 2015 at 10:20 PM, Tuncer Ayaz wrote: > > On Fri, Jan 2, 2015 at 9:49 PM, Tuncer Ayaz wrote: > > > Does anybody know if the following travis-ci Erlang distro > > > breakage is due to some kind of recent compartmentalization > > > changes, and if so, what changes are required to continue using > > > travis-ci for Erlang? > > > > > > Can we use only short names for nodes now? > > > > > > curl > > > https://s3.amazonaws.com/archive.travis-ci.org/jobs/45606731/log.txt > > > | tail -n 45 > > > > The relevant bit: > > {error_logger,{{2015,1,1},{13,48,39}}, > > "Can't set long node name!\nPlease check your configuration\n",[]} > > > > Also, I should have tried -sname before asking the list: > > https://github.com/rebar/rebar/pull/421 > > After switching -name to -sname, the test runs into: > {error_logger,{{2015,1,2},{21,22,11}}, > \"Protocol: ~tp: the name ct_rt3@REDACTED seems to be in use by > another Erlang node\", [\"inet_tcp\"]} > > Clearly, it didn't fix the problem, and since the tests ran fine > previously, I'll have to assume something's changed in travis-ci's > Erlang support. They're out-of-office for the moment, so we'll > hopefully find out what changed once they're back. This has since been fixed by switching to Travis-CI's new container-based workers by adding sudo: false to .travis.yml. During the tests, it came to light that the host config wasn't correct enough for `erl -name ct_rt3` to work automatically. We could have resorted to using ct_rt@REDACTED, but that would be IPv4-specific and require us to remember it for any and all distributed node spun up in tests. Thanks to Hiro Asari from Travis-CI for suggesting the new workers, as that env works out of the box by returning a proper FQDN. From roberto.ostinelli@REDACTED Wed Jan 14 21:43:49 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 21:43:49 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: <54B6B754.6080400@ninenines.eu> References: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> <54B6B754.6080400@ninenines.eu> Message-ID: <075EBCB8-E565-4C23-A609-449A41515413@widetag.com> I indeed meant 60000. The question is why the connection is gone in the first place. :) Unfortunately the retry code eats up the error, as it exits with "gone". Thank you Lo?c, r. > On 14/gen/2015, at 19:37, Lo?c Hoguin wrote: > > On 01/14/2015 07:26 PM, Roberto Ostinelli wrote: >>> 2. How consecutive are these calls? gun's default keepalive setting is 5 seconds. >> >> I've set the keepalive to 6000. > > 6000 is 6 seconds, that's not much more. Maybe you meant 60000? > > Anyway it's very possible that Gun has a bug, it's still fairly early code and it's only tested against Cowboy at this point. 'gone' means the connection is gone and all retries have failed (in this case, all 0 of them). > > Would be interesting to trace the process and see what it's doing. > > -- > Lo?c Hoguin > http://ninenines.eu From essen@REDACTED Wed Jan 14 21:55:04 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 14 Jan 2015 21:55:04 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: <075EBCB8-E565-4C23-A609-449A41515413@widetag.com> References: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> <54B6B754.6080400@ninenines.eu> <075EBCB8-E565-4C23-A609-449A41515413@widetag.com> Message-ID: <54B6D7A8.9020005@ninenines.eu> Yep, but that's not a problem if you can just trace the process. :-) Need any help with that? On 01/14/2015 09:43 PM, Roberto Ostinelli wrote: > I indeed meant 60000. > > The question is why the connection is gone in the first place. :) Unfortunately the retry code eats up the error, as it exits with "gone". > > Thank you Lo?c, > r. > > >> On 14/gen/2015, at 19:37, Lo?c Hoguin wrote: >> >> On 01/14/2015 07:26 PM, Roberto Ostinelli wrote: >>>> 2. How consecutive are these calls? gun's default keepalive setting is 5 seconds. >>> >>> I've set the keepalive to 6000. >> >> 6000 is 6 seconds, that's not much more. Maybe you meant 60000? >> >> Anyway it's very possible that Gun has a bug, it's still fairly early code and it's only tested against Cowboy at this point. 'gone' means the connection is gone and all retries have failed (in this case, all 0 of them). >> >> Would be interesting to trace the process and see what it's doing. >> >> -- >> Lo?c Hoguin >> http://ninenines.eu -- Lo?c Hoguin http://ninenines.eu From roberto.ostinelli@REDACTED Wed Jan 14 22:40:52 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 14 Jan 2015 22:40:52 +0100 Subject: [erlang-questions] Gun process gone In-Reply-To: <54B6D7A8.9020005@ninenines.eu> References: <6B654004-E3AA-4AED-AEF1-8C0C18D438E6@widetag.com> <54B6B754.6080400@ninenines.eu> <075EBCB8-E565-4C23-A609-449A41515413@widetag.com> <54B6D7A8.9020005@ninenines.eu> Message-ID: <38AA1E0C-E1C1-4726-AACD-E885C730B9C1@widetag.com> > Yep, but that's not a problem if you can just trace the process. :-) > > Need any help with that? Not really :) But thanks for asking. Best, r. From william@REDACTED Thu Jan 15 00:27:29 2015 From: william@REDACTED (William) Date: Thu, 15 Jan 2015 09:57:29 +1030 Subject: [erlang-questions] Eldap and start_tls issues In-Reply-To: References: <20150114221553.26a07fbb@franky.ipa.blackhats.net.au> Message-ID: <68113291-FE78-4F95-8A1E-717F25F9AE86@firstyear.id.au> > On 15 Jan 2015, at 01:21, Alex Shneyderman wrote: > > you need to start ssl application. Run this > > application:ensure_all_started(ssl) > > before eldap calls. Fantastic! That fixed the issue immediately. Thanks for your fast assistance! From zxq9@REDACTED Thu Jan 15 09:51:58 2015 From: zxq9@REDACTED (zxq9) Date: Thu, 15 Jan 2015 17:51:58 +0900 Subject: [erlang-questions] How to identify the process which died and restart it? In-Reply-To: References: Message-ID: <27337654.WzfZP7oKfD@changa> On 2015?1?14? ??? 05:52:41 Harit Himanshu wrote: > Yes, because I am not on the chapter which teaches about it yet. My > understanding is that the same semantics could be reaching without using it > as well. > > Having said that, I do believe supervisor would be the best solution, but > this problem has to be used without Supervisor. I guess the exercise is > given to understand the semantics of Supervisor works I'm working on a (very rough, still very immature) intermediate text that starts with a raw, relatively featureless, non-OTP codebase -- which might illustrate what you want to know. First, understand the "genesis" function here: https://github.com/zxq9/erlmud/blob/master/erlmud-0.1/locman.erl#L33 It is using the list comprehension to pack a list of tuples like {Id, Pid}, where Id is the location's id within the game (usually a coordinate, but it could be any term) and the Pid belong's to the location process, which is linked. This managing process is trapping exits, so it will receive {'EXIT', Pid, Reason} messages when any one of the locations dies. Second, check the receive clause here: https://github.com/zxq9/erlmud/blob/master/erlmud-0.1/locman.erl#L44-L46 Note the receive clause above it, which matches on this processes' manager (these are really *much* smaller supervisor/worker pairs in the OTP version of this, so "manager" equates to a supervisor). If an exit message does not come from the parent, then we must check if it is from one of the processes in that Live list that genesis/1 packed during initialization. That calls the handle_exit/3 function here: https://github.com/zxq9/erlmud/blob/master/erlmud-0.1/locman.erl#L68-L80 First we check if the Pid is present in the list of processes we think are live. If it is, we know a game location just died, and needs to be restarted. We first scrub the list to rid ourselves of the now dead entry in the live list, then get the configuration data for that location ID from the list of location conf entries (which may have changed since the room was started last), restart the room, and add its {ID, Pid} tuple back to the list, and return to our loop. So the trick is all in how you interpret the exit message, and how you keep track of your linked or monitored processes. A very similar process happens in wayman, here: https://github.com/zxq9/erlmud/blob/master/erlmud-0.1/wayman.erl#L75-L85 But note that wayman is monitoring, not linking. This is because the locations are linked to their own ways, so the wayman is more of an ad hoc registry (which is completely unneccessary in the OTP version of this project) than a real supervisor. Because it maintains monitors, not links, it is receiving on any 'DOWN' message and checking its registry of live ways from there. Hopefully this sheds a bit of light on how this sort of thing can be handled manually. In 99% of cases you really will want to use supervisors and think through how you can best restore a stable, known state to things that die based on the facilities OTP provides than go to the effort of writing supervisory processes by hand all the time. The basic problem is that even if your hand-written supervisors are perfect all the time, they essentially wind up representing boilerplate that is often larger than the part of your code that actually solves user problems! Hand-written supervisors are also a tremendously tempting place to put "just one more feature" that supervisors shouldn't have to begin with -- thus increasing the chance your supervisors will themselves crash, potentially threatening your "crash kernel" (or at least bringing real disaster one step closer to it). Please forgive the incomplete, convoluted, and poorly explained nature of the code and the project -- I'm nowhere near done with it (if only I had the time!). -Craig From vladdu55@REDACTED Thu Jan 15 14:01:48 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 15 Jan 2015 14:01:48 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: Nice first step! Thanks! regards, Vlad On Wed, Jan 14, 2015 at 3:52 PM, Imants Cekusins wrote: > Here is v1: > > https://github.com/aminishiki/kvas > > > Works with simple code. Try, anyone? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.pedro.magalhaes@REDACTED Thu Jan 15 14:36:00 2015 From: jose.pedro.magalhaes@REDACTED (=?UTF-8?Q?Jos=C3=A9_Pedro_Magalh=C3=A3es?=) Date: Thu, 15 Jan 2015 13:36:00 +0000 Subject: [erlang-questions] Mathematics of Program Construction (MPC 2015): final call for papers Message-ID: Apologies for multiple copies. FINAL CALL FOR PAPERS 12th International Conference on Mathematics of Program Construction, MPC 2015 K?nigswinter, Germany, 29 June - 1 July 2015 http://www.cs.ox.ac.uk/conferences/MPC2015/ BACKGROUND The MPC conferences aim to promote the development of mathematical principles and techniques that are demonstrably practical and effective in the process of constructing computer programs, broadly interpreted. The 2015 MPC conference will be held in K?nigswinter, Germany, from 29th June to 1st July 2015. The previous conferences were held in Twente, The Netherlands (1989), Oxford, UK (1992), Kloster Irsee, Germany (1995), Marstrand, Sweden (1998), Ponte de Lima, Portugal (2000), Dagstuhl, Germany (2002), Stirling, UK (2004, colocated with AMAST), Kuressaare, Estonia (2006, colocated with AMAST), Marseille, France (2008), Qu?bec City, Canada (2010, colocated with AMAST), and Madrid, Spain (2012). TOPICS Papers are solicited on mathematical methods and tools put to use in program construction. Topics of interest range from algorithmics to support for program construction in programming languages and systems. The notion of "program" is broad, from algorithms to hardware. Some typical areas are type systems, program analysis and transformation, programming-language semantics, security, and program logics. Theoretical contributions are welcome, provided that their relevance to program construction is clear. Reports on applications are welcome, provided that their mathematical basis is evident. We also encourage the submission of "pearls": elegant, instructive, and fun essays on the mathematics of program construction. IMPORTANT DATES * Submission of abstracts: 26 January 2015 * Submission of full papers: 2 February 2015 * Notification to authors: 16 March 2015 * Final version: 13 April 2015 SUBMISSION Submission is in two stages. Abstracts (plain text, 10 to 20 lines) must be submitted by 26 January 2015. Full papers (pdf) adhering to the LaTeX llncs style must be submitted by 2 February 2015. There is no official page limit, but authors should strive for brevity. The web-based system EasyChair will be used for submission (https://easychair.org/conferences/?conf=mpc2015). Papers must report previously unpublished work, and must not be submitted concurrently to a journal or to another conference with refereed proceedings. Accepted papers must be presented at the conference by one of the authors. Please feel free to write to mpc2015@REDACTED with any questions about academic matters. The proceedings of MPC 2015 will be published in Springer-Verlag's Lecture Notes in Computer Science series, as have all the previous editions. Authors of accepted papers will be expected to transfer copyright to Springer for this purpose. After the conference, authors of the best papers will be invited to submit revised versions to a special issue of the Elsevier journal Science of Computer Programming. PROGRAMME COMMITTEE Ralf Hinze University of Oxford, UK (chair) Eerke Boiten University of Kent, UK Jules Desharnais Universit? Laval, Canada Lindsay Groves Victoria University of Wellington, New Zealand Zhenjiang Hu National Institute of Informatics, Japan Graham Hutton University of Nottingham, UK Johan Jeuring Utrecht University and Open University, The Netherlands Jay McCarthy Vassar College, US Larissa Meinicke The University of Queensland, Australia Bernhard M?ller Universit?t Augsburg, Germany Shin-Cheng Mu Academia Sinica, Taiwan Dave Naumann Stevens Institute of Technology, US Pablo Nogueira Universidad Polit?cnica de Madrid, Spain Ulf Norell University of Gothenburg, Sweden Bruno C. d. S. Oliveira The University of Hong Kong, Hong Kong Jos? Nuno Oliveira Universidade do Minho, Portugal Alberto Pardo Universidad de la Rep?blica, Uruguay Christine Paulin-Mohring INRIA-Universit? Paris-Sud, France Tom Schrijvers KU Leuven, Belgium Emil Sekerinski McMaster University, Canada Tim Sheard Portland State University, US Anya Tafliovich University of Toronto Scarborough, Canada Tarmo Uustalu Institute of Cybernetics, Estonia Janis Voigtl?nder Universit?t Bonn, Germany VENUE The conference will take place in K?nigswinter, Maritim Hotel, where accommodation has been reserved. K?nigswinter is situated on the right bank of the river Rhine, opposite Germany's former capital Bonn, at the foot of the Siebengebirge. LOCAL ORGANIZERS Ralf Hinze University of Oxford, UK (co-chair) Janis Voigtl?nder Universit?t Bonn, Germany (co-chair) Jos? Pedro Magalh?es University of Oxford, UK Nicolas Wu University of Oxford, UK For queries about local matters, please write to jv@REDACTED From garry@REDACTED Thu Jan 15 16:50:13 2015 From: garry@REDACTED (Garry Hodgson) Date: Thu, 15 Jan 2015 10:50:13 -0500 Subject: [erlang-questions] tls triple handshake vulnerabilty Message-ID: <54B7E1B5.3080509@research.att.com> Are the erlang ssl libraries subject to the TLS triple handshake vulnerability described at https://secure-resumption.com? If so, are there configuration options that can mitigate the risk? I've read through the erlang ssl docs, but don't understand the subject well enough to tell. From the.silly.sad@REDACTED Thu Jan 15 20:08:13 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Thu, 15 Jan 2015 20:08:13 +0100 Subject: [erlang-questions] What is allowed in guards? Message-ID: <54B8101D.2000907@gmail.com> Is there a complete list of operations allowed in guards? organized in a human readable way as a "reference-book". From felixgallo@REDACTED Thu Jan 15 20:11:01 2015 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 15 Jan 2015 11:11:01 -0800 Subject: [erlang-questions] What is allowed in guards? In-Reply-To: <54B8101D.2000907@gmail.com> References: <54B8101D.2000907@gmail.com> Message-ID: does http://erlang.org/doc/reference_manual/expressions.html#id81911 look like what you're asking for? F. On Thu, Jan 15, 2015 at 11:08 AM, the.silly.sad wrote: > Is there a complete list of operations allowed in guards? > organized in a human readable way as a "reference-book". > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From the.silly.sad@REDACTED Thu Jan 15 20:16:51 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Thu, 15 Jan 2015 20:16:51 +0100 Subject: [erlang-questions] What is allowed in guards? In-Reply-To: References: <54B8101D.2000907@gmail.com> Message-ID: <54B81223.5060006@gmail.com> On 01/15/2015 08:11 PM, Felix Gallo wrote: > does http://erlang.org/doc/reference_manual/expressions.html#id81911 look > like what you're asking for? is it complete? From bob@REDACTED Thu Jan 15 20:22:27 2015 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Jan 2015 06:22:27 +1100 Subject: [erlang-questions] What is allowed in guards? In-Reply-To: <54B81223.5060006@gmail.com> References: <54B8101D.2000907@gmail.com> <54B81223.5060006@gmail.com> Message-ID: On Thursday, January 15, 2015, the.silly.sad wrote: > On 01/15/2015 08:11 PM, Felix Gallo wrote: > >> does http://erlang.org/doc/reference_manual/expressions.html#id81911 look >> like what you're asking for? >> > > is it complete? Yes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From the.silly.sad@REDACTED Thu Jan 15 20:22:26 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Thu, 15 Jan 2015 20:22:26 +0100 Subject: [erlang-questions] What is allowed in guards? In-Reply-To: References: <54B8101D.2000907@gmail.com> <54B81223.5060006@gmail.com> Message-ID: <54B81372.2060101@gmail.com> On 01/15/2015 08:22 PM, Bob Ippolito wrote: > On Thursday, January 15, 2015, the.silly.sad > wrote: > >> On 01/15/2015 08:11 PM, Felix Gallo wrote: >> >>> does http://erlang.org/doc/reference_manual/expressions.html#id81911 look >>> like what you're asking for? >>> >> >> is it complete? > > > Yes. thanx. it is what i needed. From shawn@REDACTED Thu Jan 15 20:49:48 2015 From: shawn@REDACTED (Shawn Debnath) Date: Thu, 15 Jan 2015 19:49:48 +0000 Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests Message-ID: Hi there, Wondering if anyone can shed some light on this issue I am seeing. I am trying to understand what may be causing **error:{badmatch,{error,socket_closed_remotely}} when using the same inets process to do 2 POST/PUT requests. The same code works just fine when issuing the same rest API calls to two separate inets processes or when issuing 2 GET requests to the same inets process. Using webmachine on the server side which is not reporting any errors, and looking at the WMTRACEs, it correctly returns HTTP 401 error, which is expected. Test source and output can be found at: http://pastebin.com/9DFzxLps Thanks, Shawn -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Jan 15 22:05:36 2015 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 15 Jan 2015 13:05:36 -0800 Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests In-Reply-To: References: Message-ID: httpc has some heisenbugs which I wish I had time to examine and try to fix. The fact that a debug print statement makes the problem goes away suggests it's a race condition of some kind. The smart money is on trying to use either hackney (https://github.com/benoitc/hackney) or one of the lhttpc forks (https://github.com/Cloven/lhttpc). F. On Thu, Jan 15, 2015 at 11:49 AM, Shawn Debnath wrote: > Hi there, > > Wondering if anyone can shed some light on this issue I am seeing. > > I am trying to understand what may be causing > **error:{badmatch,{error,socket_closed_remotely}} when using the same inets > process to do 2 POST/PUT requests. The same code works just fine when > issuing the same rest API calls to two separate inets processes or when > issuing 2 GET requests to the same inets process. > > Using webmachine on the server side which is not reporting any errors, > and looking at the WMTRACEs, it correctly returns HTTP 401 error, which is > expected. > > Test source and output can be found at: http://pastebin.com/9DFzxLps > > Thanks, > Shawn > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Thu Jan 15 22:37:58 2015 From: marc@REDACTED (Marc Worrell) Date: Thu, 15 Jan 2015 22:37:58 +0100 Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests In-Reply-To: References: Message-ID: <710ADCB2-6D87-4F5A-B459-DB94F599EA91@worrell.nl> My feeling is that the error is exactly what it says: the remote server closes the socket. Adding a debug statement gives httpc the time to catch that close. Without the debug statement httpc will try to reuse the open connection to the server, at the moment the server is closing the connection. Try to use HTTP/1.0 and an extra request header "Connection: close". That might solve your problem. Also check the server. Do you have a proxy in front of webmachine? Maybe check if the socket is closed after the first request? - marc Sent from my iPad > On 15 jan. 2015, at 22:05, Felix Gallo wrote: > > httpc has some heisenbugs which I wish I had time to examine and try to fix. The fact that a debug print statement makes the problem goes away suggests it's a race condition of some kind. The smart money is on trying to use either hackney (https://github.com/benoitc/hackney) or one of the lhttpc forks (https://github.com/Cloven/lhttpc). > > F. > >> On Thu, Jan 15, 2015 at 11:49 AM, Shawn Debnath wrote: >> Hi there, >> >> Wondering if anyone can shed some light on this issue I am seeing. >> >> I am trying to understand what may be causing **error:{badmatch,{error,socket_closed_remotely}} when using the same inets process to do 2 POST/PUT requests. The same code works just fine when issuing the same rest API calls to two separate inets processes or when issuing 2 GET requests to the same inets process. >> >> Using webmachine on the server side which is not reporting any errors, and looking at the WMTRACEs, it correctly returns HTTP 401 error, which is expected. >> >> Test source and output can be found at: http://pastebin.com/9DFzxLps >> >> Thanks, >> Shawn >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From shawn@REDACTED Thu Jan 15 21:00:41 2015 From: shawn@REDACTED (Shawn Debnath) Date: Thu, 15 Jan 2015 20:00:41 +0000 Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests Message-ID: <78DD9B35-5871-46C8-89B6-C13ED010433B@debnath.net> Interesting finding, after adding some debug printing to print out headers, now I can?t repro the issue: 64 65 %% Invoke REST API calls via same inets/httpc processes 66 same_inets_for_calls_test() -> 67 68 inets:start(), 69 70 BeginUrl = ?API_BASE ++ "begin_media_post", 71 BeginH1 = [{"x-pryvy-authorization-status","Unknown"}], 72 BeginHeaders = [{"x-pryvy-authorized-userid","100"} | BeginH1], 73 {ok, {{_, BeginStatus, BeginResponseReason}, BeginResponseHeaders, 74 BeginResponseBody}} = 75 httpc:request(post, {BeginUrl, BeginHeaders, ?CONTENT_TYPE_JSON, 76 ?SAMPLE_JSON_CONTENT}, [], []), 77 io:format(standard_io, "resason: ~p~n headers: ~p~n body: ~p~n", 78 [BeginResponseReason, BeginResponseHeaders, BeginResponseBody]), 79 ?assertEqual(?HTTP_UNAUTH, BeginStatus), 80 81 EndUrl = ?API_BASE ++ "end_media_post", 82 EndH1 = [{"x-pryvy-authorization-status","Unknown"}], 83 EndHeaders = [{"x-pryvy-authorized-userid","100"} | EndH1], 84 {ok, {{_, EndStatus, EndResponseReason}, EndResponseHeaders, 85 EndResponseBody}} = 86 httpc:request(post, {EndUrl, EndHeaders, ?CONTENT_TYPE_JSON, 87 ?SAMPLE_JSON_CONTENT}, [], []), 88 io:format(standard_io, "resason: ~p~n headers: ~p~n body: ~p~n", 89 [EndResponseReason, EndResponseHeaders, EndResponseBody]), 90 ?assertEqual(?HTTP_UNAUTH, EndStatus), 91 92 inets:stop(). Smells like the inets/httpc timing issue? Thanks, Shawn From: Shawn Debnath Date: Thursday, January 15, 2015 at 11:49 AM To: Erlang Questions Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests Hi there, Wondering if anyone can shed some light on this issue I am seeing. I am trying to understand what may be causing **error:{badmatch,{error,socket_closed_remotely}} when using the same inets process to do 2 POST/PUT requests. The same code works just fine when issuing the same rest API calls to two separate inets processes or when issuing 2 GET requests to the same inets process. Using webmachine on the server side which is not reporting any errors, and looking at the WMTRACEs, it correctly returns HTTP 401 error, which is expected. Test source and output can be found at: http://pastebin.com/9DFzxLps Thanks, Shawn -------------- next part -------------- An HTML attachment was scrubbed... URL: From kwayne@REDACTED Thu Jan 15 21:01:36 2015 From: kwayne@REDACTED (Ken Wayne) Date: Thu, 15 Jan 2015 14:01:36 -0600 Subject: [erlang-questions] Newbie - Erlang VS other functional languages Message-ID: I've been investigating functional languages and the concepts that lead to increased speed, reliability, and decreased maintenance. Erlang seems to have a distinct advantage over other functional languages when you need to scale across multiple servers because it's a natural part of the language. Can anyone confirm/deny or elaborate on the observation? Without wax, Ken Wayne kwayne@REDACTED Desk: 715.261.9412 -------------- next part -------------- An HTML attachment was scrubbed... URL: From shawn@REDACTED Thu Jan 15 23:15:22 2015 From: shawn@REDACTED (Shawn Debnath) Date: Thu, 15 Jan 2015 22:15:22 +0000 Subject: [erlang-questions] inets/httpc bad match issue for 2 post requests In-Reply-To: <710ADCB2-6D87-4F5A-B459-DB94F599EA91@worrell.nl> References: <710ADCB2-6D87-4F5A-B459-DB94F599EA91@worrell.nl> Message-ID: <366D3E9A-E5A7-4A50-BEC9-BC6739420F2B@debnath.net> Well looks like that was the issue. Using just HTTP/1.0 without the "Connection: close" header does the trick as it forces the connection close. Tried with ?Connection: keep-alive? header without the HTTP/1.0 option but that was a no-go as well. Looks like its time to follow up with webmachine guys. Thanks Marc and Felix. From: Marc Worrell Date: Thursday, January 15, 2015 at 1:37 PM To: Felix Gallo Cc: Shawn Debnath, Erlang Questions Subject: Re: [erlang-questions] inets/httpc bad match issue for 2 post requests My feeling is that the error is exactly what it says: the remote server closes the socket. Adding a debug statement gives httpc the time to catch that close. Without the debug statement httpc will try to reuse the open connection to the server, at the moment the server is closing the connection. Try to use HTTP/1.0 and an extra request header "Connection: close". That might solve your problem. Also check the server. Do you have a proxy in front of webmachine? Maybe check if the socket is closed after the first request? - marc Sent from my iPad On 15 jan. 2015, at 22:05, Felix Gallo > wrote: httpc has some heisenbugs which I wish I had time to examine and try to fix. The fact that a debug print statement makes the problem goes away suggests it's a race condition of some kind. The smart money is on trying to use either hackney (https://github.com/benoitc/hackney) or one of the lhttpc forks (https://github.com/Cloven/lhttpc). F. On Thu, Jan 15, 2015 at 11:49 AM, Shawn Debnath > wrote: Hi there, Wondering if anyone can shed some light on this issue I am seeing. I am trying to understand what may be causing **error:{badmatch,{error,socket_closed_remotely}} when using the same inets process to do 2 POST/PUT requests. The same code works just fine when issuing the same rest API calls to two separate inets processes or when issuing 2 GET requests to the same inets process. Using webmachine on the server side which is not reporting any errors, and looking at the WMTRACEs, it correctly returns HTTP 401 error, which is expected. Test source and output can be found at: http://pastebin.com/9DFzxLps Thanks, Shawn _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu Jan 15 23:38:43 2015 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Jan 2015 11:38:43 +1300 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: References: Message-ID: I'd agree with that observation. Erlang is particularly well designed for reliability and ease of maintenance/debugging. I wouldn't necessarily say that these properties are due to the language, it's really the environments that Erlang has been deployed in that shaped the VM and libraries in this way. The tooling and libraries have at least a decade head start for this kind of industrial usage over just about any other functional language. On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne wrote: > I've been investigating functional languages and the concepts that lead to > increased speed, reliability, and decreased maintenance. Erlang seems to > have a distinct advantage over other functional languages when you need to > scale across multiple servers because it's a natural part of the language. > Can anyone confirm/deny or elaborate on the observation? > > Without wax, > Ken Wayne > kwayne@REDACTED > Desk: 715.261.9412 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jan 16 04:05:40 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 16 Jan 2015 16:05:40 +1300 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: On 14/01/2015, at 2:57 am, Richard Carlsson wrote: > What really should be done is that the current interpreting debugger should be destroyed with fire and a real bytecode-level debugger should be implemented (with low level functionality done in C), which would be way faster, only require updating when new Beam operations are added, and not require the abstract code to be present in the beam files. There are certainly advantages to a byte code debugger, but byte-code compilation destroys information like variable names and source locations. The thing I want above all from a debugging tool is that it should minimise the amount of output I have to look at. I want to be able to tell it my expectations and have it only show me things that are surprising. Given the amount of filtering and suppression of repetition that are appropriate to economise on *my* time, it is not clear that working at the byte code level *would* be any faster over all. That is, after all, one of the Erlang Lessons: speed is a property of whole systems, not just of compilers. From achowdhury918@REDACTED Fri Jan 16 06:17:03 2015 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Fri, 16 Jan 2015 00:17:03 -0500 Subject: [erlang-questions] cowboy websocket crash, need help Message-ID: All, I am trying to establish an websocket connection using cowboy. I am using the base example given in cowboy websocket github. I am getting following crash: 2015-01-15 20:40:01.777 [error] emulator Error in process <0.185.0> on node 'erws@REDACTED' with exit value: {[{re ason,undef},{mfa,{erws_handler,terminate,3}},{stacktrace,[{lager,debug,["terminate"],[]},{erws_handler,terminate,3,[{fi le,"erws_handler.erl"},{line,48}]},{cowboy_handler,terminate,4,[{file,"cowboy_handler.erl"},{line,69}]},... 2015-01-15 20:40:01.781 [error] <0.81.0> Ranch listener http had connection process started with cowboy_protocol:start_ link/4 at <0.185.0> exit with reason: {[{reason,undef},{mfa,{erws_handler,terminate,3}},{stacktrace,[{lager,debug,["ter minate"],[]},{erws_handler,terminate,3,[{file,"erws_handler.erl"},{line,48}]},{cowboy_handler,terminate,4,[{file,"cowbo y_handler.erl"},{line,69}]},{cowboy_handler,execute,2,[{file,"cowboy_handler.erl"},{line,54}]},{cowboy_protocol,execute ,4,[{file,"cowboy_protocol.erl"},{line,467}]}]},{req,[{socket,#Port<0.2498>},{transport,ranch_tcp},{connection,keepaliv e},{pid,<0.185.0>},{method,<<"GET">>},{version,'HTTP/1.1'},{peer,{{10,69,209,221},49793}},{host,<<"sas755dev.hss.vzwcor p.com ">>},{host_info,undefined},{port,10100},{path,<<"/websocket">>},{path_info,undefined},{qs,<<>>},{bindings,[]},{hea ders,[{<<"host">>,<<"sas755dev.hss.xxxxxx.com:10100">>},{<<"user-agent">>,<<"Mozilla/5.0 (Windows NT 6.1; rv:34.0) Gec ko/20100101 Firefox/34.0">>},{<<"accept">>,<<"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8">>},{<<"a ccept-language">>,<<"en-US,en;q=0.5">>},{<<"accept-encoding">>,<<"gzip, deflate">>},{<<"sec-websocket-version">>,<<"13" >>},{<<"origin">>,<<"null">>},{<<"sec-websocket-key">>,<<"Uj2HLRfte83Kyu9r9qEesQ==">>},{<<"connection">>,<<"keep-alive, Upgrade">>},{<<"pragma">>,<<"no-cache">>},{<<"cache-control">>,<<"no-cache">>},{<<"upgrade">>,<<"websocket">>}]},{meta ,[]},{body_state,waiting},{buffer,<<>>},{multipart,undefined},{resp_compress,false},{resp_state,waiting},{resp_headers, []},{resp_body,<<>>},{onresponse,undefined}]},{state,[]},{terminate_reason,{crash,error,undef}}],[{cowboy_handler,execu te,2,[{file,"cowboy_handler.erl"},{line,59}]},{cowboy_protocol,execute,4,[{file,"cowboy_protocol.erl"},{line,467}]}]} Here is my erws_handler.erl file which is almost like example file provided in cowboy websocket github -module(erws_handler). -behaviour(cowboy_http_handler). -behaviour(cowboy_websocket_handler). -export([init/3, handle/2, terminate/3]). -export([ websocket_init/3, websocket_handle/3, websocket_info/3, websocket_terminate/3 ]). init({tcp, http}, _Req, _Opts) -> {upgrade, protocol, cowboy_websocket}. handle(Req, State) -> lager:debug("Request not expected: ~p", [Req]), {ok, Req2} = cowboy_http_req:reply(404, [{'Content-Type', <<"text/html">>}]), {ok, Req2, State}. websocket_init(_TransportName, Req, _Opts) -> lager:debug("init websocket"), {ok, Req, undefined_state}. websocket_handle({text, Msg}, Req, State) -> lager:debug("Got Data: ~p", [Msg]), {reply, {text, << "responding to ", Msg/binary >>}, Req, State, hibernate }; websocket_handle(_Any, Req, State) -> lager:debug("Got Any"), {reply, {text, << "whut?">>}, Req, State, hibernate }. websocket_info({timeout, _Ref, Msg}, Req, State) -> lager:debug("websocket info_timeout"), {reply, {text, Msg}, Req, State}; websocket_info(_Info, Req, State) -> lager:debug("websocket info"), {ok, Req, State, hibernate}. websocket_terminate(_Reason, _Req, _State) -> % lager:debug("websocket terminate. reason: ~p",[_Reason]), lager:debug("websocket terminate"), ok. terminate(_Reason, _Req, _State) -> % lager:debug("terminate. Reason : ~p",[_Reason]), lager:debug("terminate"), ok. I am trying from a simple html file through Mozilla Firefox : ws = new WebSocket("ws://sas755dev.hss.xxxxxx.com:10100/websocket"); I need to provide port number with host name. Is that ok? Does cowboy support it? Is providing port number causing this issue? If not, then what else? Can anyone help me? Any help regarding this will be highly appreciated. Thanks. - Akash -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Fri Jan 16 11:52:19 2015 From: grahamrhay@REDACTED (Graham Hay) Date: Fri, 16 Jan 2015 10:52:19 +0000 Subject: [erlang-questions] cowboy websocket crash, need help In-Reply-To: References: Message-ID: Hi, Have you run the parse transform, and started the lager app? Also, is there a reason you're implementing both the cowboy_http_handler & cowboy_websocket_handler behaviours? (That's not the problem, just curious). Graham On 16 January 2015 at 05:17, Akash Chowdhury wrote: > All, > I am trying to establish an websocket connection using cowboy. I am using > the base example given in cowboy websocket github. I am getting following > crash: > > 2015-01-15 20:40:01.777 [error] emulator Error in process <0.185.0> on > node 'erws@REDACTED' with exit value: {[{re > > ason,undef},{mfa,{erws_handler,terminate,3}},{stacktrace,[{lager,debug,["terminate"],[]},{erws_handler,terminate,3,[{fi > > le,"erws_handler.erl"},{line,48}]},{cowboy_handler,terminate,4,[{file,"cowboy_handler.erl"},{line,69}]},... > > > 2015-01-15 20:40:01.781 [error] <0.81.0> Ranch listener http had > connection process started with cowboy_protocol:start_ > link/4 at <0.185.0> exit with reason: > {[{reason,undef},{mfa,{erws_handler,terminate,3}},{stacktrace,[{lager,debug,["ter > > minate"],[]},{erws_handler,terminate,3,[{file,"erws_handler.erl"},{line,48}]},{cowboy_handler,terminate,4,[{file,"cowbo > > y_handler.erl"},{line,69}]},{cowboy_handler,execute,2,[{file,"cowboy_handler.erl"},{line,54}]},{cowboy_protocol,execute > > ,4,[{file,"cowboy_protocol.erl"},{line,467}]}]},{req,[{socket,#Port<0.2498>},{transport,ranch_tcp},{connection,keepaliv > > e},{pid,<0.185.0>},{method,<<"GET">>},{version,'HTTP/1.1'},{peer,{{10,69,209,221},49793}},{host,<<"sas755dev.hss.vzwcor > p.com > ">>},{host_info,undefined},{port,10100},{path,<<"/websocket">>},{path_info,undefined},{qs,<<>>},{bindings,[]},{hea > ders,[{<<"host">>,<<"sas755dev.hss.xxxxxx.com:10100">>},{<<"user-agent">>,<<"Mozilla/5.0 > (Windows NT 6.1; rv:34.0) Gec > ko/20100101 > Firefox/34.0">>},{<<"accept">>,<<"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8">>},{<<"a > ccept-language">>,<<"en-US,en;q=0.5">>},{<<"accept-encoding">>,<<"gzip, > deflate">>},{<<"sec-websocket-version">>,<<"13" > > >>},{<<"origin">>,<<"null">>},{<<"sec-websocket-key">>,<<"Uj2HLRfte83Kyu9r9qEesQ==">>},{<<"connection">>,<<"keep-alive, > > Upgrade">>},{<<"pragma">>,<<"no-cache">>},{<<"cache-control">>,<<"no-cache">>},{<<"upgrade">>,<<"websocket">>}]},{meta > > ,[]},{body_state,waiting},{buffer,<<>>},{multipart,undefined},{resp_compress,false},{resp_state,waiting},{resp_headers, > > []},{resp_body,<<>>},{onresponse,undefined}]},{state,[]},{terminate_reason,{crash,error,undef}}],[{cowboy_handler,execu > > te,2,[{file,"cowboy_handler.erl"},{line,59}]},{cowboy_protocol,execute,4,[{file,"cowboy_protocol.erl"},{line,467}]}]} > > Here is my erws_handler.erl file which is almost like example file > provided in cowboy websocket github > > -module(erws_handler). > -behaviour(cowboy_http_handler). > -behaviour(cowboy_websocket_handler). > -export([init/3, handle/2, terminate/3]). > -export([ > websocket_init/3, websocket_handle/3, > websocket_info/3, websocket_terminate/3 > ]). > > init({tcp, http}, _Req, _Opts) -> > {upgrade, protocol, cowboy_websocket}. > > > handle(Req, State) -> > lager:debug("Request not expected: ~p", [Req]), > {ok, Req2} = cowboy_http_req:reply(404, [{'Content-Type', > <<"text/html">>}]), > {ok, Req2, State}. > > > websocket_init(_TransportName, Req, _Opts) -> > lager:debug("init websocket"), > {ok, Req, undefined_state}. > > websocket_handle({text, Msg}, Req, State) -> > lager:debug("Got Data: ~p", [Msg]), > {reply, {text, << "responding to ", Msg/binary >>}, Req, State, > hibernate }; > > > websocket_handle(_Any, Req, State) -> > lager:debug("Got Any"), > {reply, {text, << "whut?">>}, Req, State, hibernate }. > > websocket_info({timeout, _Ref, Msg}, Req, State) -> > lager:debug("websocket info_timeout"), > {reply, {text, Msg}, Req, State}; > > websocket_info(_Info, Req, State) -> > lager:debug("websocket info"), > {ok, Req, State, hibernate}. > > websocket_terminate(_Reason, _Req, _State) -> > % lager:debug("websocket terminate. reason: ~p",[_Reason]), > lager:debug("websocket terminate"), > ok. > > terminate(_Reason, _Req, _State) -> > % lager:debug("terminate. Reason : ~p",[_Reason]), > lager:debug("terminate"), > ok. > > I am trying from a simple html file through Mozilla Firefox : > > ws = new WebSocket("ws://sas755dev.hss.xxxxxx.com:10100/websocket"); > > I need to provide port number with host name. Is that ok? Does cowboy > support it? Is providing port number causing this issue? If not, then what > else? Can anyone help me? Any help regarding this will be highly > appreciated. > Thanks. > - > Akash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Fri Jan 16 12:07:49 2015 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 16 Jan 2015 12:07:49 +0100 Subject: [erlang-questions] Differences in code_change callback between gen_server and gen_fsm In-Reply-To: References: Message-ID: Hi Florian! The way gen_fsm (and gen_event) is specified is actually the original - i.e. the code_change should always succeed. The function spec in gen_server was updated in OTP R15B to allow an error return since this was required (and had always been used) by supervisor ( https://github.com/erlang/otp/commit/038b9dd3a1f9bdd86cbb83bf3484ab1529d4fca2). The release_handler was updated at the same time (so even if the supervisor did use it before, the result would probably not have been too nice if it actually failed). Anyway... I guess there would be no harm in updating the spec in gen_fsm to reflect that an error return is allowed (even if it is not a good idea - it will fail the complete upgrade and roll back to the old release. A fact which would have to be stated in the reference manual.) I could write at ticket for this, but the chance that it would be prioritized within reasonable time is extremely small. A contribution would help, of course... When it comes to gen_event, the change would be a bit more complicated. Errors are not handled at all in the code as it is now. Regards /siri 2015-01-06 19:41 GMT+01:00 Florian Waas : > In gen_server the callback for code_change is spec?d as > > Module:code_change(OldVsn, State, Extra) -> {ok, NewState} | {error, > Reason} > > in gen_fsm as > > Module:code_change(OldVsn, StateName, StateData, Extra) -> {ok, > NextStateName, NewStateData} > > i.e., upgrades on FSM?s had better succeed? The underlying implementation > in sys.erl is the same for both and allows for an error case. > Am I missing some subtlety here? > > Thanks, > -fl. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Fri Jan 16 12:16:48 2015 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 16 Jan 2015 12:16:48 +0100 Subject: [erlang-questions] Stopping a supervisor running under a supervisor In-Reply-To: References: Message-ID: Hi Chris, what does you childspec for my_sup (i.e. the childspec specified in my_supersup) look like - or really, is the Shutdown element set to brutal_kill? If supervisor:terminate_child/2 causes the processes to be killed, that is probably the case. The recommendation is that Shutdown is set to infinity if the child is a supervisor, which will give it unlimited time to gracefully shut down all it's children and then itself. Regards /siri 2015-01-06 16:31 GMT+01:00 Chris Clark : > Hi Sean > > supervisor:terminate_child/2 does work however it causes errors in my logs > so i am looking for a way to gracefully terminate the children. > > Thanks > > On Tue, Jan 6, 2015 at 4:20 PM, Sean Cribbs wrote: > >> Did you try supervisor:terminate_child/2? >> >> On Mon, Jan 5, 2015 at 11:44 PM, Chris Clark >> wrote: >> >>> Hi Antonio >>> >>> Thanks for the assistance. From what I have read a supervisor should >>> stop if it is sent a shutdown signal using exit(Pid, shutdown). but it >>> will only accept that signal from its parent process in an OTP application. >>> I tried creating a function in the parent supervisor(my_supersup) >>> stop_child(Pid) that then sends the exit(pid, shutdown) signal and then >>> call that from the shell but that doesn't seem to stop it either. >>> >>> Thanks, >>> >>> On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias >> > wrote: >>> >>>> Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message >>>> and use that to stop the process? >>>> >>>> ADias >>>> >>>> On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> I am fairly new to erlang and have a question regarding properly >>>>> stopping a supervisor that resides directly under another supervisor. >>>>> >>>>> My application consist of a top level supervisor (my_supersup) which >>>>> is a simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >>>>> supervises a gen_server, serv and another supervisor that is also a >>>>> simple_one_for_one supervisor of some workers >>>>> >>>>> my_supersup >>>>> (simple_one_for_one, transient) >>>>> | >>>>> |_______ ... >>>>> | >>>>> my_sup (one_for_all, transient) >>>>> |________________ >>>>> | | >>>>> | serv >>>>> worker_sup >>>>> (simple_one_for_one, transient) >>>>> |___________ ... >>>>> | | >>>>> w1 w2 >>>>> >>>>> How do shutdown an instance of my_sup gracefully without killing it. >>>>> At present I can do it with supervisor:terminate_child and >>>>> supervisor:delete_child but I would like them to shutdown rather than be >>>>> killed if possible? >>>>> >>>>> Thanks in advance. >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Sean Cribbs >> Sr. Software Engineer >> Basho Technologies, Inc. >> http://basho.com/ >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Fri Jan 16 12:38:49 2015 From: davidnwelton@REDACTED (David Welton) Date: Fri, 16 Jan 2015 12:38:49 +0100 Subject: [erlang-questions] Graphical display of messages In-Reply-To: References: Message-ID: I threw this together, for the heck of it: https://github.com/davidw/tracemap It works, but is far from polished or something that should be run on a production system! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From jesper.louis.andersen@REDACTED Fri Jan 16 13:07:54 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 16 Jan 2015 12:07:54 +0000 Subject: [erlang-questions] Stopping a supervisor running under a supervisor References: Message-ID: I'd go with a solution where you mark the children as 'transient' and then build code inside them to stop gracefully. That way, if the termination is due to a 'normal' exit, there should be no indication of anything in the log files. On Tue Jan 06 2015 at 4:32:01 PM Chris Clark wrote: > Hi Sean > > supervisor:terminate_child/2 does work however it causes errors in my logs > so i am looking for a way to gracefully terminate the children. > > Thanks > > On Tue, Jan 6, 2015 at 4:20 PM, Sean Cribbs wrote: > >> Did you try supervisor:terminate_child/2? >> >> On Mon, Jan 5, 2015 at 11:44 PM, Chris Clark >> wrote: >> >>> Hi Antonio >>> >>> Thanks for the assistance. From what I have read a supervisor should >>> stop if it is sent a shutdown signal using exit(Pid, shutdown). but it >>> will only accept that signal from its parent process in an OTP application. >>> I tried creating a function in the parent supervisor(my_supersup) >>> stop_child(Pid) that then sends the exit(pid, shutdown) signal and then >>> call that from the shell but that doesn't seem to stop it either. >>> >>> Thanks, >>> >>> On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias >> > wrote: >>> >>>> Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message >>>> and use that to stop the process? >>>> >>>> ADias >>>> >>>> On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> I am fairly new to erlang and have a question regarding properly >>>>> stopping a supervisor that resides directly under another supervisor. >>>>> >>>>> My application consist of a top level supervisor (my_supersup) which >>>>> is a simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >>>>> supervises a gen_server, serv and another supervisor that is also a >>>>> simple_one_for_one supervisor of some workers >>>>> >>>>> my_supersup >>>>> (simple_one_for_one, transient) >>>>> | >>>>> |_______ ... >>>>> | >>>>> my_sup (one_for_all, transient) >>>>> |________________ >>>>> | | >>>>> | serv >>>>> worker_sup >>>>> (simple_one_for_one, transient) >>>>> |___________ ... >>>>> | | >>>>> w1 w2 >>>>> >>>>> How do shutdown an instance of my_sup gracefully without killing it. >>>>> At present I can do it with supervisor:terminate_child and >>>>> supervisor:delete_child but I would like them to shutdown rather than be >>>>> killed if possible? >>>>> >>>>> Thanks in advance. >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Sean Cribbs >> Sr. Software Engineer >> Basho Technologies, Inc. >> http://basho.com/ >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jan 16 13:29:39 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 16 Jan 2015 12:29:39 +0000 Subject: [erlang-questions] Newbie - Erlang VS other functional languages References: Message-ID: I think your observation is correct. An Erlang program works by having many small processes, all isolated from each other. The way to communicate between processes is to send a message, asynchronously. This in turn leads to the key observation: when you send messages, you don't care about *where* the other process is. It could be local or on a completely different machine. The syntax and the semantics are the same, and you would program the system much in the same way. The environment is thus very homogeneous, compared to other solutions where you need to communicate on two levels: one for local messaging and one for distributed messaging. I also second Bob's observation: The design feature of being functional forces a lot of properties which are beneficial to programs where correctness matters more than squeezing out the last ounces of performance from a tight computational kernel. But there is more to it than that. A good example is the choice of standard data structures which have no pathological problems in corner cases. Or the deep continued focus on scaling to multiple cores rather than looking for efficient single-core performance. On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: > I'd agree with that observation. Erlang is particularly well designed for > reliability and ease of maintenance/debugging. I wouldn't necessarily say > that these properties are due to the language, it's really the environments > that Erlang has been deployed in that shaped the VM and libraries in this > way. The tooling and libraries have at least a decade head start for this > kind of industrial usage over just about any other functional language. > > On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne wrote: > >> I've been investigating functional languages and the concepts that lead >> to increased speed, reliability, and decreased maintenance. Erlang seems >> to have a distinct advantage over other functional languages when you need >> to scale across multiple servers because it's a natural part of the >> language. Can anyone confirm/deny or elaborate on the observation? >> >> Without wax, >> Ken Wayne >> kwayne@REDACTED >> Desk: 715.261.9412 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Fri Jan 16 15:06:21 2015 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 16 Jan 2015 15:06:21 +0100 Subject: [erlang-questions] [ANN] Call for Paper: Codemotion Rome 2015 March 27-28 Message-ID: Hi everyone, There is an interesting conference in Rome that could use some Erlang talks. Codemotion Rome 2015 is looking for talk submission until *27th January*. Topics can be: *mobile to UX, DevOps, cloud, big data, gamedev, security, methods, languages, web, Internet of things*. Codemotion Rome 2015 is taking place on March 27th and 28th, 2015. Submission: http://speaker.codemotionworld.com/c4p.php More information: http://rome2015.codemotionworld.com/ Best regards, Bruce -- Bruce Yinhe Erlang Community Manager Industrial Erlang User Group community-manager@REDACTED +46 72 311 43 89 -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Jan 16 16:03:13 2015 From: g@REDACTED (Garrett Smith) Date: Fri, 16 Jan 2015 09:03:13 -0600 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: References: Message-ID: I don't think Erlang has an edge over any other language in terms of scaling across multiple servers - not at all. Other functional languages have access to network APIs the same way Erlang does. I suppose it's less fiddly to send Erlang terms across the wire. Erlang has some built in facilities for building distributed applications. Erlang's been doing this sort of thing for a long time. But other languages can do it - particularly with the many multi-language messaging libraries and tools available today. What Erlang is special for IMO is it concurrency model, which is implemented in and enforced by the *VM* - there's no shared memory across threads of execution. One thread cannot corrupt the memory used by another. To communicate between threads of execution, threads must pass and receive messages (copied data). I use the word threads generically here - in Erlang they're called processes. This changes everything. It's completely transformative of the way you build software. But the real payoff is in your locally running program and less so in the ability to "distribute". Try building software using single threaded, isolated processes (no shared memory). How do you do it? If you use Erlang, you're forced to do it - there's no choice. It's the same as the operating system level. How do you build a LAMP stack? (sorry, I'm older then 24) You install independent components (Apache, PHP (fork exec'd), MySQL). Then you configure them to work together. Then you start them up in a particular order. It's coordinated communication independent functions. If one blows up, the others keep working. Imagine every part of your program working like this and you have an Erlang application - actually, an Erlang *system*. To replicate this model at the OS level, you'd write dozens of small, independent applications that communicated with each other over pipes or sockets. The ZeroMQ community is familiar with this approach. The payoff for this, as I see it, is flexibility and speed of introducing new functionality and fixing bugs. To illustrate at a higher level, imagine a smart phone today that didn't provide isolation across applications. What would you expect from it? I'd expect it to not work unless the applications were all nearly perfect. That means it will either not work, or the app ecosystem would be very limited. But today, smart phones all have kernels and user space where apps are isolated from one another. So I can install some random thing from an app store and have a pretty high confidence that it's not going to ruin my phone. The result is *huge* app ecosystems with phones that pretty much work (shockingly well for what they're asked to do). When you use Erlang, your program becomes this ecosystem of "apps" that you can add to, modify, remove very freely without concern for the whole system. Your program will be evolvable *and stable* in the same way your phone is evolvable and stable. It's a scalable programming model! It's truly fantastic - and seldom mentioned. (Folks here have heard this line from me before - this is my particular drum that I like to beat on :) Use Erlang! Garrett P.S. I routinely run into critical memory problems in Java apps that host ecosystems of other "apps" (plugins). It's a completely unsolvable problem when your language/VM encourages intractable memory graphs across threads unless you are incredibly careful about what you run. If you want to make a system pluggable in Java (in one JVM), be prepared for it to stop working at some point. So your either limited in what you can do (and how fast) or in the stability of your program. On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen wrote: > I think your observation is correct. > > An Erlang program works by having many small processes, all isolated from > each other. The way to communicate between processes is to send a message, > asynchronously. This in turn leads to the key observation: when you send > messages, you don't care about *where* the other process is. It could be > local or on a completely different machine. The syntax and the semantics are > the same, and you would program the system much in the same way. The > environment is thus very homogeneous, compared to other solutions where you > need to communicate on two levels: one for local messaging and one for > distributed messaging. > > I also second Bob's observation: The design feature of being functional > forces a lot of properties which are beneficial to programs where > correctness matters more than squeezing out the last ounces of performance > from a tight computational kernel. But there is more to it than that. A good > example is the choice of standard data structures which have no pathological > problems in corner cases. Or the deep continued focus on scaling to multiple > cores rather than looking for efficient single-core performance. > > > On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: >> >> I'd agree with that observation. Erlang is particularly well designed for >> reliability and ease of maintenance/debugging. I wouldn't necessarily say >> that these properties are due to the language, it's really the environments >> that Erlang has been deployed in that shaped the VM and libraries in this >> way. The tooling and libraries have at least a decade head start for this >> kind of industrial usage over just about any other functional language. >> >> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne wrote: >>> >>> I've been investigating functional languages and the concepts that lead >>> to increased speed, reliability, and decreased maintenance. Erlang seems to >>> have a distinct advantage over other functional languages when you need to >>> scale across multiple servers because it's a natural part of the language. >>> Can anyone confirm/deny or elaborate on the observation? >>> >>> Without wax, >>> Ken Wayne >>> kwayne@REDACTED >>> Desk: 715.261.9412 >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mononcqc@REDACTED Fri Jan 16 17:17:54 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 16 Jan 2015 11:17:54 -0500 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: References: Message-ID: <20150116161753.GE7939@ferdair.local> On 01/16, Garrett Smith wrote: > I don't think Erlang has an edge over any other language in terms of > scaling across multiple servers - not at all. Other functional > languages have access to network APIs the same way Erlang does. > > I suppose it's less fiddly to send Erlang terms across the wire. > Erlang has some built in facilities for building distributed > applications. Erlang's been doing this sort of thing for a long time. > But other languages can do it - particularly with the many > multi-language messaging libraries and tools available today. > > What Erlang is special for IMO is it concurrency model, which is > implemented in and enforced by the *VM* - there's no shared memory > across threads of execution. One thread cannot corrupt the memory used > by another. To communicate between threads of execution, threads must > pass and receive messages (copied data). > Ah but this is why Erlang is also good for distributed systems! These things are related, not independent. Distributed systems communicate by message passing and cannot share anything (well, they can, but I encourage you to read some papers on distributed object garbage collection on why this is a bad idea). This requirement to do it locally ends up giving us timers, links, monitors, and all other kinds of goodies that are useful when doing distributed programming. Of course there are caveats; a local process that dies can be monitored and linked and will pretty much always be dead when you receive that message (bar some rare VM bug in old versions). Aremote process can die with the reason 'nodedown', meaning that a remote node is gone. We can't know if the process is truly dead, the node timing out, the communication pipe saturated. That requires special consideration. But Erlang has *all* of that tooling already in place. Hell, there's a version id in each pid, port, and reference that the VM sets (through EPMD) to make it possible to know (with up to 3 restarts) if the pid is the same as the one you talked to if they share the same number. It is right that Erlang doesn't take the hard problems out of distributed computing: a lot of libraries shipping with the language don't even consider netsplits a possibility (some do -- global, pg2, etc.). However, it has put in place a lot of scaffolding to solve almost all of the easy problems. This means that when you're building a distributed system, you still have to fight all the nasty bugs and assumption that go in your design. But at least, you don't nearly have to fight all the nasty bugs and assumption in all the communication mechanisms that your system is built on as often as you would in most other languages out there. Regards, Fred. From jim.rosenblum@REDACTED Fri Jan 16 17:47:05 2015 From: jim.rosenblum@REDACTED (jim rosenblum) Date: Fri, 16 Jan 2015 11:47:05 -0500 Subject: [erlang-questions] Trouble with JInterface In-Reply-To: References: Message-ID: Utilizing Vlad's suggestion of tracing via -DOtpConnection.trace showed us where our problem was. And it was OUR problem, JInterface is just fine. Thanks for the help. On Tue, Jan 13, 2015 at 10:01 AM, Vlad Dumitrescu wrote: > Hi Jim! > > Does this happen every time, for all mailboxes? > > Did you try to trace the Java code, to see if there is more information to > get? > Use -DOtpConnection.trace=3 and you will get all the activity on the > connection from the Java side (it's global, so it will be for _everything_, > you might want to try starting with a value of 1 at first and increase if > it's not enough). > > regards, > Vlad > > > On Tue, Jan 13, 2015 at 3:39 PM, jim rosenblum > wrote: > >> Folks, >> >> We are having a problem with Jinterface and Erlang 17.2. Essentially we >> have a Java/Clojure application that is using a home-brewed, Erlang, >> two-node, in-memory cache. We think the problem is either a bug with >> Jinterface or a misunderstanding on how to use it. >> >> On the Java side, we create a node and 1 to 12 unnamed mailboxes. We have >> a receive loop on each mailbox: receiving messages from Erlang, doing the >> right things with those messages, catching any errors and looping back to >> the receive. In general, everything works fine. >> >> When one of the Erlang cache-nodes goes down, the application recreates >> its mailboxes, connects to the ?other? Erlang node via the new mailboxes >> and attempts to carry on. Here is where we have the problem: >> >> The Failover Protocol: >> For each mailbox >> 1. Receive loop gets OtpErlangExit >> 2. Existing mailbox is closed >> 3. New unnamed mailbox created >> 4. Application appropriate message is sent to the failover node via this >> new mailbox >> 5. Loop to the receive >> >> Symptom: >> If the receive loop has a timeout, no message is received by the new >> mailbox until AFTER the timeout fires and the loop iterates back to the >> receive. If there is no timeout specified the receive hangs (seemingly) >> forever. We have confirmed that the failover Erlang node received the >> message (step 4) sent via the new mailbox, and that the failover Erlang >> node is sending messages to the PID representing the new mailbox ? >> inspecting my server using Observer, I can see that it is sending the >> expected message to the correct PID. >> >> Our workaround is to use a short timeout in our mailbox receive loop so >> that we don?t hang for too long, but we are troubled by the thought that we >> might not understand JInterface best-practices as we are skeptical that the >> JInterface code is anything but perfect :) >> >> Is this a known issue, or does someone have any advice on how to proceed? >> >> >> Thanks, >> >> Jr0 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From the.silly.sad@REDACTED Fri Jan 16 17:58:14 2015 From: the.silly.sad@REDACTED (the.silly.sad) Date: Fri, 16 Jan 2015 17:58:14 +0100 Subject: [erlang-questions] is anyone hiring? Message-ID: <54B94326.1010709@gmail.com> Hello, all. Could you suggest a good method of erlang-related job search? Or some business entities that are worth sending a junior CV to? Would you like to share some links or advises or maybe anyone of you guys is hiring? From lloyd@REDACTED Fri Jan 16 19:31:55 2015 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Fri, 16 Jan 2015 13:31:55 -0500 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: References: Message-ID: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Ah Professor Smith, Once again I must confess my ignorance in hope that you can bring clarity. I've been working happily with sequential Erlang now for several years--- most recently an ambitious Nitrogen app. And I've heard from the beginning the virtues of of distributed concurrent processes happily chattering among themselves. But where the rubber meets the road, e.g. as I design my current app, I just don't get how to transform these virtues from virtual to real. Conceptually, I understand spawned processes, message passing, and supervisors. I've read and reread the Erlang canon. I've built several gen-servers and have worked my way through your e2 tutorial. It all makes good sense. But as I consider these principles in context of the architectural design of my current app, it's not at all clear how to apply them. The goal of my webapp is to deliver a set of data-based "tools" to "users" (wouldn't it be nice if we had many of these elusive critters) where each user owns his/her own data. In my case, users are non-technical author-publishers where each has a personal page for accessing tools and managing data related to project, marketing, and business management. Best I can tell, Cowboy and Nitrogen deal with "connections" and "sessions" under the hood, so I don't have to worry about them. But I struggle with such questions as: -- Should each "tool" be implemented as a gen-server? -- Should the user interface and database be further factored as separate processes? -- Should each "tool" be developed as a separate Erlang application then integrated as dependencies of a higher-level portal? Or should they be developed as a set of modules in one application? -- Or maybe each tool should be a totally independent "microservice," whatever that means. In other words, my attentive studies of Erlang have left me with very few PRACTICAL architectural techniques and tools; that is an insufficient bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of building robust Erlang systems. I'd much appreciate any guidelines to help me thrash through my confusion. But more, I wonder if others struggle with the same issues? And if so, how can I work with wizards like you to shed light on this corner of Erlang technology? All the best, Lloyd Sent from my iPad > On Jan 16, 2015, at 10:03 AM, Garrett Smith wrote: > > I don't think Erlang has an edge over any other language in terms of > scaling across multiple servers - not at all. Other functional > languages have access to network APIs the same way Erlang does. > > I suppose it's less fiddly to send Erlang terms across the wire. > Erlang has some built in facilities for building distributed > applications. Erlang's been doing this sort of thing for a long time. > But other languages can do it - particularly with the many > multi-language messaging libraries and tools available today. > > What Erlang is special for IMO is it concurrency model, which is > implemented in and enforced by the *VM* - there's no shared memory > across threads of execution. One thread cannot corrupt the memory used > by another. To communicate between threads of execution, threads must > pass and receive messages (copied data). > > I use the word threads generically here - in Erlang they're called processes. > > This changes everything. It's completely transformative of the way you > build software. But the real payoff is in your locally running program > and less so in the ability to "distribute". > > Try building software using single threaded, isolated processes (no > shared memory). How do you do it? If you use Erlang, you're forced to > do it - there's no choice. > > It's the same as the operating system level. How do you build a LAMP > stack? (sorry, I'm older then 24) You install independent components > (Apache, PHP (fork exec'd), MySQL). Then you configure them to work > together. Then you start them up in a particular order. It's > coordinated communication independent functions. If one blows up, the > others keep working. > > Imagine every part of your program working like this and you have an > Erlang application - actually, an Erlang *system*. > > To replicate this model at the OS level, you'd write dozens of small, > independent applications that communicated with each other over pipes > or sockets. The ZeroMQ community is familiar with this approach. > > The payoff for this, as I see it, is flexibility and speed of > introducing new functionality and fixing bugs. > > To illustrate at a higher level, imagine a smart phone today that > didn't provide isolation across applications. What would you expect > from it? I'd expect it to not work unless the applications were all > nearly perfect. That means it will either not work, or the app > ecosystem would be very limited. But today, smart phones all have > kernels and user space where apps are isolated from one another. So I > can install some random thing from an app store and have a pretty high > confidence that it's not going to ruin my phone. The result is *huge* > app ecosystems with phones that pretty much work (shockingly well for > what they're asked to do). > > When you use Erlang, your program becomes this ecosystem of "apps" > that you can add to, modify, remove very freely without concern for > the whole system. Your program will be evolvable *and stable* in the > same way your phone is evolvable and stable. It's a scalable > programming model! > > It's truly fantastic - and seldom mentioned. (Folks here have heard > this line from me before - this is my particular drum that I like to > beat on :) > > Use Erlang! > > Garrett > > P.S. I routinely run into critical memory problems in Java apps that > host ecosystems of other "apps" (plugins). It's a completely > unsolvable problem when your language/VM encourages intractable memory > graphs across threads unless you are incredibly careful about what you > run. If you want to make a system pluggable in Java (in one JVM), be > prepared for it to stop working at some point. So your either limited > in what you can do (and how fast) or in the stability of your program. > > On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen > wrote: >> I think your observation is correct. >> >> An Erlang program works by having many small processes, all isolated from >> each other. The way to communicate between processes is to send a message, >> asynchronously. This in turn leads to the key observation: when you send >> messages, you don't care about *where* the other process is. It could be >> local or on a completely different machine. The syntax and the semantics are >> the same, and you would program the system much in the same way. The >> environment is thus very homogeneous, compared to other solutions where you >> need to communicate on two levels: one for local messaging and one for >> distributed messaging. >> >> I also second Bob's observation: The design feature of being functional >> forces a lot of properties which are beneficial to programs where >> correctness matters more than squeezing out the last ounces of performance >> from a tight computational kernel. But there is more to it than that. A good >> example is the choice of standard data structures which have no pathological >> problems in corner cases. Or the deep continued focus on scaling to multiple >> cores rather than looking for efficient single-core performance. >> >> >>> On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: >>> >>> I'd agree with that observation. Erlang is particularly well designed for >>> reliability and ease of maintenance/debugging. I wouldn't necessarily say >>> that these properties are due to the language, it's really the environments >>> that Erlang has been deployed in that shaped the VM and libraries in this >>> way. The tooling and libraries have at least a decade head start for this >>> kind of industrial usage over just about any other functional language. >>> >>>> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne wrote: >>>> >>>> I've been investigating functional languages and the concepts that lead >>>> to increased speed, reliability, and decreased maintenance. Erlang seems to >>>> have a distinct advantage over other functional languages when you need to >>>> scale across multiple servers because it's a natural part of the language. >>>> Can anyone confirm/deny or elaborate on the observation? >>>> >>>> Without wax, >>>> Ken Wayne >>>> kwayne@REDACTED >>>> Desk: 715.261.9412 >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sergej.jurecko@REDACTED Fri Jan 16 19:40:05 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Fri, 16 Jan 2015 19:40:05 +0100 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: <14C92C82-610B-42DD-A697-B3203ABBC3B6@gmail.com> Don?t go using technology just because you think you should be. When building web apps, you usually don't need to spawn processes and gen_servers. Doing so would just makes things unnecessarily complex. Sergej On 16 Jan 2015, at 19:31, Lloyd R. Prentice wrote: > Ah Professor Smith, > > Once again I must confess my ignorance in hope that you can bring clarity. I've been working happily with sequential Erlang now for several years--- most recently an ambitious Nitrogen app. And I've heard from the beginning the virtues of of distributed concurrent processes happily chattering among themselves. > > But where the rubber meets the road, e.g. as I design my current app, I just don't get how to transform these virtues from virtual to real. > > Conceptually, I understand spawned processes, message passing, and supervisors. I've read and reread the Erlang canon. I've built several gen-servers and have worked my way through your e2 tutorial. It all makes good sense. > > But as I consider these principles in context of the architectural design of my current app, it's not at all clear how to apply them. > > The goal of my webapp is to deliver a set of data-based "tools" to "users" (wouldn't it be nice if we had many of these elusive critters) where each user owns his/her own data. > > In my case, users are non-technical author-publishers where each has a personal page for accessing tools and managing data related to project, marketing, and business management. > > Best I can tell, Cowboy and Nitrogen deal with "connections" and "sessions" under the hood, so I don't have to worry about them. But I struggle with such questions as: > > -- Should each "tool" be implemented as a gen-server? > -- Should the user interface and database be further factored as separate processes? > -- Should each "tool" be developed as a separate Erlang application then integrated as dependencies of a higher-level portal? Or should they be developed as a set of modules in one application? > -- Or maybe each tool should be a totally independent "microservice," whatever that means. > > In other words, my attentive studies of Erlang have left me with very few PRACTICAL architectural techniques and tools; that is an insufficient bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of building robust Erlang systems. > > I'd much appreciate any guidelines to help me thrash through my confusion. > > But more, I wonder if others struggle with the same issues? And if so, how can I work with wizards like you to shed light on this corner of Erlang technology? > > All the best, > > Lloyd > > Sent from my iPad > >> On Jan 16, 2015, at 10:03 AM, Garrett Smith wrote: >> >> I don't think Erlang has an edge over any other language in terms of >> scaling across multiple servers - not at all. Other functional >> languages have access to network APIs the same way Erlang does. >> >> I suppose it's less fiddly to send Erlang terms across the wire. >> Erlang has some built in facilities for building distributed >> applications. Erlang's been doing this sort of thing for a long time. >> But other languages can do it - particularly with the many >> multi-language messaging libraries and tools available today. >> >> What Erlang is special for IMO is it concurrency model, which is >> implemented in and enforced by the *VM* - there's no shared memory >> across threads of execution. One thread cannot corrupt the memory used >> by another. To communicate between threads of execution, threads must >> pass and receive messages (copied data). >> >> I use the word threads generically here - in Erlang they're called processes. >> >> This changes everything. It's completely transformative of the way you >> build software. But the real payoff is in your locally running program >> and less so in the ability to "distribute". >> >> Try building software using single threaded, isolated processes (no >> shared memory). How do you do it? If you use Erlang, you're forced to >> do it - there's no choice. >> >> It's the same as the operating system level. How do you build a LAMP >> stack? (sorry, I'm older then 24) You install independent components >> (Apache, PHP (fork exec'd), MySQL). Then you configure them to work >> together. Then you start them up in a particular order. It's >> coordinated communication independent functions. If one blows up, the >> others keep working. >> >> Imagine every part of your program working like this and you have an >> Erlang application - actually, an Erlang *system*. >> >> To replicate this model at the OS level, you'd write dozens of small, >> independent applications that communicated with each other over pipes >> or sockets. The ZeroMQ community is familiar with this approach. >> >> The payoff for this, as I see it, is flexibility and speed of >> introducing new functionality and fixing bugs. >> >> To illustrate at a higher level, imagine a smart phone today that >> didn't provide isolation across applications. What would you expect >> from it? I'd expect it to not work unless the applications were all >> nearly perfect. That means it will either not work, or the app >> ecosystem would be very limited. But today, smart phones all have >> kernels and user space where apps are isolated from one another. So I >> can install some random thing from an app store and have a pretty high >> confidence that it's not going to ruin my phone. The result is *huge* >> app ecosystems with phones that pretty much work (shockingly well for >> what they're asked to do). >> >> When you use Erlang, your program becomes this ecosystem of "apps" >> that you can add to, modify, remove very freely without concern for >> the whole system. Your program will be evolvable *and stable* in the >> same way your phone is evolvable and stable. It's a scalable >> programming model! >> >> It's truly fantastic - and seldom mentioned. (Folks here have heard >> this line from me before - this is my particular drum that I like to >> beat on :) >> >> Use Erlang! >> >> Garrett >> >> P.S. I routinely run into critical memory problems in Java apps that >> host ecosystems of other "apps" (plugins). It's a completely >> unsolvable problem when your language/VM encourages intractable memory >> graphs across threads unless you are incredibly careful about what you >> run. If you want to make a system pluggable in Java (in one JVM), be >> prepared for it to stop working at some point. So your either limited >> in what you can do (and how fast) or in the stability of your program. >> >> On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen >> wrote: >>> I think your observation is correct. >>> >>> An Erlang program works by having many small processes, all isolated from >>> each other. The way to communicate between processes is to send a message, >>> asynchronously. This in turn leads to the key observation: when you send >>> messages, you don't care about *where* the other process is. It could be >>> local or on a completely different machine. The syntax and the semantics are >>> the same, and you would program the system much in the same way. The >>> environment is thus very homogeneous, compared to other solutions where you >>> need to communicate on two levels: one for local messaging and one for >>> distributed messaging. >>> >>> I also second Bob's observation: The design feature of being functional >>> forces a lot of properties which are beneficial to programs where >>> correctness matters more than squeezing out the last ounces of performance >>> from a tight computational kernel. But there is more to it than that. A good >>> example is the choice of standard data structures which have no pathological >>> problems in corner cases. Or the deep continued focus on scaling to multiple >>> cores rather than looking for efficient single-core performance. >>> >>> >>>> On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: >>>> >>>> I'd agree with that observation. Erlang is particularly well designed for >>>> reliability and ease of maintenance/debugging. I wouldn't necessarily say >>>> that these properties are due to the language, it's really the environments >>>> that Erlang has been deployed in that shaped the VM and libraries in this >>>> way. The tooling and libraries have at least a decade head start for this >>>> kind of industrial usage over just about any other functional language. >>>> >>>>> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne wrote: >>>>> >>>>> I've been investigating functional languages and the concepts that lead >>>>> to increased speed, reliability, and decreased maintenance. Erlang seems to >>>>> have a distinct advantage over other functional languages when you need to >>>>> scale across multiple servers because it's a natural part of the language. >>>>> Can anyone confirm/deny or elaborate on the observation? >>>>> >>>>> Without wax, >>>>> Ken Wayne >>>>> kwayne@REDACTED >>>>> Desk: 715.261.9412 >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Fri Jan 16 19:47:20 2015 From: zxq9@REDACTED (zxq9) Date: Sat, 17 Jan 2015 03:47:20 +0900 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: <4020083.AdbBK9Fxph@changa> On 2015?1?16? ??? 13:31:55 Lloyd R. Prentice wrote: > Ah Professor Smith, > > Once again I must confess my ignorance in hope that you can bring clarity. > I've been working happily with sequential Erlang now for several years--- > most recently an ambitious Nitrogen app. And I've heard from the beginning > the virtues of of distributed concurrent processes happily chattering among > themselves. > > ... I don't believe the typical web application is generally written in such a way that the parts where many processes chatter away at each other is really apparent to the author of the web-facing bits. Some things that go on with websockets, on the other hand, are more like real applications, and there you certainly (may) get this, but not with serving web pages. There is a lot of context missing from this complaint (if that is even what this is). Which is why I think a more reasonable illustration of the concurrency issue is in something like a game server backend, or even a locally process-prolific pub-sub reliant desktop business application that pushes data updates as message flows which eventually hit the GUI bits... or whatever. But not serving web pages. From bob@REDACTED Fri Jan 16 20:04:20 2015 From: bob@REDACTED (Bob Ippolito) Date: Sat, 17 Jan 2015 08:04:20 +1300 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: Build a specific "tool" and then build another one. Repeat until finished, making sure to spend some time cleaning things up and generalizing along the way. Don't ask for advice until after you've started doing something. Make sure to provide precise information for what it needs to do, and include as much code as possible, when you ask. You don't know enough about problems you're not yet solving to ask the right questions. On Sat, Jan 17, 2015 at 7:31 AM, Lloyd R. Prentice wrote: > Ah Professor Smith, > > Once again I must confess my ignorance in hope that you can bring clarity. > I've been working happily with sequential Erlang now for several years--- > most recently an ambitious Nitrogen app. And I've heard from the beginning > the virtues of of distributed concurrent processes happily chattering among > themselves. > > But where the rubber meets the road, e.g. as I design my current app, I > just don't get how to transform these virtues from virtual to real. > > Conceptually, I understand spawned processes, message passing, and > supervisors. I've read and reread the Erlang canon. I've built several > gen-servers and have worked my way through your e2 tutorial. It all makes > good sense. > > But as I consider these principles in context of the architectural design > of my current app, it's not at all clear how to apply them. > > The goal of my webapp is to deliver a set of data-based "tools" to "users" > (wouldn't it be nice if we had many of these elusive critters) where each > user owns his/her own data. > > In my case, users are non-technical author-publishers where each has a > personal page for accessing tools and managing data related to project, > marketing, and business management. > > Best I can tell, Cowboy and Nitrogen deal with "connections" and > "sessions" under the hood, so I don't have to worry about them. But I > struggle with such questions as: > > -- Should each "tool" be implemented as a gen-server? > -- Should the user interface and database be further factored as separate > processes? > -- Should each "tool" be developed as a separate Erlang application then > integrated as dependencies of a higher-level portal? Or should they be > developed as a set of modules in one application? > -- Or maybe each tool should be a totally independent "microservice," > whatever that means. > > In other words, my attentive studies of Erlang have left me with very few > PRACTICAL architectural techniques and tools; that is an insufficient > bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of > building robust Erlang systems. > > I'd much appreciate any guidelines to help me thrash through my confusion. > > But more, I wonder if others struggle with the same issues? And if so, how > can I work with wizards like you to shed light on this corner of Erlang > technology? > > All the best, > > Lloyd > > Sent from my iPad > > > On Jan 16, 2015, at 10:03 AM, Garrett Smith wrote: > > > > I don't think Erlang has an edge over any other language in terms of > > scaling across multiple servers - not at all. Other functional > > languages have access to network APIs the same way Erlang does. > > > > I suppose it's less fiddly to send Erlang terms across the wire. > > Erlang has some built in facilities for building distributed > > applications. Erlang's been doing this sort of thing for a long time. > > But other languages can do it - particularly with the many > > multi-language messaging libraries and tools available today. > > > > What Erlang is special for IMO is it concurrency model, which is > > implemented in and enforced by the *VM* - there's no shared memory > > across threads of execution. One thread cannot corrupt the memory used > > by another. To communicate between threads of execution, threads must > > pass and receive messages (copied data). > > > > I use the word threads generically here - in Erlang they're called > processes. > > > > This changes everything. It's completely transformative of the way you > > build software. But the real payoff is in your locally running program > > and less so in the ability to "distribute". > > > > Try building software using single threaded, isolated processes (no > > shared memory). How do you do it? If you use Erlang, you're forced to > > do it - there's no choice. > > > > It's the same as the operating system level. How do you build a LAMP > > stack? (sorry, I'm older then 24) You install independent components > > (Apache, PHP (fork exec'd), MySQL). Then you configure them to work > > together. Then you start them up in a particular order. It's > > coordinated communication independent functions. If one blows up, the > > others keep working. > > > > Imagine every part of your program working like this and you have an > > Erlang application - actually, an Erlang *system*. > > > > To replicate this model at the OS level, you'd write dozens of small, > > independent applications that communicated with each other over pipes > > or sockets. The ZeroMQ community is familiar with this approach. > > > > The payoff for this, as I see it, is flexibility and speed of > > introducing new functionality and fixing bugs. > > > > To illustrate at a higher level, imagine a smart phone today that > > didn't provide isolation across applications. What would you expect > > from it? I'd expect it to not work unless the applications were all > > nearly perfect. That means it will either not work, or the app > > ecosystem would be very limited. But today, smart phones all have > > kernels and user space where apps are isolated from one another. So I > > can install some random thing from an app store and have a pretty high > > confidence that it's not going to ruin my phone. The result is *huge* > > app ecosystems with phones that pretty much work (shockingly well for > > what they're asked to do). > > > > When you use Erlang, your program becomes this ecosystem of "apps" > > that you can add to, modify, remove very freely without concern for > > the whole system. Your program will be evolvable *and stable* in the > > same way your phone is evolvable and stable. It's a scalable > > programming model! > > > > It's truly fantastic - and seldom mentioned. (Folks here have heard > > this line from me before - this is my particular drum that I like to > > beat on :) > > > > Use Erlang! > > > > Garrett > > > > P.S. I routinely run into critical memory problems in Java apps that > > host ecosystems of other "apps" (plugins). It's a completely > > unsolvable problem when your language/VM encourages intractable memory > > graphs across threads unless you are incredibly careful about what you > > run. If you want to make a system pluggable in Java (in one JVM), be > > prepared for it to stop working at some point. So your either limited > > in what you can do (and how fast) or in the stability of your program. > > > > On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen > > wrote: > >> I think your observation is correct. > >> > >> An Erlang program works by having many small processes, all isolated > from > >> each other. The way to communicate between processes is to send a > message, > >> asynchronously. This in turn leads to the key observation: when you send > >> messages, you don't care about *where* the other process is. It could be > >> local or on a completely different machine. The syntax and the > semantics are > >> the same, and you would program the system much in the same way. The > >> environment is thus very homogeneous, compared to other solutions where > you > >> need to communicate on two levels: one for local messaging and one for > >> distributed messaging. > >> > >> I also second Bob's observation: The design feature of being functional > >> forces a lot of properties which are beneficial to programs where > >> correctness matters more than squeezing out the last ounces of > performance > >> from a tight computational kernel. But there is more to it than that. A > good > >> example is the choice of standard data structures which have no > pathological > >> problems in corner cases. Or the deep continued focus on scaling to > multiple > >> cores rather than looking for efficient single-core performance. > >> > >> > >>> On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: > >>> > >>> I'd agree with that observation. Erlang is particularly well designed > for > >>> reliability and ease of maintenance/debugging. I wouldn't necessarily > say > >>> that these properties are due to the language, it's really the > environments > >>> that Erlang has been deployed in that shaped the VM and libraries in > this > >>> way. The tooling and libraries have at least a decade head start for > this > >>> kind of industrial usage over just about any other functional language. > >>> > >>>> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne > wrote: > >>>> > >>>> I've been investigating functional languages and the concepts that > lead > >>>> to increased speed, reliability, and decreased maintenance. Erlang > seems to > >>>> have a distinct advantage over other functional languages when you > need to > >>>> scale across multiple servers because it's a natural part of the > language. > >>>> Can anyone confirm/deny or elaborate on the observation? > >>>> > >>>> Without wax, > >>>> Ken Wayne > >>>> kwayne@REDACTED > >>>> Desk: 715.261.9412 > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Jan 16 20:28:17 2015 From: g@REDACTED (Garrett Smith) Date: Fri, 16 Jan 2015 13:28:17 -0600 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: Web apps naturally federate activities into separate threads - each one handling a request. You already have a lot isolation in effect off the bat - any HTTP request should be able to (directly) impact another. The material benefit is that your app can handle a *lot* of concurrent access without becoming corrupt. You can even be sloppy and still probably get away with it. This is *not* true of say a Java app - if you don't pay very close attention to the way memory is allocated and accessed, a long running app like a web server will fall over eventually. This btw is why you still see advocates of CGI style web apps (fork exec). They introduce substantial overhead by using separate OS processes but they're *stable* in the face of load, sloppiness, bugs, unexpected events. Give that web apps are trivial to scale horizontally, people who prefer their software to run without falling over will opt for the extra cost per HTTP request. That part's easy - you're already getting it. A win! As for the rest, you're also already getting it :) You have no choice here. Sorry. You cannot create the ball-of-twine that often emerges from a monolithic app managing a single shared heap. Another win! As for the rest... just solve the problems that you have! You have expressed angst and wonder and hope. These are all good and important topics but they're not problems to your app. What's the next thing you need to do for your app? Provide a specific problem and it will be easier to elaborate some options - and then pick the one that is the most sensible. I think one of the reasons Erlangers don't talk a lot about architecture is that process oriented apps can evolve very happily without much forethought. That's true of systems. As much as we like to fancy ourselves as architects and designers, good systems evolve incrementally. If you have the right underlying abstractions to support evolution, you'll get that without trying. I believe Erlang provides those abstractions: - Isolated processes - Links (enable supervision and a cascading process death) - OTP managed processes (servers/services and supervisors) - OTP apps (system startup and upgrades) Okay, that's all high level and more of the stuff you've heard and I know it doesn't answer your questions :) Starting with your app, each HTTP request handler is running in a separate process. That's how they run concurrently. Each handler will want to do something: serve a page, read from a database, write to a database, etc. You ought to provide a nice API for each of the things these handlers do. The API will be implemented via functions that live in a module. You have no choice. The decision then becomes how are these functions implemented. Do they just crank through sequential Erlang without sending or receiving a single message? That's your best bet if you can swing it - a nice side effect free function! A good example of that would be serving a web template. Using a great library like Erlydtl you'd just call your compiled template module - it will render the data and off it goes to the client. There's no gen_server here. It's just each HTTP request handler doing work alongside the other. Super scalable. Super simple. But now accessing a database... This depends on the database library you're using and whether or not its API supports concurrent access. I imagine most would - but via what? There'll be something that is shared here - a connection, a pool, a registered process. You should understand that that thing is and then consider how all your HTTP request handlers will access it. If you're sharing a single connection, for example, that can only manage a single transaction at a time, obviously you don't want each request piling onto the same transaction. In that case you'll be either serializing access to this connection (you'd use a gen_server for that - or an e2 service) or use multiple connections via a connection pool (which also requires serialized access, again via a gen_server/e2 service). But really, if you get this wrong, you're going to run smack dab into a very concrete problem that you *must* solve to move forward. You don't really need to design this stuff. Once you run into a few cases of these problems, you're going recognize them and know how to fix them. I *think* it comes down to this: do you need to serialize access to something? If yes, your API sits in front of a process, which implements the functionality safely in the server loop (no concurrent access). If no, your API implements the functionality directly. More... Use OTP (or e2) and run your processes under supervision. If your processes are short lived, use a simple-one-for-one supervisor (e2 task supervisor + tasks) to start them. Don't solve problems you don't have. In particular, don't add or use something unless you have to. That goes for OTP apps. Just use one. Don't implement functionality behind a gen_server unless you know why you're doing that (e.g. to strictly control access to something). Don't use a separate db for each customer unless you know why you need that. Etc. I don't know, I'm running out of ideas here :) On Fri, Jan 16, 2015 at 12:31 PM, Lloyd R. Prentice wrote: > Ah Professor Smith, > > Once again I must confess my ignorance in hope that you can bring clarity. > I've been working happily with sequential Erlang now for several years--- > most recently an ambitious Nitrogen app. And I've heard from the beginning > the virtues of of distributed concurrent processes happily chattering among > themselves. > > But where the rubber meets the road, e.g. as I design my current app, I > just don't get how to transform these virtues from virtual to real. > > Conceptually, I understand spawned processes, message passing, and > supervisors. I've read and reread the Erlang canon. I've built several > gen-servers and have worked my way through your e2 tutorial. It all makes > good sense. > > But as I consider these principles in context of the architectural design > of my current app, it's not at all clear how to apply them. > > The goal of my webapp is to deliver a set of data-based "tools" to "users" > (wouldn't it be nice if we had many of these elusive critters) where each > user owns his/her own data. > > In my case, users are non-technical author-publishers where each has a > personal page for accessing tools and managing data related to project, > marketing, and business management. > > Best I can tell, Cowboy and Nitrogen deal with "connections" and > "sessions" under the hood, so I don't have to worry about them. But I > struggle with such questions as: > > -- Should each "tool" be implemented as a gen-server? > -- Should the user interface and database be further factored as separate > processes? > -- Should each "tool" be developed as a separate Erlang application then > integrated as dependencies of a higher-level portal? Or should they be > developed as a set of modules in one application? > -- Or maybe each tool should be a totally independent "microservice," > whatever that means. > > In other words, my attentive studies of Erlang have left me with very few > PRACTICAL architectural techniques and tools; that is an insufficient > bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of > building robust Erlang systems. > > I'd much appreciate any guidelines to help me thrash through my confusion. > > But more, I wonder if others struggle with the same issues? And if so, how > can I work with wizards like you to shed light on this corner of Erlang > technology? > > All the best, > > Lloyd > > Sent from my iPad > > > On Jan 16, 2015, at 10:03 AM, Garrett Smith wrote: > > > > I don't think Erlang has an edge over any other language in terms of > > scaling across multiple servers - not at all. Other functional > > languages have access to network APIs the same way Erlang does. > > > > I suppose it's less fiddly to send Erlang terms across the wire. > > Erlang has some built in facilities for building distributed > > applications. Erlang's been doing this sort of thing for a long time. > > But other languages can do it - particularly with the many > > multi-language messaging libraries and tools available today. > > > > What Erlang is special for IMO is it concurrency model, which is > > implemented in and enforced by the *VM* - there's no shared memory > > across threads of execution. One thread cannot corrupt the memory used > > by another. To communicate between threads of execution, threads must > > pass and receive messages (copied data). > > > > I use the word threads generically here - in Erlang they're called > processes. > > > > This changes everything. It's completely transformative of the way you > > build software. But the real payoff is in your locally running program > > and less so in the ability to "distribute". > > > > Try building software using single threaded, isolated processes (no > > shared memory). How do you do it? If you use Erlang, you're forced to > > do it - there's no choice. > > > > It's the same as the operating system level. How do you build a LAMP > > stack? (sorry, I'm older then 24) You install independent components > > (Apache, PHP (fork exec'd), MySQL). Then you configure them to work > > together. Then you start them up in a particular order. It's > > coordinated communication independent functions. If one blows up, the > > others keep working. > > > > Imagine every part of your program working like this and you have an > > Erlang application - actually, an Erlang *system*. > > > > To replicate this model at the OS level, you'd write dozens of small, > > independent applications that communicated with each other over pipes > > or sockets. The ZeroMQ community is familiar with this approach. > > > > The payoff for this, as I see it, is flexibility and speed of > > introducing new functionality and fixing bugs. > > > > To illustrate at a higher level, imagine a smart phone today that > > didn't provide isolation across applications. What would you expect > > from it? I'd expect it to not work unless the applications were all > > nearly perfect. That means it will either not work, or the app > > ecosystem would be very limited. But today, smart phones all have > > kernels and user space where apps are isolated from one another. So I > > can install some random thing from an app store and have a pretty high > > confidence that it's not going to ruin my phone. The result is *huge* > > app ecosystems with phones that pretty much work (shockingly well for > > what they're asked to do). > > > > When you use Erlang, your program becomes this ecosystem of "apps" > > that you can add to, modify, remove very freely without concern for > > the whole system. Your program will be evolvable *and stable* in the > > same way your phone is evolvable and stable. It's a scalable > > programming model! > > > > It's truly fantastic - and seldom mentioned. (Folks here have heard > > this line from me before - this is my particular drum that I like to > > beat on :) > > > > Use Erlang! > > > > Garrett > > > > P.S. I routinely run into critical memory problems in Java apps that > > host ecosystems of other "apps" (plugins). It's a completely > > unsolvable problem when your language/VM encourages intractable memory > > graphs across threads unless you are incredibly careful about what you > > run. If you want to make a system pluggable in Java (in one JVM), be > > prepared for it to stop working at some point. So your either limited > > in what you can do (and how fast) or in the stability of your program. > > > > On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen > > wrote: > >> I think your observation is correct. > >> > >> An Erlang program works by having many small processes, all isolated > from > >> each other. The way to communicate between processes is to send a > message, > >> asynchronously. This in turn leads to the key observation: when you send > >> messages, you don't care about *where* the other process is. It could be > >> local or on a completely different machine. The syntax and the > semantics are > >> the same, and you would program the system much in the same way. The > >> environment is thus very homogeneous, compared to other solutions where > you > >> need to communicate on two levels: one for local messaging and one for > >> distributed messaging. > >> > >> I also second Bob's observation: The design feature of being functional > >> forces a lot of properties which are beneficial to programs where > >> correctness matters more than squeezing out the last ounces of > performance > >> from a tight computational kernel. But there is more to it than that. A > good > >> example is the choice of standard data structures which have no > pathological > >> problems in corner cases. Or the deep continued focus on scaling to > multiple > >> cores rather than looking for efficient single-core performance. > >> > >> > >>> On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito wrote: > >>> > >>> I'd agree with that observation. Erlang is particularly well designed > for > >>> reliability and ease of maintenance/debugging. I wouldn't necessarily > say > >>> that these properties are due to the language, it's really the > environments > >>> that Erlang has been deployed in that shaped the VM and libraries in > this > >>> way. The tooling and libraries have at least a decade head start for > this > >>> kind of industrial usage over just about any other functional language. > >>> > >>>> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne > wrote: > >>>> > >>>> I've been investigating functional languages and the concepts that > lead > >>>> to increased speed, reliability, and decreased maintenance. Erlang > seems to > >>>> have a distinct advantage over other functional languages when you > need to > >>>> scale across multiple servers because it's a natural part of the > language. > >>>> Can anyone confirm/deny or elaborate on the observation? > >>>> > >>>> Without wax, > >>>> Ken Wayne > >>>> kwayne@REDACTED > >>>> Desk: 715.261.9412 > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Jan 16 20:31:39 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 16 Jan 2015 14:31:39 -0500 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: <20150116193138.GA8831@ferdair.local> On 01/16, Lloyd R. Prentice wrote: > But I struggle with such questions as: > > -- Should each "tool" be implemented as a gen-server? > -- Should the user interface and database be further factored as separate processes? > -- Should each "tool" be developed as a separate Erlang application then integrated as dependencies of a higher-level portal? Or should they be developed as a set of modules in one application? > -- Or maybe each tool should be a totally independent "microservice," whatever that means. > > In other words, my attentive studies of Erlang have left me with very few PRACTICAL architectural techniques and tools; that is an insufficient bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of building robust Erlang systems. > > I'd much appreciate any guidelines to help me thrash through my confusion. > > But more, I wonder if others struggle with the same issues? And if so, how can I work with wizards like you to shed light on this corner of Erlang technology? > I'd call this the 'zen of Erlang', or just "thinking in Erlang". After a bit of time, the separation of these things starts coming naturally. The initial level of confusion with Erlang and OTP, I think, drains its power in the same confusion as design patterns in OO. You are presented a bunch of tools and told "here, those tools are great, learn them all." And then you go on your way to solve problems. And every problem you meet, you think "which of my tools is the most appropriate for this?" It's possible you look at some tool and go "well, many of these seem good! What do I do?" The problem there is that it's going about it the wrong way. Rather than seeing a problem and saying "which hammer do I use to smash this?" we should ask the questions: - How do I want this to fail? Does it need to take anything down with it? How will it come back up? What's my road to recovery? - What is going to fail? The only correct answer is 'everything, sooner or later'. Specifically, things you haven't planned for will crash earlier (because you haven't taken the time and attention to plan for them). - The follow-up question is then: how do I make sure I know why something failed as clearly as possible? You want to crash early and often, but also as close to the source as possible. - How do I make sure crashes are as limited in their damage as possible? The idea is always that you want the damage to be as minimal as possible, but you also want *uncertainty* to be as restrained as possible. Juggling both ideas mean you will often want to kill related things together because you don't want (or haven't taken the time) to think about what happens when things that depend on each other fail partially. It's often better to kill them at once. - Who will own and modify that code, and how often will it change? It's often a lot easier to throw out code and rewrite it than to update it (research says at 25% rewrite, you're better starting from scratch -- http://mononcqc.tumblr.com/post/31767374324/an-analysis-of-errors-in-a-reuse-oriented-development-en) - How will you deploy that code? This is particularly useful if you plan on doing hot code loading. Hot code loads will go wrong from time to time and reducing the scope of which components you'll impact may be a good way to think about it. Note that none of this is related to Erlang *at all*. It's related to thinking about your problems, your users, and your team. This is what really has a great impact on the quality of the final solution. The technology you use is a ladder. Some are better than others, some can do wonders with a rope, but an elevator is best. In any case, the tech helps you, but you have to use it right. Once you have thought about this, you will already have made a fairly good idea of what components should be independent, what kind of separation you want to have in them, how they depend on each other. You've done the hardest of the design. Once this is all done, then it's easier to go "gen_server manages resource", "gen_fsm implements protocol", "gen_event forwards messages or reacts to events". You'll probably agree that this doesn't help you much in your questions. The reason is that what to pick and how to subdivide your system is a design question, not an Erlang questions. Links, monitors, isolation, behaviours, and so on are all building blocks, but you're in charge of assembling them in a cohesive manner. Understanding what is needed for your problem to be solved is what will make or break your Erlang design, and it's difficult to apply recipes wholesale. Or at least, I have found it hard so far in my career. Regards, Fred. From mononcqc@REDACTED Fri Jan 16 20:36:47 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 16 Jan 2015 14:36:47 -0500 Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: <20150116193138.GA8831@ferdair.local> References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> <20150116193138.GA8831@ferdair.local> Message-ID: <20150116193644.GB8831@ferdair.local> On 01/16, Fred Hebert wrote: > I'd call this the 'zen of Erlang', or just "thinking in Erlang". After a > bit of time, the separation of these things starts coming naturally. > Oh, and a final word. You truly get the Zen of Erlang once you ask all these questions and go "I absolutely have no idea what to do" and say "it's okay. I'll make a best effort, put it under a supervision tree, and see how it goes". If you've at the very least split a few things up so they don't corrupt state when they die, and that they can recover quickly, you'll be ready to handle nearly any kind of failure. You can then let your system run happily, and handle new unexpected errors as they pop up, gradually making your system better and better in the face of the unexpected. Good modular and functional abstraction will help there, as you shift parts of your implementation across different processes and subtrees. It all makes managing server systems extremely pleasant and natural over time. From flw@REDACTED Fri Jan 16 20:31:40 2015 From: flw@REDACTED (Florian Waas) Date: Fri, 16 Jan 2015 19:31:40 +0000 Subject: [erlang-questions] Differences in code_change callback between gen_server and gen_fsm In-Reply-To: References: Message-ID: Thanks Siri ? that?s exactly the kind of subtlety I was looking to find out about! Sounds like changing this is not a good idea; so, let?s just stick with the current spec/behavior. Thanks, -fl. From: Siri Hansen Date: Friday, January 16, 2015 at 3:07 AM To: Florian Waas Cc: "erlang-questions@REDACTED" Subject: Re: [erlang-questions] Differences in code_change callback between gen_server and gen_fsm Hi Florian! The way gen_fsm (and gen_event) is specified is actually the original - i.e. the code_change should always succeed. The function spec in gen_server was updated in OTP R15B to allow an error return since this was required (and had always been used) by supervisor (https://github.com/erlang/otp/commit/038b9dd3a1f9bdd86cbb83bf3484ab1529d4fca2). The release_handler was updated at the same time (so even if the supervisor did use it before, the result would probably not have been too nice if it actually failed). Anyway... I guess there would be no harm in updating the spec in gen_fsm to reflect that an error return is allowed (even if it is not a good idea - it will fail the complete upgrade and roll back to the old release. A fact which would have to be stated in the reference manual.) I could write at ticket for this, but the chance that it would be prioritized within reasonable time is extremely small. A contribution would help, of course... When it comes to gen_event, the change would be a bit more complicated. Errors are not handled at all in the code as it is now. Regards /siri 2015-01-06 19:41 GMT+01:00 Florian Waas >: In gen_server the callback for code_change is spec?d as Module:code_change(OldVsn, State, Extra) -> {ok, NewState} | {error, Reason} in gen_fsm as Module:code_change(OldVsn, StateName, StateData, Extra) -> {ok, NextStateName, NewStateData} i.e., upgrades on FSM?s had better succeed? The underlying implementation in sys.erl is the same for both and allows for an error case. Am I missing some subtlety here? Thanks, -fl. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ppraveenvlr@REDACTED Sat Jan 17 08:39:53 2015 From: ppraveenvlr@REDACTED (Praven p) Date: Sat, 17 Jan 2015 08:39:53 +0100 Subject: [erlang-questions] Hiring - result oriented Erlang developer Message-ID: Hi, Looking for an Erlang developer for an ambitious startup. Exposure/experience to Zotonic CMS preferred. We are building something in what we imagine to be in the Internet of People world. We are looking for an expert (experience years not important but result oriented) with high integrity and commitment. Please reach out should you like to discuss. best wishes / P -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb_duffy@REDACTED Sat Jan 17 12:24:08 2015 From: jb_duffy@REDACTED (John Duffy) Date: Sat, 17 Jan 2015 11:24:08 +0000 Subject: [erlang-questions] Does Inets HTTP client and PATCH method? Message-ID: <830B5642-695C-44C5-A5D4-5DDDED104520@btinternet.com> Hi My Foreign Exchange broker provides a HTTP REST API for automated trading, and uses the HTTP PATCH method for updating outstanding trade orders. I can't find a reference to the PATCH method in the httpc client documentation. Is this HTTP method supported? Thanks in advance John Duffy Sent from my iPhone From harit.subscriptions@REDACTED Sat Jan 17 20:23:33 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 17 Jan 2015 11:23:33 -0800 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: Message-ID: This is really dumb question and I am pretty sure that there is limit to how much distributed programming could be learnt using single machine(I am using Mac in this case). > > I am reading through Programming Erlang, Chapter 14, Distributed > Programming where Joe talk about how to run Name Server in distributed mode > as > > > 1. I write and test my program in a regular nondistributed Erlang > session. This is what we?ve been doing up to now, so it presents no new > challenges. > 2. I test my program on two different Erlang nodes running on the > same computer. > 3. I test my program on two different Erlang nodes running on two > physically separated computers either in the same local area network or > anywhere on the Internet. > > > So, I have reached to the part where #1 and #2 could be tested. > > I wanted to check there is absolutely no way to mimic different nodes as > separate machines (on a single node) to test #3. > > Thanks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmeiklejohn@REDACTED Sat Jan 17 20:46:17 2015 From: cmeiklejohn@REDACTED (Christopher Meiklejohn) Date: Sat, 17 Jan 2015 20:46:17 +0100 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: Message-ID: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> > On Jan 17, 2015, at 8:23 PM, Harit Himanshu wrote: > > > This is really dumb question and I am pretty sure that there is limit to how much distributed programming could be learnt using single machine(I am using Mac in this case). > > I am reading through Programming Erlang, Chapter 14, Distributed Programming where Joe talk about how to run Name Server in distributed mode as > > I write and test my program in a regular nondistributed Erlang session. This is what we?ve been doing up to now, so it presents no new challenges. > I test my program on two different Erlang nodes running on the same computer. > I test my program on two different Erlang nodes running on two physically separated computers either in the same local area network or anywhere on the Internet. You can get pretty far using all of the networking tools provided with your operating system to simulate various network conditions. I do all of my daily distributed programming and research using a stock MacBook. Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in several major distributed databases. There are also tools such as ?tc? which can be used to add arbitrary latency between processes. For what it?s worth, Basho does a significant amount of fault testing on a single machine by using fault-injection tools, or by facilities like ?intercepts? provided by Riak Test. My advice is this: do everything locally on your machine until you have a reason to move to separate infrastructure; it will be easier to develop and debug. - Chris Christopher Meiklejohn Senior Software Engineer Basho Technologies, Inc. cmeiklejohn@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sat Jan 17 20:46:25 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sat, 17 Jan 2015 14:46:25 -0500 (EST) Subject: [erlang-questions] Newbie - Erlang VS other functional languages In-Reply-To: References: <07EE4A22-E82E-4258-AE54-3941C9D77C0C@writersglen.com> Message-ID: <1421523985.014425312@apps.rackspace.com> Hello, Wow! Thanks all for the terrific insights. It will take me awhile to digest all this wisdom and apply to my current project. Over the next few days I'll try to organize, summarize, and re-post these ideas for benefit of noobies who follow. Best wishes to all, Lloyd -----Original Message----- From: "Garrett Smith" Sent: Friday, January 16, 2015 2:28pm To: "Lloyd R. Prentice" Cc: "Jesper Louis Andersen" , "erlang-questions" Subject: Re: [erlang-questions] Newbie - Erlang VS other functional languages Web apps naturally federate activities into separate threads - each one handling a request. You already have a lot isolation in effect off the bat - any HTTP request should be able to (directly) impact another. The material benefit is that your app can handle a *lot* of concurrent access without becoming corrupt. You can even be sloppy and still probably get away with it. This is *not* true of say a Java app - if you don't pay very close attention to the way memory is allocated and accessed, a long running app like a web server will fall over eventually. This btw is why you still see advocates of CGI style web apps (fork exec). They introduce substantial overhead by using separate OS processes but they're *stable* in the face of load, sloppiness, bugs, unexpected events. Give that web apps are trivial to scale horizontally, people who prefer their software to run without falling over will opt for the extra cost per HTTP request. That part's easy - you're already getting it. A win! As for the rest, you're also already getting it :) You have no choice here. Sorry. You cannot create the ball-of-twine that often emerges from a monolithic app managing a single shared heap. Another win! As for the rest... just solve the problems that you have! You have expressed angst and wonder and hope. These are all good and important topics but they're not problems to your app. What's the next thing you need to do for your app? Provide a specific problem and it will be easier to elaborate some options - and then pick the one that is the most sensible. I think one of the reasons Erlangers don't talk a lot about architecture is that process oriented apps can evolve very happily without much forethought. That's true of systems. As much as we like to fancy ourselves as architects and designers, good systems evolve incrementally. If you have the right underlying abstractions to support evolution, you'll get that without trying. I believe Erlang provides those abstractions: - Isolated processes - Links (enable supervision and a cascading process death) - OTP managed processes (servers/services and supervisors) - OTP apps (system startup and upgrades) Okay, that's all high level and more of the stuff you've heard and I know it doesn't answer your questions :) Starting with your app, each HTTP request handler is running in a separate process. That's how they run concurrently. Each handler will want to do something: serve a page, read from a database, write to a database, etc. You ought to provide a nice API for each of the things these handlers do. The API will be implemented via functions that live in a module. You have no choice. The decision then becomes how are these functions implemented. Do they just crank through sequential Erlang without sending or receiving a single message? That's your best bet if you can swing it - a nice side effect free function! A good example of that would be serving a web template. Using a great library like Erlydtl you'd just call your compiled template module - it will render the data and off it goes to the client. There's no gen_server here. It's just each HTTP request handler doing work alongside the other. Super scalable. Super simple. But now accessing a database... This depends on the database library you're using and whether or not its API supports concurrent access. I imagine most would - but via what? There'll be something that is shared here - a connection, a pool, a registered process. You should understand that that thing is and then consider how all your HTTP request handlers will access it. If you're sharing a single connection, for example, that can only manage a single transaction at a time, obviously you don't want each request piling onto the same transaction. In that case you'll be either serializing access to this connection (you'd use a gen_server for that - or an e2 service) or use multiple connections via a connection pool (which also requires serialized access, again via a gen_server/e2 service). But really, if you get this wrong, you're going to run smack dab into a very concrete problem that you *must* solve to move forward. You don't really need to design this stuff. Once you run into a few cases of these problems, you're going recognize them and know how to fix them. I *think* it comes down to this: do you need to serialize access to something? If yes, your API sits in front of a process, which implements the functionality safely in the server loop (no concurrent access). If no, your API implements the functionality directly. More... Use OTP (or e2) and run your processes under supervision. If your processes are short lived, use a simple-one-for-one supervisor (e2 task supervisor + tasks) to start them. Don't solve problems you don't have. In particular, don't add or use something unless you have to. That goes for OTP apps. Just use one. Don't implement functionality behind a gen_server unless you know why you're doing that (e.g. to strictly control access to something). Don't use a separate db for each customer unless you know why you need that. Etc. I don't know, I'm running out of ideas here :) On Fri, Jan 16, 2015 at 12:31 PM, Lloyd R. Prentice <[ lloyd@REDACTED ]( mailto:lloyd@REDACTED )> wrote: Ah Professor Smith, Once again I must confess my ignorance in hope that you can bring clarity. I've been working happily with sequential Erlang now for several years--- most recently an ambitious Nitrogen app. And I've heard from the beginning the virtues of of distributed concurrent processes happily chattering among themselves. But where the rubber meets the road, e.g. as I design my current app, I just don't get how to transform these virtues from virtual to real. Conceptually, I understand spawned processes, message passing, and supervisors. I've read and reread the Erlang canon. I've built several gen-servers and have worked my way through your e2 tutorial. It all makes good sense. But as I consider these principles in context of the architectural design of my current app, it's not at all clear how to apply them. The goal of my webapp is to deliver a set of data-based "tools" to "users" (wouldn't it be nice if we had many of these elusive critters) where each user owns his/her own data. In my case, users are non-technical author-publishers where each has a personal page for accessing tools and managing data related to project, marketing, and business management. Best I can tell, Cowboy and Nitrogen deal with "connections" and "sessions" under the hood, so I don't have to worry about them. But I struggle with such questions as: -- Should each "tool" be implemented as a gen-server? -- Should the user interface and database be further factored as separate processes? -- Should each "tool" be developed as a separate Erlang application then integrated as dependencies of a higher-level portal? Or should they be developed as a set of modules in one application? -- Or maybe each tool should be a totally independent "microservice," whatever that means. In other words, my attentive studies of Erlang have left me with very few PRACTICAL architectural techniques and tools; that is an insufficient bridge between the PRINCIPLES of concurrent Erlang and the PRACTICE of building robust Erlang systems. I'd much appreciate any guidelines to help me thrash through my confusion. But more, I wonder if others struggle with the same issues? And if so, how can I work with wizards like you to shed light on this corner of Erlang technology? All the best, Lloyd Sent from my iPad > On Jan 16, 2015, at 10:03 AM, Garrett Smith <[ g@REDACTED ]( mailto:g@REDACTED )> wrote: > > I don't think Erlang has an edge over any other language in terms of > scaling across multiple servers - not at all. Other functional > languages have access to network APIs the same way Erlang does. > > I suppose it's less fiddly to send Erlang terms across the wire. > Erlang has some built in facilities for building distributed > applications. Erlang's been doing this sort of thing for a long time. > But other languages can do it - particularly with the many > multi-language messaging libraries and tools available today. > > What Erlang is special for IMO is it concurrency model, which is > implemented in and enforced by the *VM* - there's no shared memory > across threads of execution. One thread cannot corrupt the memory used > by another. To communicate between threads of execution, threads must > pass and receive messages (copied data). > > I use the word threads generically here - in Erlang they're called processes. > > This changes everything. It's completely transformative of the way you > build software. But the real payoff is in your locally running program > and less so in the ability to "distribute". > > Try building software using single threaded, isolated processes (no > shared memory). How do you do it? If you use Erlang, you're forced to > do it - there's no choice. > > It's the same as the operating system level. How do you build a LAMP > stack? (sorry, I'm older then 24) You install independent components > (Apache, PHP (fork exec'd), MySQL). Then you configure them to work > together. Then you start them up in a particular order. It's > coordinated communication independent functions. If one blows up, the > others keep working. > > Imagine every part of your program working like this and you have an > Erlang application - actually, an Erlang *system*. > > To replicate this model at the OS level, you'd write dozens of small, > independent applications that communicated with each other over pipes > or sockets. The ZeroMQ community is familiar with this approach. > > The payoff for this, as I see it, is flexibility and speed of > introducing new functionality and fixing bugs. > > To illustrate at a higher level, imagine a smart phone today that > didn't provide isolation across applications. What would you expect > from it? I'd expect it to not work unless the applications were all > nearly perfect. That means it will either not work, or the app > ecosystem would be very limited. But today, smart phones all have > kernels and user space where apps are isolated from one another. So I > can install some random thing from an app store and have a pretty high > confidence that it's not going to ruin my phone. The result is *huge* > app ecosystems with phones that pretty much work (shockingly well for > what they're asked to do). > > When you use Erlang, your program becomes this ecosystem of "apps" > that you can add to, modify, remove very freely without concern for > the whole system. Your program will be evolvable *and stable* in the > same way your phone is evolvable and stable. It's a scalable > programming model! > > It's truly fantastic - and seldom mentioned. (Folks here have heard > this line from me before - this is my particular drum that I like to > beat on :) > > Use Erlang! > > Garrett > > P.S. I routinely run into critical memory problems in Java apps that > host ecosystems of other "apps" (plugins). It's a completely > unsolvable problem when your language/VM encourages intractable memory > graphs across threads unless you are incredibly careful about what you > run. If you want to make a system pluggable in Java (in one JVM), be > prepared for it to stop working at some point. So your either limited > in what you can do (and how fast) or in the stability of your program. > > On Fri, Jan 16, 2015 at 6:29 AM, Jesper Louis Andersen > <[ jesper.louis.andersen@REDACTED ]( mailto:jesper.louis.andersen@REDACTED )> wrote: >> I think your observation is correct. >> >> An Erlang program works by having many small processes, all isolated from >> each other. The way to communicate between processes is to send a message, >> asynchronously. This in turn leads to the key observation: when you send >> messages, you don't care about *where* the other process is. It could be >> local or on a completely different machine. The syntax and the semantics are >> the same, and you would program the system much in the same way. The >> environment is thus very homogeneous, compared to other solutions where you >> need to communicate on two levels: one for local messaging and one for >> distributed messaging. >> >> I also second Bob's observation: The design feature of being functional >> forces a lot of properties which are beneficial to programs where >> correctness matters more than squeezing out the last ounces of performance >> from a tight computational kernel. But there is more to it than that. A good >> example is the choice of standard data structures which have no pathological >> problems in corner cases. Or the deep continued focus on scaling to multiple >> cores rather than looking for efficient single-core performance. >> >> >>> On Thu Jan 15 2015 at 11:38:52 PM Bob Ippolito <[ bob@REDACTED ]( mailto:bob@REDACTED )> wrote: >>> >>> I'd agree with that observation. Erlang is particularly well designed for >>> reliability and ease of maintenance/debugging. I wouldn't necessarily say >>> that these properties are due to the language, it's really the environments >>> that Erlang has been deployed in that shaped the VM and libraries in this >>> way. The tooling and libraries have at least a decade head start for this >>> kind of industrial usage over just about any other functional language. >>> >>>> On Fri, Jan 16, 2015 at 9:01 AM, Ken Wayne <[ kwayne@REDACTED ]( mailto:kwayne@REDACTED )> wrote: >>>> >>>> I've been investigating functional languages and the concepts that lead >>>> to increased speed, reliability, and decreased maintenance. Erlang seems to >>>> have a distinct advantage over other functional languages when you need to >>>> scale across multiple servers because it's a natural part of the language. >>>> Can anyone confirm/deny or elaborate on the observation? >>>> >>>> Without wax, >>>> Ken Wayne >>>> [ kwayne@REDACTED ]( mailto:kwayne@REDACTED ) >>>> Desk: 715.261.9412 >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) >>>> [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) >>> [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) >> >> >> _______________________________________________ >> erlang-questions mailing list >> [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) >> [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) > _______________________________________________ > erlang-questions mailing list > [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) > [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Sat Jan 17 20:53:58 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sat, 17 Jan 2015 20:53:58 +0100 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: I have had 14 different machines running on my 2 year old MacBook Air (granted they where not doing a whole lot) by just using docker. Each docker instance has its own IP and name ect. Worked really well. Make sure that in each docker you run tmux so yo can check both the output and do things with the machine. What I do is put the Dockerfile in a sub folder of the project (because docker copies all things in the folder the Dockerfile is hosted in into the container and that is slow) and then map ../ to a mountpoint inside the docker container. If your host is the same as the docker container then you can build locally and just restart the process in the containers. -Mark On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn < cmeiklejohn@REDACTED> wrote: > > On Jan 17, 2015, at 8:23 PM, Harit Himanshu > wrote: > > > This is really dumb question and I am pretty sure that there is limit to > how much distributed programming could be learnt using single machine(I am > using Mac in this case). > >> >> I am reading through Programming Erlang, Chapter 14, Distributed >> Programming where Joe talk about how to run Name Server in distributed mode >> as >> >> >> 1. I write and test my program in a regular nondistributed Erlang >> session. This is what we?ve been doing up to now, so it presents no new >> challenges. >> 2. I test my program on two different Erlang nodes running on the >> same computer. >> 3. I test my program on two different Erlang nodes running on two >> physically separated computers either in the same local area network or >> anywhere on the Internet. >> >> You can get pretty far using all of the networking tools provided with > your operating system to simulate various network conditions. I do all of > my daily distributed programming and research using a stock MacBook. > > Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in > several major distributed databases. There are also tools such as ?tc? > which can be used to add arbitrary latency between processes. > > For what it?s worth, Basho does a significant amount of fault testing on a > single machine by using fault-injection tools, or by facilities like > ?intercepts? provided by Riak Test. > > My advice is this: do everything locally on your machine until you have a > reason to move to separate infrastructure; it will be easier to develop and > debug. > > - Chris > > Christopher Meiklejohn > Senior Software Engineer > Basho Technologies, Inc. > cmeiklejohn@REDACTED > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 17 21:05:00 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 17 Jan 2015 12:05:00 -0800 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: Wow! I am glad to know I am wrong because this opens up opportunities to do distributed programming on my Macbook :) Since I am very new to this, do you mind sharing any resources, links that I can read to learn about "How to do" them? Thanks a lot Christopher On Sat, Jan 17, 2015 at 11:46 AM, Christopher Meiklejohn < cmeiklejohn@REDACTED> wrote: > > On Jan 17, 2015, at 8:23 PM, Harit Himanshu > wrote: > > > This is really dumb question and I am pretty sure that there is limit to > how much distributed programming could be learnt using single machine(I am > using Mac in this case). > >> >> I am reading through Programming Erlang, Chapter 14, Distributed >> Programming where Joe talk about how to run Name Server in distributed mode >> as >> >> >> 1. I write and test my program in a regular nondistributed Erlang >> session. This is what we?ve been doing up to now, so it presents no new >> challenges. >> 2. I test my program on two different Erlang nodes running on the >> same computer. >> 3. I test my program on two different Erlang nodes running on two >> physically separated computers either in the same local area network or >> anywhere on the Internet. >> >> You can get pretty far using all of the networking tools provided with > your operating system to simulate various network conditions. I do all of > my daily distributed programming and research using a stock MacBook. > > Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in > several major distributed databases. There are also tools such as ?tc? > which can be used to add arbitrary latency between processes. > > For what it?s worth, Basho does a significant amount of fault testing on a > single machine by using fault-injection tools, or by facilities like > ?intercepts? provided by Riak Test. > > My advice is this: do everything locally on your machine until you have a > reason to move to separate infrastructure; it will be easier to develop and > debug. > > - Chris > > Christopher Meiklejohn > Senior Software Engineer > Basho Technologies, Inc. > cmeiklejohn@REDACTED > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 17 21:06:48 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 17 Jan 2015 12:06:48 -0800 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: Thanks Mark, this sounds like a very good approach to learn about distributed programming. Since I am new in this arena, do you mind sharing resources on how to achieve something you do? That ways I can get some direction on what to Google and take it forward Thanks a lot! + Harit Himanshu On Sat, Jan 17, 2015 at 11:53 AM, Mark Nijhof < mark.nijhof@REDACTED> wrote: > I have had 14 different machines running on my 2 year old MacBook Air > (granted they where not doing a whole lot) by just using docker. Each > docker instance has its own IP and name ect. Worked really well. Make sure > that in each docker you run tmux so yo can check both the output and do > things with the machine. > > What I do is put the Dockerfile in a sub folder of the project (because > docker copies all things in the folder the Dockerfile is hosted in into the > container and that is slow) and then map ../ to a mountpoint inside the > docker container. If your host is the same as the docker container then you > can build locally and just restart the process in the containers. > > -Mark > > > On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn < > cmeiklejohn@REDACTED> wrote: > >> >> On Jan 17, 2015, at 8:23 PM, Harit Himanshu < >> harit.subscriptions@REDACTED> wrote: >> >> >> This is really dumb question and I am pretty sure that there is limit to >> how much distributed programming could be learnt using single machine(I am >> using Mac in this case). >> >>> >>> I am reading through Programming Erlang, Chapter 14, Distributed >>> Programming where Joe talk about how to run Name Server in distributed mode >>> as >>> >>> >>> 1. I write and test my program in a regular nondistributed Erlang >>> session. This is what we?ve been doing up to now, so it presents no new >>> challenges. >>> 2. I test my program on two different Erlang nodes running on the >>> same computer. >>> 3. I test my program on two different Erlang nodes running on two >>> physically separated computers either in the same local area network or >>> anywhere on the Internet. >>> >>> You can get pretty far using all of the networking tools provided with >> your operating system to simulate various network conditions. I do all of >> my daily distributed programming and research using a stock MacBook. >> >> Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in >> several major distributed databases. There are also tools such as ?tc? >> which can be used to add arbitrary latency between processes. >> >> For what it?s worth, Basho does a significant amount of fault testing on >> a single machine by using fault-injection tools, or by facilities like >> ?intercepts? provided by Riak Test. >> >> My advice is this: do everything locally on your machine until you have a >> reason to move to separate infrastructure; it will be easier to develop and >> debug. >> >> - Chris >> >> Christopher Meiklejohn >> Senior Software Engineer >> Basho Technologies, Inc. >> cmeiklejohn@REDACTED >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Jan 17 21:18:20 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 17 Jan 2015 21:18:20 +0100 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: <54BAC38C.8010004@ninenines.eu> On 01/17/2015 09:05 PM, Harit Himanshu wrote: > Wow! I am glad to know I am wrong Being wrong is fine, everyone is wrong most of the time, the trick is to recognize that you are. On the other hand I'll give you one piece of advice: there are no dumb questions. A question might, however, be wrong, in the sense that it was triggered by misunderstanding things beforehand. Helping others figure out their mistake is much better than labeling it a "dumb question". Just my 2 cents. Enjoy! :-) -- Lo?c Hoguin http://ninenines.eu From lloyd@REDACTED Sat Jan 17 21:23:16 2015 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sat, 17 Jan 2015 15:23:16 -0500 (EST) Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: <1421526196.198831996@apps.rackspace.com> Hi Mark, Are you by chance running riak or riak CS on any of your machines? Thanks, LRP -----Original Message----- From: "Mark Nijhof" Sent: Saturday, January 17, 2015 2:53pm To: "Christopher Meiklejohn" Cc: "erlang questions" Subject: Re: [erlang-questions] Ideas on Distributed Programming on single machine I have had 14 different machines running on my 2 year old MacBook Air (granted they where not doing a whole lot) by just using docker. Each docker instance has its own IP and name ect. Worked really well. Make sure that in each docker you run tmux so yo can check both the output and do things with the machine. What I do is put the Dockerfile in a sub folder of the project (because docker copies all things in the folder the Dockerfile is hosted in into the container and that is slow) and then map ../ to a mountpoint inside the docker container. If your host is the same as the docker container then you can build locally and just restart the process in the containers. -Mark On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn <[ cmeiklejohn@REDACTED ]( mailto:cmeiklejohn@REDACTED )> wrote: On Jan 17, 2015, at 8:23 PM, Harit Himanshu <[ harit.subscriptions@REDACTED ]( mailto:harit.subscriptions@REDACTED )> wrote: This is really dumb question and I am pretty sure that there is limit to how much distributed programming could be learnt using single machine(I am using Mac in this case). I am reading through Programming Erlang, Chapter 14, Distributed Programming where Joe talk about how to run Name Server in distributed mode as I write and test my program in a regular nondistributed Erlang session. This is what we?ve been doing up to now, so it presents no new challenges. I test my program on two different Erlang nodes running on the same computer. I test my program on two different Erlang nodes running on two physically separated computers either in the same local area network or anywhere on the Internet. You can get pretty far using all of the networking tools provided with your operating system to simulate various network conditions. I do all of my daily distributed programming and research using a stock MacBook. Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in several major distributed databases. There are also tools such as ?tc? which can be used to add arbitrary latency between processes. For what it?s worth, Basho does a significant amount of fault testing on a single machine by using fault-injection tools, or by facilities like ?intercepts? provided by Riak Test. My advice is this: do everything locally on your machine until you have a reason to move to separate infrastructure; it will be easier to develop and debug. - Chris Christopher Meiklejohn Senior Software Engineer Basho Technologies, Inc. [ cmeiklejohn@REDACTED ]( mailto:cmeiklejohn@REDACTED ) _______________________________________________ erlang-questions mailing list [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) -- Mark Nijhof t: [ @MarkNijhof ]( https://twitter.com/MarkNijhof ) s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Sat Jan 17 21:43:55 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sat, 17 Jan 2015 21:43:55 +0100 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: This is my Dockerfile that can build Erlang: https://github.com/MarkNijhof/erlang_docker let me know if you have questions On Sat, Jan 17, 2015 at 9:06 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > Thanks Mark, this sounds like a very good approach to learn about > distributed programming. Since I am new in this arena, do you mind sharing > resources on how to achieve something you do? That ways I can get some > direction on what to Google and take it forward > > Thanks a lot! > + Harit Himanshu > > On Sat, Jan 17, 2015 at 11:53 AM, Mark Nijhof < > mark.nijhof@REDACTED> wrote: > >> I have had 14 different machines running on my 2 year old MacBook Air >> (granted they where not doing a whole lot) by just using docker. Each >> docker instance has its own IP and name ect. Worked really well. Make sure >> that in each docker you run tmux so yo can check both the output and do >> things with the machine. >> >> What I do is put the Dockerfile in a sub folder of the project (because >> docker copies all things in the folder the Dockerfile is hosted in into the >> container and that is slow) and then map ../ to a mountpoint inside the >> docker container. If your host is the same as the docker container then you >> can build locally and just restart the process in the containers. >> >> -Mark >> >> >> On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn < >> cmeiklejohn@REDACTED> wrote: >> >>> >>> On Jan 17, 2015, at 8:23 PM, Harit Himanshu < >>> harit.subscriptions@REDACTED> wrote: >>> >>> >>> This is really dumb question and I am pretty sure that there is limit to >>> how much distributed programming could be learnt using single machine(I am >>> using Mac in this case). >>> >>>> >>>> I am reading through Programming Erlang, Chapter 14, Distributed >>>> Programming where Joe talk about how to run Name Server in distributed mode >>>> as >>>> >>>> >>>> 1. I write and test my program in a regular nondistributed Erlang >>>> session. This is what we?ve been doing up to now, so it presents no new >>>> challenges. >>>> 2. I test my program on two different Erlang nodes running on the >>>> same computer. >>>> 3. I test my program on two different Erlang nodes running on two >>>> physically separated computers either in the same local area network or >>>> anywhere on the Internet. >>>> >>>> You can get pretty far using all of the networking tools provided with >>> your operating system to simulate various network conditions. I do all of >>> my daily distributed programming and research using a stock MacBook. >>> >>> Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in >>> several major distributed databases. There are also tools such as ?tc? >>> which can be used to add arbitrary latency between processes. >>> >>> For what it?s worth, Basho does a significant amount of fault testing on >>> a single machine by using fault-injection tools, or by facilities like >>> ?intercepts? provided by Riak Test. >>> >>> My advice is this: do everything locally on your machine until you have >>> a reason to move to separate infrastructure; it will be easier to develop >>> and debug. >>> >>> - Chris >>> >>> Christopher Meiklejohn >>> Senior Software Engineer >>> Basho Technologies, Inc. >>> cmeiklejohn@REDACTED >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Mark Nijhof >> t: @MarkNijhof >> s: marknijhof >> >> > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Sat Jan 17 21:46:08 2015 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Sat, 17 Jan 2015 21:46:08 +0100 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: <1421526196.198831996@apps.rackspace.com> References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> <1421526196.198831996@apps.rackspace.com> Message-ID: Not yet, but Riak is something I want to test out to see if it would be a good fit for what I am doing On Sat, Jan 17, 2015 at 9:23 PM, wrote: > Hi Mark, > > > > Are you by chance running riak or riak CS on any of your machines? > > > > Thanks, > > > > LRP > > > > -----Original Message----- > From: "Mark Nijhof" > Sent: Saturday, January 17, 2015 2:53pm > To: "Christopher Meiklejohn" > Cc: "erlang questions" > Subject: Re: [erlang-questions] Ideas on Distributed Programming on single > machine > > I have had 14 different machines running on my 2 year old MacBook Air > (granted they where not doing a whole lot) by just using docker. Each > docker instance has its own IP and name ect. Worked really well. Make sure > that in each docker you run tmux so yo can check both the output and do > things with the machine. > What I do is put the Dockerfile in a sub folder of the project (because > docker copies all things in the folder the Dockerfile is hosted in into the > container and that is slow) and then map ../ to a mountpoint inside the > docker container. If your host is the same as the docker container then you > can build locally and just restart the process in the containers. > -Mark > > On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn < > cmeiklejohn@REDACTED> wrote: > >> >> On Jan 17, 2015, at 8:23 PM, Harit Himanshu < >> harit.subscriptions@REDACTED> wrote: >> >> >> This is really dumb question and I am pretty sure that there is limit to >> how much distributed programming could be learnt using single machine(I am >> using Mac in this case). >> >>> I am reading through Programming Erlang, Chapter 14, Distributed >>> Programming where Joe talk about how to run Name Server in distributed mode >>> as >>> >>> 1. I write and test my program in a regular nondistributed Erlang >>> session. This is what we?ve been doing up to now, so it presents no new >>> challenges. >>> 2. I test my program on two different Erlang nodes running on the >>> same computer. >>> 3. I test my program on two different Erlang nodes running on two >>> physically separated computers either in the same local area network or >>> anywhere on the Internet. >>> >>> You can get pretty far using all of the networking tools provided >> with your operating system to simulate various network conditions. I do >> all of my daily distributed programming and research using a stock MacBook. >> >> Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in >> several major distributed databases. There are also tools such as ?tc? >> which can be used to add arbitrary latency between processes. >> For what it?s worth, Basho does a significant amount of fault testing on >> a single machine by using fault-injection tools, or by facilities like >> ?intercepts? provided by Riak Test. >> My advice is this: do everything locally on your machine until you have a >> reason to move to separate infrastructure; it will be easier to develop and >> debug. >> - Chris >> Christopher Meiklejohn >> Senior Software Engineer >> Basho Technologies, Inc. >> cmeiklejohn@REDACTED >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 17 21:46:39 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 17 Jan 2015 12:46:39 -0800 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: <634021C0-55DC-4614-ACBA-D7CFA731492C@basho.com> Message-ID: Thanks Mark, I will learn and try to use it, I will let you know if I have any questions Thanks a lot for your help On Sat, Jan 17, 2015 at 12:43 PM, Mark Nijhof < mark.nijhof@REDACTED> wrote: > This is my Dockerfile that can build Erlang: > https://github.com/MarkNijhof/erlang_docker let me know if you have > questions > > On Sat, Jan 17, 2015 at 9:06 PM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> Thanks Mark, this sounds like a very good approach to learn about >> distributed programming. Since I am new in this arena, do you mind sharing >> resources on how to achieve something you do? That ways I can get some >> direction on what to Google and take it forward >> >> Thanks a lot! >> + Harit Himanshu >> >> On Sat, Jan 17, 2015 at 11:53 AM, Mark Nijhof < >> mark.nijhof@REDACTED> wrote: >> >>> I have had 14 different machines running on my 2 year old MacBook Air >>> (granted they where not doing a whole lot) by just using docker. Each >>> docker instance has its own IP and name ect. Worked really well. Make sure >>> that in each docker you run tmux so yo can check both the output and do >>> things with the machine. >>> >>> What I do is put the Dockerfile in a sub folder of the project (because >>> docker copies all things in the folder the Dockerfile is hosted in into the >>> container and that is slow) and then map ../ to a mountpoint inside the >>> docker container. If your host is the same as the docker container then you >>> can build locally and just restart the process in the containers. >>> >>> -Mark >>> >>> >>> On Sat, Jan 17, 2015 at 8:46 PM, Christopher Meiklejohn < >>> cmeiklejohn@REDACTED> wrote: >>> >>>> >>>> On Jan 17, 2015, at 8:23 PM, Harit Himanshu < >>>> harit.subscriptions@REDACTED> wrote: >>>> >>>> >>>> This is really dumb question and I am pretty sure that there is limit >>>> to how much distributed programming could be learnt using single machine(I >>>> am using Mac in this case). >>>> >>>>> >>>>> I am reading through Programming Erlang, Chapter 14, Distributed >>>>> Programming where Joe talk about how to run Name Server in distributed mode >>>>> as >>>>> >>>>> >>>>> 1. I write and test my program in a regular nondistributed Erlang >>>>> session. This is what we?ve been doing up to now, so it presents no new >>>>> challenges. >>>>> 2. I test my program on two different Erlang nodes running on the >>>>> same computer. >>>>> 3. I test my program on two different Erlang nodes running on two >>>>> physically separated computers either in the same local area network or >>>>> anywhere on the Internet. >>>>> >>>>> You can get pretty far using all of the networking tools provided with >>>> your operating system to simulate various network conditions. I do all of >>>> my daily distributed programming and research using a stock MacBook. >>>> >>>> Consider Kyle Kingsbury?s work on Jepsen where he?s able to find bug in >>>> several major distributed databases. There are also tools such as ?tc? >>>> which can be used to add arbitrary latency between processes. >>>> >>>> For what it?s worth, Basho does a significant amount of fault testing >>>> on a single machine by using fault-injection tools, or by facilities like >>>> ?intercepts? provided by Riak Test. >>>> >>>> My advice is this: do everything locally on your machine until you have >>>> a reason to move to separate infrastructure; it will be easier to develop >>>> and debug. >>>> >>>> - Chris >>>> >>>> Christopher Meiklejohn >>>> Senior Software Engineer >>>> Basho Technologies, Inc. >>>> cmeiklejohn@REDACTED >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> Mark Nijhof >>> t: @MarkNijhof >>> s: marknijhof >>> >>> >> > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Jan 18 00:34:14 2015 From: zxq9@REDACTED (zxq9) Date: Sun, 18 Jan 2015 08:34:14 +0900 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: References: Message-ID: <2464140.6mU7PopJx6@changa> Quite a bit of the stuff I've written or helped maintain (in Erlang and otherwise) has wound up living inside VM instances (usually of Linux or BSD) in a KVM cluster somewhere. Sometimes these instances are on the same hardware host, sometimes not -- but it doesn't really matter, its just the way quite a lot of infrastructure actually runs these days (or docker, or VMWare, or whatever). Obviously, this reflects in testing as well. Using a full-blown virtualization platform like KVM it is very easy to simulate netsplits, paused hosts, congestion, host crashes, misrouting, overload on a single node, etc. the trick is learning enough about the environment and hosts you want to run to make that truly feasible. ...and having enough RAM to make running lots of instances realistic -- but for *most* scenarios you can learn a lot by using 3, 5, 7 or 8 instances. -Craig On 2015?1?17? ??? 12:46:39 Harit Himanshu wrote: > Thanks Mark, > > I will learn and try to use it, I will let you know if I have any questions > > Thanks a lot for your help > > On Sat, Jan 17, 2015 at 12:43 PM, Mark Nijhof < > > mark.nijhof@REDACTED> wrote: > > This is my Dockerfile that can build Erlang: > > https://github.com/MarkNijhof/erlang_docker let me know if you have > > questions > > > > On Sat, Jan 17, 2015 at 9:06 PM, Harit Himanshu < > > > > harit.subscriptions@REDACTED> wrote: > >> Thanks Mark, this sounds like a very good approach to learn about > >> distributed programming. Since I am new in this arena, do you mind > >> sharing > >> resources on how to achieve something you do? That ways I can get some > >> direction on what to Google and take it forward > >> > >> Thanks a lot! > >> + Harit Himanshu > >> > >> On Sat, Jan 17, 2015 at 11:53 AM, Mark Nijhof < > >> > >> mark.nijhof@REDACTED> wrote: > >>> I have had 14 different machines running on my 2 year old MacBook Air > >>> (granted they where not doing a whole lot) by just using docker. Each > >>> docker instance has its own IP and name ect. Worked really well. Make > >>> sure > >>> that in each docker you run tmux so yo can check both the output and do > >>> things with the machine. [ugh, all topposts... like a naive, panicky, CYA-inspired management discussion in Outlook] From rvirding@REDACTED Sun Jan 18 01:12:34 2015 From: rvirding@REDACTED (Robert Virding) Date: Sun, 18 Jan 2015 01:12:34 +0100 Subject: [erlang-questions] luerl stacktraces In-Reply-To: References: Message-ID: Btw the reason that luerl gives you an erlang stacktrace and not a proper luerl one is that the luerl machine is a bit of a hybrid. It uses erlang function calls for calling functions and loops while inside a function the code is compiled to a VM. What I plan to do is add more info to the luerl stack so it can be used to generate a bit of a luerl stacktrace. Robert On 12 January 2015 at 19:13, Robert Virding wrote: > Yes, I know. > > But I am doing some work on the Luerl VM and I am going to try to add more > Lua call stack information. Function names can be a bit difficult but line > numbers should be easier. > > Robert > > > On 12 January 2015 at 18:57, Max Lapshin wrote: > >> Hi. >> >> luerl (https://github.com/rvirding/luerl ) is a wonderful project, >> thanks for it. >> >> >> 3 years ago Robert told that it is a big problem to add proper >> stacktraces to lua errors. >> >> Right now it is: >> >> {lua_error,{undef_function,nil},{luerl,{array,25,100,undefined,{{{table,{array,0,10,nil,10},{{{{empty,<<"_G">>,{tref,0},empty,<<"_VERSION">>,<<"Lua >> 5.2">>,empty},<<"assert">>,{function,#Fun},{empty,<<"auth_url">>,<<" >> http://go.commpass.tv/flussonic/auth >> ">>,empty,<<"bit32">>,{tref,11},empty},<<"collectgarbage">>,{function,#Fun},{empty,<<"comet">>,{tref,19},empty,<<"converter">>,{tref,18},empty}},<<"dofile">>,{function,#Fun},{{empty,<<"eprint">>,{function,#Fun},empty},<<"error">>,..... >> >> >> For something like: >> >> mytable.unexisting_function("Hello world") >> >> >> Are there any severe problems to add stacktraces? >> >> It would be very good at least to have file and line of the error to show >> user where he has error. >> >> >> Perhaps some library function like: luerl_lib:unpack_stacktrace(Error) >> -> ... ? >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jan 18 09:26:50 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Jan 2015 12:26:50 +0400 Subject: [erlang-questions] luerl stacktraces In-Reply-To: References: Message-ID: Yes, I understand it. So the solution may be exactly as you describe: keep track of current lua stack. But I don't even understand where to put it. On Sun, Jan 18, 2015 at 3:12 AM, Robert Virding wrote: > Btw the reason that luerl gives you an erlang stacktrace and not a proper > luerl one is that the luerl machine is a bit of a hybrid. It uses erlang > function calls for calling functions and loops while inside a function the > code is compiled to a VM. What I plan to do is add more info to the luerl > stack so it can be used to generate a bit of a luerl stacktrace. > > Robert > > > On 12 January 2015 at 19:13, Robert Virding wrote: > >> Yes, I know. >> >> But I am doing some work on the Luerl VM and I am going to try to add >> more Lua call stack information. Function names can be a bit difficult but >> line numbers should be easier. >> >> Robert >> >> >> On 12 January 2015 at 18:57, Max Lapshin wrote: >> >>> Hi. >>> >>> luerl (https://github.com/rvirding/luerl ) is a wonderful project, >>> thanks for it. >>> >>> >>> 3 years ago Robert told that it is a big problem to add proper >>> stacktraces to lua errors. >>> >>> Right now it is: >>> >>> {lua_error,{undef_function,nil},{luerl,{array,25,100,undefined,{{{table,{array,0,10,nil,10},{{{{empty,<<"_G">>,{tref,0},empty,<<"_VERSION">>,<<"Lua >>> 5.2">>,empty},<<"assert">>,{function,#Fun},{empty,<<"auth_url">>,<<" >>> http://go.commpass.tv/flussonic/auth >>> ">>,empty,<<"bit32">>,{tref,11},empty},<<"collectgarbage">>,{function,#Fun},{empty,<<"comet">>,{tref,19},empty,<<"converter">>,{tref,18},empty}},<<"dofile">>,{function,#Fun},{{empty,<<"eprint">>,{function,#Fun},empty},<<"error">>,..... >>> >>> >>> For something like: >>> >>> mytable.unexisting_function("Hello world") >>> >>> >>> Are there any severe problems to add stacktraces? >>> >>> It would be very good at least to have file and line of the error to >>> show user where he has error. >>> >>> >>> Perhaps some library function like: luerl_lib:unpack_stacktrace(Error) >>> -> ... ? >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kunthar@REDACTED Sun Jan 18 11:01:35 2015 From: kunthar@REDACTED (Gokhan Boranalp) Date: Sun, 18 Jan 2015 12:01:35 +0200 Subject: [erlang-questions] Ideas on Distributed Programming on single machine In-Reply-To: <2464140.6mU7PopJx6@changa> References: <2464140.6mU7PopJx6@changa> Message-ID: riak docker cluster. http://basho.com/riak-quick-start-with-docker/ latest docker repo https://github.com/hectcastro/docker-riak afaik method changed to connect cluster members. On 18 Jan 2015 01:34, "zxq9" wrote: > Quite a bit of the stuff I've written or helped maintain (in Erlang and > otherwise) has wound up living inside VM instances (usually of Linux or > BSD) > in a KVM cluster somewhere. Sometimes these instances are on the same > hardware > host, sometimes not -- but it doesn't really matter, its just the way > quite a > lot of infrastructure actually runs these days (or docker, or VMWare, or > whatever). > > Obviously, this reflects in testing as well. > > Using a full-blown virtualization platform like KVM it is very easy to > simulate netsplits, paused hosts, congestion, host crashes, misrouting, > overload on a single node, etc. the trick is learning enough about the > environment and hosts you want to run to make that truly feasible. > > ...and having enough RAM to make running lots of instances realistic -- but > for *most* scenarios you can learn a lot by using 3, 5, 7 or 8 instances. > > -Craig > > On 2015?1?17? ??? 12:46:39 Harit Himanshu wrote: > > Thanks Mark, > > > > I will learn and try to use it, I will let you know if I have any > questions > > > > Thanks a lot for your help > > > > On Sat, Jan 17, 2015 at 12:43 PM, Mark Nijhof < > > > > mark.nijhof@REDACTED> wrote: > > > This is my Dockerfile that can build Erlang: > > > https://github.com/MarkNijhof/erlang_docker let me know if you have > > > questions > > > > > > On Sat, Jan 17, 2015 at 9:06 PM, Harit Himanshu < > > > > > > harit.subscriptions@REDACTED> wrote: > > >> Thanks Mark, this sounds like a very good approach to learn about > > >> distributed programming. Since I am new in this arena, do you mind > > >> sharing > > >> resources on how to achieve something you do? That ways I can get some > > >> direction on what to Google and take it forward > > >> > > >> Thanks a lot! > > >> + Harit Himanshu > > >> > > >> On Sat, Jan 17, 2015 at 11:53 AM, Mark Nijhof < > > >> > > >> mark.nijhof@REDACTED> wrote: > > >>> I have had 14 different machines running on my 2 year old MacBook Air > > >>> (granted they where not doing a whole lot) by just using docker. Each > > >>> docker instance has its own IP and name ect. Worked really well. Make > > >>> sure > > >>> that in each docker you run tmux so yo can check both the output and > do > > >>> things with the machine. > > [ugh, all topposts... like a naive, panicky, CYA-inspired management > discussion in Outlook] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lunar@REDACTED Sun Jan 18 14:24:06 2015 From: lunar@REDACTED (=?iso-8859-1?B?Suly6W15?= Bobbio) Date: Sun, 18 Jan 2015 14:24:06 +0100 Subject: [erlang-questions] Usage of the 'time' attribute in BEAM files Message-ID: <20150118132406.GC5659@loar> Hi! We are a group of Debian contributors working on ?reproducible builds??[1]: compiling a source package should always result in the same binaries for a given build environment. Our tests have shown one difference?[2] in Erlang BEAM files produced by the compiler. The time of the build is recorded in the 'time' attribute. How important is this attribute? Chris West has noticed that when using the `+slim` option, it did not get written to the output?[3]. Would it make sense to add a `+deterministic` build option?[4] that would do the same? [1]: https://wiki.debian.org/ReproducibleBuilds [2]:?https://wiki.debian.org/ReproducibleBuilds/TimestampsInBeamFiles?action=AttachFile&do=get&target=ejabberd-contrib_0.2014.09.22-1.debbindiff.html [3]:?http://sources.debian.net/src/erlang/1:17.3-dfsg-3/lib/compiler/src/beam_asm.erl/?hl=225:233#L225 [4]: This is inspired by binutils. See the `D` option of `ar` for an example: http://manpages.debian.org/ar Thanks, -- Lunar .''`. lunar@REDACTED : :? : # apt-get install anarchism `. `'` `- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: From achowdhury918@REDACTED Sun Jan 18 16:03:54 2015 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Sun, 18 Jan 2015 10:03:54 -0500 Subject: [erlang-questions] Hiring - result oriented Erlang developer In-Reply-To: References: Message-ID: Hi, I know few of good Erlang developer who are looking. I will send you their info in your e-mail. Thanks. On Sat, Jan 17, 2015 at 2:39 AM, Praven p wrote: > Hi, > > Looking for an Erlang developer for an ambitious startup. > Exposure/experience to Zotonic CMS preferred. We are building something in > what we imagine to be in the Internet of People world. > > We are looking for an expert (experience years not important but result > oriented) with high integrity and commitment. Please reach out should you > like to discuss. > > best wishes / P > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowdhury918@REDACTED Sun Jan 18 16:05:06 2015 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Sun, 18 Jan 2015 10:05:06 -0500 Subject: [erlang-questions] is anyone hiring? In-Reply-To: <54B94326.1010709@gmail.com> References: <54B94326.1010709@gmail.com> Message-ID: Could you please send me your resume? I can help you regarding this. Thanks. On Fri, Jan 16, 2015 at 11:58 AM, the.silly.sad wrote: > Hello, all. > Could you suggest a good method of erlang-related job search? > Or some business entities that are worth sending a junior CV to? > Would you like to share some links or advises or maybe anyone of you guys > is hiring? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowdhury918@REDACTED Sun Jan 18 16:19:26 2015 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Sun, 18 Jan 2015 10:19:26 -0500 Subject: [erlang-questions] symbol SSLeay: referenced symbol not found Message-ID: All, I am initializing websocket request to my erlang server which uses cowboy. I have installed openssl too. My operating system is Unix Solaris (sparc-sun-solaris2.10). I have installed openssl-1.0.1l. I am getting following error : Unable to load crypto library. Failed with error: "load_failed, Failed to load NIF library: 'ld.so.1: beam.smp: fatal: relocation error: file /apps/erlang/otp_src_17.4/lib/crypto/priv/lib/sparc-sun-solaris2.10/crypto.so: symbol SSLeay: referenced symbol not found'" OpenSSL might not be installed on this system. =WARNING REPORT==== 18-Jan-2015::06:57:16 === The on_load function for module crypto returned {error, {load_failed, "Failed to load NIF library: 'ld.so.1: beam.smp: fatal: relocation error: file /apps/erlang/otp_src_17.4/lib/crypto/priv/lib/sparc-sun-solaris2.10/crypto.so: symbol SSLeay: referenced symbol not found'"}} =ERROR REPORT==== 18-Jan-2015::06:57:16 === Error in process <0.160.0> on node 'erws@REDACTED' with exit value: {undef,[{crypto,hash,[sha,<<60 bytes>>],[]},{cowboy_websocket,websocket_handshake,3,[{file,"cowboy_websocket.erl"},{line,154}]},{cowboy_protocol,execute,4,[{file,"cowboy_protocol.erl"},{line,467}]}]} I have ld.so.1 in my path too. Can anyone help me regarding this? Resolution or Work-around anything will be highly appreciated. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Jan 18 20:12:18 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 18 Jan 2015 23:12:18 +0400 Subject: [erlang-questions] Usage of the 'time' attribute in BEAM files In-Reply-To: <20150118132406.GC5659@loar> References: <20150118132406.GC5659@loar> Message-ID: Maybe you should try to remove this attribute from beam? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wgwi@REDACTED Mon Jan 19 06:28:27 2015 From: wgwi@REDACTED (=?utf-8?B?KQ==?=) Date: Mon, 19 Jan 2015 13:28:27 +0800 Subject: [erlang-questions] snmpcl timeout problem In-Reply-To: References: Message-ID: Hi All, I try to monitor remote machines SNMP oids, Python, Java walk well, but while I use the snmpcl sample demo on github, each time give me fellowing error. Eshell V5.10.4 (abort with ^G) 1> application:start(snmpcl). ok 2> snmpcl:walk("172.24.3.5", [1,3,6,1,4,1,2606,4,2,4,5,2,1,5,14]). ok 3> =ERROR REPORT==== 19-Jan-2015::13:15:24 === Can't send request to 172.24.3.5 due to {timeout,1028266095} Can anyone help me regarding this? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Jan 19 13:21:53 2015 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Jan 2015 13:21:53 +0100 Subject: [erlang-questions] Tracing and debugging In-Reply-To: References: Message-ID: <1988CF41-8DE1-40C9-B5DE-A9D503509DCE@feuerlabs.com> > On 13 Jan 2015, at 13:36, Vlad Dumitrescu wrote: > > The main reason I use the debugger is to check that the intermediary values in a computation are the expected ones. The alternative (which works without messing up timeouts) is to print out values at points of interest, but it is messy (there's a lot of boilerplate to type and the interesting code becomes hard to read). > > So, I thought, what if, instead of interpreting a module in order to debug it, we compile it with a special parse transform that inserts tracing calls after each expression in the code, automatically keeping track of the variables visible in the scope and their values? Hmm? I don?t consider myself qualified to discuss the details of a bytecode-level debugger, but it seems to me as if it would be a very difficult thing to get right, balancing usability during debugging with efficiency of bytecode while not debugging (not to mention the effects of aggressive compile-time optimizations). Then again, anything written after ?I dont consider myself qualified? but? can probably safely be ignored. In terms of writing debuggable code, there are some things that can be done without hacking the VM: * Write many small functions * Separate out side-effects, keeping as many functions as possible side-effect free After this, tracing can help you capture function inputs, and you can then single-step through a troublesome function at your leisure. * If your functions take records as input, use the shell function rr(Module) so you can use record syntax from the shell. For me, this takes care of the vast majority of cases where single-stepping can come into play. Possibly due to my own personal bent, I always end up trying to debug distributed coordination logic, in which case, this is would be my current recommendation: * Instead of io:format() debugging, call an empty function that can be traced when needed I?ve been using this when debugging the ?locks? application, and while it didn?t make debugging leader election during netsplits exactly easy, it helped a great deal. Example below. In the locks application, I wrote a module, locks_ttb.erl, which is a thin wrapper around the trace tool ttb. To begin with, I use a debug macro I call ?event() in the code: -define(event(E), event(?LINE, E, none)). -define(event(E, S), event(?LINE, E, S)). The event/3 function itself is as light as possible: event(_Line, _Event, _State) -> ok. It contains a convenient function for starting a multi-node trace on my strategic ?debug functions?. Another function, locks_ttb:format(File, OutFile) takes a merged trace log (containing timestamped traces from all nodes) and pretty-prints the traces, not least checking for record field metadata and using record syntax if possible. In particular, I use this in the unit test for leader_election with netsplit: https://github.com/uwiger/locks/blob/master/test/locks_leader_tests.erl#L73 The setup call to locks_ttb enables tracing on the test nodes as well as the test function itself (which also uses ?event? function debug entries). The trace data is collected only if the test fails. It can then be analyzed with the help of locks_ttb:format/2. I?ve actually been meaning to test this style of trace/debug for years, but never quite got around to it. There are quite a few things that could be done beyond this - those to remember the PROTEST project may recall a few more or less wild ideas. What I?ve done in ?locks' is actually quite basic. I?ve been thinking about taking this a step further, writing it up and perhaps giving a presentation about it, but I?ve been having too much fun. The only thing locks-specific in locks_ttb.erl, BTW, is the function default_pattern/0. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.rascao@REDACTED Mon Jan 19 12:19:29 2015 From: luis.rascao@REDACTED (=?UTF-8?B?THVpcyBSYXNjw6Nv?=) Date: Mon, 19 Jan 2015 11:19:29 +0000 Subject: [erlang-questions] High reduction count on init:loop Message-ID: Hi all, I'm running load tests based on R16B03-1 and i'm looking at the reduction count using Fred's excellent recon tool (https://github.com/ferd/recon). I'm seeing a high reduction count on the init process ({<0.0.0>), the top 5 ranking for reduction count is below, init's reductions are a bit higher than the other 4 which are poolboy managers. [{<0.0.0>,37015347, [init, {current_function,{init,loop,1}}, {initial_call,{otp_ring0,start,2}}]}, {<0.7754.0>,29789978, [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.8309.0>,28671516, [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.8478.0>,28597415, [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.7796.0>,28185014, [{current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}] Does anyone have any insight as to the reason behind init's high reduction count? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Jan 19 13:56:01 2015 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 19 Jan 2015 13:56:01 +0100 Subject: [erlang-questions] High reduction count on init:loop In-Reply-To: References: Message-ID: <1FE4AE0A-189E-4813-B718-04113E8156EE@feuerlabs.com> > On 19 Jan 2015, at 12:19, Luis Rasc?o wrote: > > I'm running load tests based on R16B03-1 and i'm looking at the reduction count using Fred's excellent recon tool (https://github.com/ferd/recon ). I'm seeing a high reduction count on the init process ({<0.0.0>), the top 5 ranking for reduction count is below, init's reductions are a bit higher than the other 4 which are poolboy managers. The init process evaluates the boot script. In a fairly large system, it can be expected to rack up a lot of reductions during system startup. After that, it should stay pretty quiet. Example, starting a riak node: (dev1@REDACTED)2> process_info(whereis(init),reductions). {reductions,257559} A few minutes later: (dev1@REDACTED)3> process_info(whereis(init),reductions). {reductions,257957} BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Jan 19 14:46:46 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 19 Jan 2015 14:46:46 +0100 Subject: [erlang-questions] Does Inets HTTP client and PATCH method? In-Reply-To: <830B5642-695C-44C5-A5D4-5DDDED104520@btinternet.com> References: <830B5642-695C-44C5-A5D4-5DDDED104520@btinternet.com> Message-ID: On Sat, Jan 17, 2015 at 12:24 PM, John Duffy wrote: > My Foreign Exchange broker provides a HTTP REST API for automated trading, > and uses the HTTP PATCH method for updating outstanding trade orders. I > can't find a reference to the PATCH method in the httpc client > documentation. Is this HTTP method supported? No, it is not. Only the basic ones listed in RFC 7231, https://tools.ietf.org/html/rfc7231#section-4 are supported[0]. The registered method extensions listed in the HTTP method registry: http://www.iana.org/assignments/http-methods/http-methods.xhtml seems to be without support. It should be fairly easy to add missing methods however. An alternative route would be to employ one of the many http client libraries out there. I'm partial to 'hackney' by Beno?t Chesnau, but there are numerous others. On top of my head: lhttpc (Two branches: Fred Hebert, and one by Erlang Solutions Ltd.), gun (Lo?c Hoguin), ibrowse (Chandrashekhar Mullaparthi), fusco (Diana Corbacho) and probably a couple I've forgotten. Different implementations enjoy different properties, so picking the one for your project may be something that takes a little time. I well known problem with the built-in httpc is that it has problems when experiencing high load. But I've used it successfully for a low-traffic site for years (only GET requests though). [0] though I have my doubts about CONNECT. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jan 19 15:56:08 2015 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Jan 2015 18:56:08 +0400 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: Looks like it lacks the most interesting: https://code.google.com/p/telepresence/source/browse/#svn%2Ftrunk%2Fsource%2Fgateways%2Fwebrtc%253Fstate%253Dclosed Nothing about webrtc -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoine.koener@REDACTED Mon Jan 19 16:15:13 2015 From: antoine.koener@REDACTED (Antoine Koener) Date: Mon, 19 Jan 2015 16:15:13 +0100 Subject: [erlang-questions] DTLS/SRTP for WebRTC In-Reply-To: References: <651666163.987017.1416222640143.JavaMail.zimbra@tpip.net> <1561626763.987018.1416222640677.JavaMail.zimbra@tpip.net> Message-ID: <4568A64F-63FE-4363-9CCA-D9BEB80DA389@gmail.com> I can say that when i've compiled the code following the "technical guide" the webrtc was successfully tested with the http://conf-call.org site... May be see: Https://doubango.googlecode.com/ Which is mandatory to build telepresence. > On 19 Jan 2015, at 15:56, Max Lapshin wrote: > > Looks like it lacks the most interesting: https://code.google.com/p/telepresence/source/browse/#svn%2Ftrunk%2Fsource%2Fgateways%2Fwebrtc%253Fstate%253Dclosed > > Nothing about webrtc -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 19 17:05:31 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 19 Jan 2015 08:05:31 -0800 Subject: [erlang-questions] Erlang Nodes can't communicate with each other Message-ID: Hi I am learning Distributed programming and in order to work through it, I started 2 docker containers using images provided by Mark Nijhof ( https://github.com/MarkNijhof/erlang_docker) So now when I start my docker containers (on same Macbook), I observe that they are not able to talk to each other *Server* root@REDACTED:/code# erl -sname gru -setcookie abc Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0 (abort with ^G) (gru@REDACTED)1> kvs:start(). true (gru@REDACTED)2> *Minion* ? erlang_docker git:(master) docker run -t -i erlang-build-box /bin/bash root@REDACTED:/# mkdir code root@REDACTED:/# cd code root@REDACTED:/code# vi kvs.erl root@REDACTED:/code# erl -sname minion1 -setcookie abc Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0 (abort with ^G) (minion1@REDACTED)1> rpc:call(gru@REDACTED, kvs, store, [weather, fine]). {badrpc,nodedown} (minion1@REDACTED)2> rpc:call(gru@REDACTED, kvs, store, [weather, fine]). {badrpc,nodedown} (minion1@REDACTED)3> c(kvs). {ok,kvs} (minion1@REDACTED)4> rpc:call(gru@REDACTED, kvs, store, [weather, fine]). {badrpc,nodedown} (minion1@REDACTED)5> My code looks like following and it works correctly when both server and minion are on same node -module(kvs). -author("harith"). %% API -export([start/0, store/2, lookup/1]). start() -> register(kvs, spawn(fun() -> loop() end)). store(Key, Value) -> rpc({store, Key, Value}). lookup(Key) -> rpc({lookup, Key}). rpc(Q) -> kvs ! {self(), Q}, receive {kvs, Reply} -> Reply end. loop() -> receive {From, {store, Key, Value}} -> put(Key, {ok, Value}), From ! {kvs, true}, loop(); {From, {lookup, Key}} -> From ! {kvs, get(Key)}, loop() end. What am I missing here? Thanks + Harit -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.widgren@REDACTED Mon Jan 19 17:22:52 2015 From: daniel.widgren@REDACTED (Daniel Hallin Widgren) Date: Mon, 19 Jan 2015 17:22:52 +0100 Subject: [erlang-questions] Erlang Nodes can't communicate with each other In-Reply-To: References: Message-ID: Hi, Have you tried to use -name mynode@REDACTED instead of -sname mynode? I think you need long name activated to talk to each other. regards, Daniel 2015-01-19 17:05 GMT+01:00 Harit Himanshu : > Hi > > I am learning Distributed programming and in order to work through it, I > started 2 docker containers using images provided by Mark Nijhof ( > https://github.com/MarkNijhof/erlang_docker) > > So now when I start my docker containers (on same Macbook), I observe that > they are not able to talk to each other > > *Server* > > root@REDACTED:/code# erl -sname gru -setcookie abc > > Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > > Eshell V6.0 (abort with ^G) > > (gru@REDACTED)1> kvs:start(). > > true > > (gru@REDACTED)2> > > *Minion* > > ? erlang_docker git:(master) docker run -t -i erlang-build-box /bin/bash > > root@REDACTED:/# mkdir code > > root@REDACTED:/# cd code > > root@REDACTED:/code# vi kvs.erl > > root@REDACTED:/code# erl -sname minion1 -setcookie abc > > Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > > Eshell V6.0 (abort with ^G) > > (minion1@REDACTED)1> rpc:call(gru@REDACTED, kvs, store, [weather, > fine]). > > {badrpc,nodedown} > > (minion1@REDACTED)2> rpc:call(gru@REDACTED, kvs, store, [weather, > fine]). > > {badrpc,nodedown} > > (minion1@REDACTED)3> c(kvs). > > {ok,kvs} > > (minion1@REDACTED)4> rpc:call(gru@REDACTED, kvs, store, [weather, > fine]). > > {badrpc,nodedown} > > (minion1@REDACTED)5> > > > > My code looks like following and it works correctly when both server and > minion are on same node > > -module(kvs). > -author("harith"). > > %% API > -export([start/0, store/2, lookup/1]). > > start() -> > register(kvs, spawn(fun() -> loop() end)). > > store(Key, Value) -> > rpc({store, Key, Value}). > > lookup(Key) -> > rpc({lookup, Key}). > > rpc(Q) -> > kvs ! {self(), Q}, > receive > {kvs, Reply} -> > Reply > end. > > loop() -> > receive > {From, {store, Key, Value}} -> > put(Key, {ok, Value}), > From ! {kvs, true}, > loop(); > {From, {lookup, Key}} -> > From ! {kvs, get(Key)}, > loop() > end. > > > What am I missing here? > > Thanks > + Harit > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 19 17:44:58 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 19 Jan 2015 08:44:58 -0800 Subject: [erlang-questions] Erlang Nodes can't communicate with each other In-Reply-To: References: Message-ID: Thanks Daniel, I just tried that, I see different issue now *Server* root@REDACTED:/code# ifconfig eth0 Link encap:Ethernet HWaddr 02:42:ac:11:00:3e inet addr:172.17.0.62 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::42:acff:fe11:3e/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:15 errors:0 dropped:0 overruns:0 frame:0 TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1260 (1.2 KB) TX bytes:1088 (1.0 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:30 errors:0 dropped:0 overruns:0 frame:0 TX packets:30 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1674 (1.6 KB) TX bytes:1674 (1.6 KB) root@REDACTED:/code# erl -name gru@REDACTED -setcookie abc Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0 (abort with ^G) (gru@REDACTED)1> kvs:start() (gru@REDACTED)1> . true (gru@REDACTED)2> *Client* (minion1@REDACTED)6> rpc:call(gru@REDACTED, kvs, store, [weather, fine]). * 1: syntax error before: '.' (minion1@REDACTED)6> rpc:call('gru@REDACTED', kvs, store, [weather, fine]). {badrpc,nodedown} (minion1@REDACTED)7> =ERROR REPORT==== 19-Jan-2015::16:42:56 === ** System NOT running to use fully qualified hostnames ** ** Hostname 172.17.0.62 is illegal ** I don't understand why Host is illegal, any ideas? Thanks On Mon, Jan 19, 2015 at 8:22 AM, Daniel Hallin Widgren < daniel.widgren@REDACTED> wrote: > Hi, > > Have you tried to use -name mynode@REDACTED instead of -sname mynode? I think > you need long name activated to talk to each other. > > regards, > Daniel > > 2015-01-19 17:05 GMT+01:00 Harit Himanshu : > >> Hi >> >> I am learning Distributed programming and in order to work through it, I >> started 2 docker containers using images provided by Mark Nijhof ( >> https://github.com/MarkNijhof/erlang_docker) >> >> So now when I start my docker containers (on same Macbook), I observe >> that they are not able to talk to each other >> >> *Server* >> >> root@REDACTED:/code# erl -sname gru -setcookie abc >> >> Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] >> [hipe] [kernel-poll:false] >> >> >> Eshell V6.0 (abort with ^G) >> >> (gru@REDACTED)1> kvs:start(). >> >> true >> >> (gru@REDACTED)2> >> >> *Minion* >> >> ? erlang_docker git:(master) docker run -t -i erlang-build-box /bin/bash >> >> root@REDACTED:/# mkdir code >> >> root@REDACTED:/# cd code >> >> root@REDACTED:/code# vi kvs.erl >> >> root@REDACTED:/code# erl -sname minion1 -setcookie abc >> >> Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] >> [hipe] [kernel-poll:false] >> >> >> Eshell V6.0 (abort with ^G) >> >> (minion1@REDACTED)1> rpc:call(gru@REDACTED, kvs, store, >> [weather, fine]). >> >> {badrpc,nodedown} >> >> (minion1@REDACTED)2> rpc:call(gru@REDACTED, kvs, store, >> [weather, fine]). >> >> {badrpc,nodedown} >> >> (minion1@REDACTED)3> c(kvs). >> >> {ok,kvs} >> >> (minion1@REDACTED)4> rpc:call(gru@REDACTED, kvs, store, >> [weather, fine]). >> >> {badrpc,nodedown} >> >> (minion1@REDACTED)5> >> >> >> >> My code looks like following and it works correctly when both server and >> minion are on same node >> >> -module(kvs). >> -author("harith"). >> >> %% API >> -export([start/0, store/2, lookup/1]). >> >> start() -> >> register(kvs, spawn(fun() -> loop() end)). >> >> store(Key, Value) -> >> rpc({store, Key, Value}). >> >> lookup(Key) -> >> rpc({lookup, Key}). >> >> rpc(Q) -> >> kvs ! {self(), Q}, >> receive >> {kvs, Reply} -> >> Reply >> end. >> >> loop() -> >> receive >> {From, {store, Key, Value}} -> >> put(Key, {ok, Value}), >> From ! {kvs, true}, >> loop(); >> {From, {lookup, Key}} -> >> From ! {kvs, get(Key)}, >> loop() >> end. >> >> >> What am I missing here? >> >> Thanks >> + Harit >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 19 17:56:47 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 19 Jan 2015 08:56:47 -0800 Subject: [erlang-questions] Erlang Nodes can't communicate with each other In-Reply-To: References: Message-ID: I just caught the issue. My client was running with *-sname *(I am guessing this is issue). Now following works for me *Server* root@REDACTED:/code# erl -name gru@REDACTED -setcookie abc Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0 (abort with ^G) (gru@REDACTED)1> kvs:start(). ** exception error: undefined function kvs:start/0 (gru@REDACTED)2> c(kvs). {ok,kvs} (gru@REDACTED)3> kvs:start(). true (gru@REDACTED)4> kvs:lookup(weather). {ok,fine} (gru@REDACTED)5> kvs:lookup(code). {ok,working} (gru@REDACTED)6> *Client* root@REDACTED:/code# erl -name minion1@REDACTED -setcookie abc Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0 (abort with ^G) (minion1@REDACTED)1> rpc:call('gru@REDACTED', kvs, store, [weather, fine]). true (minion1@REDACTED)2> rpc:call('gru@REDACTED', kvs, lookup, [weather]). {ok,fine} (minion1@REDACTED)3> rpc:call('gru@REDACTED', kvs, store, [code, working]). true (minion1@REDACTED)4> rpc:call('gru@REDACTED', kvs, lookup, [code]). {ok,working} (minion1@REDACTED)5> Thanks for all your help + Harit On Mon, Jan 19, 2015 at 8:44 AM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > Thanks Daniel, I just tried that, I see different issue now > > *Server* > > root@REDACTED:/code# ifconfig > > eth0 Link encap:Ethernet HWaddr 02:42:ac:11:00:3e > > inet addr:172.17.0.62 Bcast:0.0.0.0 Mask:255.255.0.0 > > inet6 addr: fe80::42:acff:fe11:3e/64 Scope:Link > > UP BROADCAST RUNNING MTU:1500 Metric:1 > > RX packets:15 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > RX bytes:1260 (1.2 KB) TX bytes:1088 (1.0 KB) > > > lo Link encap:Local Loopback > > inet addr:127.0.0.1 Mask:255.0.0.0 > > inet6 addr: ::1/128 Scope:Host > > UP LOOPBACK RUNNING MTU:65536 Metric:1 > > RX packets:30 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:30 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:0 > > RX bytes:1674 (1.6 KB) TX bytes:1674 (1.6 KB) > > > root@REDACTED:/code# erl -name gru@REDACTED -setcookie abc > > Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > > Eshell V6.0 (abort with ^G) > > (gru@REDACTED)1> kvs:start() > > (gru@REDACTED)1> . > > true > > (gru@REDACTED)2> > > *Client* > > (minion1@REDACTED)6> rpc:call(gru@REDACTED, kvs, store, [weather, > fine]). > > * 1: syntax error before: '.' > > (minion1@REDACTED)6> rpc:call('gru@REDACTED', kvs, store, > [weather, fine]). > > {badrpc,nodedown} > > (minion1@REDACTED)7> > > =ERROR REPORT==== 19-Jan-2015::16:42:56 === > > ** System NOT running to use fully qualified hostnames ** > > ** Hostname 172.17.0.62 is illegal ** > > > > > > I don't understand why Host is illegal, any ideas? > Thanks > > On Mon, Jan 19, 2015 at 8:22 AM, Daniel Hallin Widgren < > daniel.widgren@REDACTED> wrote: > >> Hi, >> >> Have you tried to use -name mynode@REDACTED instead of -sname mynode? I think >> you need long name activated to talk to each other. >> >> regards, >> Daniel >> >> 2015-01-19 17:05 GMT+01:00 Harit Himanshu >> : >> >>> Hi >>> >>> I am learning Distributed programming and in order to work through it, I >>> started 2 docker containers using images provided by Mark Nijhof ( >>> https://github.com/MarkNijhof/erlang_docker) >>> >>> So now when I start my docker containers (on same Macbook), I observe >>> that they are not able to talk to each other >>> >>> *Server* >>> >>> root@REDACTED:/code# erl -sname gru -setcookie abc >>> >>> Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] >>> [hipe] [kernel-poll:false] >>> >>> >>> Eshell V6.0 (abort with ^G) >>> >>> (gru@REDACTED)1> kvs:start(). >>> >>> true >>> >>> (gru@REDACTED)2> >>> >>> *Minion* >>> >>> ? erlang_docker git:(master) docker run -t -i erlang-build-box >>> /bin/bash >>> >>> root@REDACTED:/# mkdir code >>> >>> root@REDACTED:/# cd code >>> >>> root@REDACTED:/code# vi kvs.erl >>> >>> root@REDACTED:/code# erl -sname minion1 -setcookie abc >>> >>> Erlang/OTP 17 [erts-6.0] [source] [64-bit] [smp:8:8] [async-threads:10] >>> [hipe] [kernel-poll:false] >>> >>> >>> Eshell V6.0 (abort with ^G) >>> >>> (minion1@REDACTED)1> rpc:call(gru@REDACTED, kvs, store, >>> [weather, fine]). >>> >>> {badrpc,nodedown} >>> >>> (minion1@REDACTED)2> rpc:call(gru@REDACTED, kvs, store, >>> [weather, fine]). >>> >>> {badrpc,nodedown} >>> >>> (minion1@REDACTED)3> c(kvs). >>> >>> {ok,kvs} >>> >>> (minion1@REDACTED)4> rpc:call(gru@REDACTED, kvs, store, >>> [weather, fine]). >>> >>> {badrpc,nodedown} >>> >>> (minion1@REDACTED)5> >>> >>> >>> >>> My code looks like following and it works correctly when both server and >>> minion are on same node >>> >>> -module(kvs). >>> -author("harith"). >>> >>> %% API >>> -export([start/0, store/2, lookup/1]). >>> >>> start() -> >>> register(kvs, spawn(fun() -> loop() end)). >>> >>> store(Key, Value) -> >>> rpc({store, Key, Value}). >>> >>> lookup(Key) -> >>> rpc({lookup, Key}). >>> >>> rpc(Q) -> >>> kvs ! {self(), Q}, >>> receive >>> {kvs, Reply} -> >>> Reply >>> end. >>> >>> loop() -> >>> receive >>> {From, {store, Key, Value}} -> >>> put(Key, {ok, Value}), >>> From ! {kvs, true}, >>> loop(); >>> {From, {lookup, Key}} -> >>> From ! {kvs, get(Key)}, >>> loop() >>> end. >>> >>> >>> What am I missing here? >>> >>> Thanks >>> + Harit >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xiut.xu@REDACTED Mon Jan 19 18:01:54 2015 From: xiut.xu@REDACTED (xu xiut) Date: Mon, 19 Jan 2015 12:01:54 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> Message-ID: For those who were wondering what type of exchange I'm wanting to build, it's a bitcoin exchange with derivative contracts. I asked in the IRC room for some more input, and here's what I received: re:failure semantics of Erlang not being appropriate for an exchange < tristan_1> failure/recovery in Erlang has nothing to do with that < tristan_1> no matter the language/framework you have to decide how to handle these things. there isn't a way you can possible write a program that it won't crash at some point < asonge> you can design your app so that you log all requests, and make sure that is as tight as possible, and that you can replay them if there's a crash (for instance) There was an interesting suggestion to use OCaml or Haskell instead of Erlang: < digitalmentat> I would pick a language that gives more formal verification of the software properties < digitalmentat> Haskell and OCaml would be great picks < digitalmentat> I use Haskell in production for my startup and 80% of the dumb shit humans do (including me) is caught by that language's type-system < digitalmentat> anything financial is a risky business to get into and most developers aren't aware of the social impact nor the legal ramifications if they fuck something up My question is, would Dialyzer not be able to offer equivalent functionality? Would using Erlang as a wrapper around a well-typed core work out okay, does anyone here have experience they could share on this? I've never done this, so it would be a completely new process for me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb_duffy@REDACTED Mon Jan 19 18:06:15 2015 From: jb_duffy@REDACTED (John Duffy) Date: Mon, 19 Jan 2015 17:06:15 +0000 Subject: [erlang-questions] Does Inets HTTP client and PATCH method? In-Reply-To: References: <830B5642-695C-44C5-A5D4-5DDDED104520@btinternet.com> Message-ID: <14FAF4E5-44EE-4F1A-8943-5309C1E0FBFF@btinternet.com> Thank you Jesper I will look into the libraries you mention. Kind regards John Sent from my iPhone > On 19 Jan 2015, at 13:46, Jesper Louis Andersen wrote: > > >> On Sat, Jan 17, 2015 at 12:24 PM, John Duffy wrote: >> My Foreign Exchange broker provides a HTTP REST API for automated trading, and uses the HTTP PATCH method for updating outstanding trade orders. I can't find a reference to the PATCH method in the httpc client documentation. Is this HTTP method supported? > > No, it is not. > > Only the basic ones listed in RFC 7231, > > https://tools.ietf.org/html/rfc7231#section-4 > > are supported[0]. The registered method extensions listed in the HTTP method registry: > > http://www.iana.org/assignments/http-methods/http-methods.xhtml > > seems to be without support. It should be fairly easy to add missing methods however. > > An alternative route would be to employ one of the many http client libraries out there. I'm partial to 'hackney' by Beno?t Chesnau, but there are numerous others. On top of my head: lhttpc (Two branches: Fred Hebert, and one by Erlang Solutions Ltd.), gun (Lo?c Hoguin), ibrowse (Chandrashekhar Mullaparthi), fusco (Diana Corbacho) and probably a couple I've forgotten. Different implementations enjoy different properties, so picking the one for your project may be something that takes a little time. > > I well known problem with the built-in httpc is that it has problems when experiencing high load. But I've used it successfully for a low-traffic site for years (only GET requests though). > > [0] though I have my doubts about CONNECT. > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 19 18:10:55 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Mon, 19 Jan 2015 09:10:55 -0800 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? Message-ID: Hi I am new to this and I am totally surprised how this works I have a server (gru) and 3 minions talking to gru. But none of the minions ever talked to each other. Still when I run nodes() on any of the minions tells me which other minions are connected. *Gru* (gru@REDACTED)7> nodes(). ['minion1@REDACTED','minion2@REDACTED', 'minion3@REDACTED'] (gru@REDACTED)8> *minion1* (minion1@REDACTED)6> nodes(). ['gru@REDACTED','minion2@REDACTED', 'minion3@REDACTED'] *minion2* (minion2@REDACTED)2> nodes(). ['gru@REDACTED','minion1@REDACTED', 'minion3@REDACTED'] (minion2@REDACTED)3> *minion3* (minion3@REDACTED)2> nodes(). ['gru@REDACTED','minion2@REDACTED', 'minion1@REDACTED'] (minion3@REDACTED)3> How does this process work? What's going on behind the scenes? Thanks + Harit -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Jan 19 18:12:04 2015 From: g@REDACTED (Garrett Smith) Date: Mon, 19 Jan 2015 11:12:04 -0600 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> Message-ID: Honestly, who sits back and relies solely on a type system to verify the correct behavior of a high risk, socially and legally impacting application (whatever that means)? Use Haskell. Use C++. Use Fortran! Who cares?? If you don't know what specific problem you're going to face enough to narrow your language choice, I suggest you stop worrying about language choice and start building your app with whatever language you're most proficient in. On Mon, Jan 19, 2015 at 11:01 AM, xu xiut wrote: > For those who were wondering what type of exchange I'm wanting to build, > it's a bitcoin exchange with derivative contracts. I asked in the IRC room > for some more input, and here's what I received: > > re:failure semantics of Erlang not being appropriate for an exchange > < tristan_1> failure/recovery in Erlang has nothing to do with that > < tristan_1> no matter the language/framework you have to decide how to > handle these things. there isn't a way you can possible write a program > that it won't crash at some point > > < asonge> you can design your app so that you log all requests, and make > sure that is as tight as possible, and that you can replay them if there's > a crash (for instance) > > There was an interesting suggestion to use OCaml or Haskell instead of > Erlang: > < digitalmentat> I would pick a language that gives more formal > verification of the software properties > < digitalmentat> Haskell and OCaml would be great picks > < digitalmentat> I use Haskell in production for my startup and 80% of the > dumb shit humans do (including me) is caught by that language's type-system > < digitalmentat> anything financial is a risky business to get into and > most developers aren't aware of the social impact nor the legal > ramifications if they fuck something up > > My question is, would Dialyzer not be able to offer equivalent > functionality? > Would using Erlang as a wrapper around a well-typed core work out okay, > does anyone here have experience they could share on this? I've never done > this, so it would be a completely new process for me. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Mon Jan 19 18:14:47 2015 From: jon@REDACTED (Jon Schneider) Date: Mon, 19 Jan 2015 17:14:47 -0000 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? In-Reply-To: References: Message-ID: <3ad16342158c1164f4b67994470e137c.squirrel@mail.jschneider.net> This is what erl does unless you start it with -connect_all false It is documented. Jon > > (minion3@REDACTED)2> nodes(). > > ['gru@REDACTED','minion2@REDACTED', > > 'minion1@REDACTED'] > > (minion3@REDACTED)3> > > > > How does this process work? What's going on behind the scenes? > > Thanks > + Harit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Mon Jan 19 18:24:29 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 19 Jan 2015 18:24:29 +0100 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> Message-ID: On Mon, Jan 19, 2015 at 6:01 PM, xu xiut wrote: > For those who were wondering what type of exchange I'm wanting to build, > it's a bitcoin exchange with derivative contracts The key requirement to ask is: what will be your major worry when writing such an exchange? Different languages will make it either easy or hard to solve certain problems in your domain. So you need to pick a language which will put you in a position where you have the right strength and can ignore the weaknesses due to them not being a factor. Erlang is an excellent choice if you value predictable latency and highly robust operation, even in the event of considerable load on the system. Here, your main vehicle for scaling would be to buy more cores. It tends to be a bad choice if you have a small computational kernel which takes all the time. In that event, you need a language where you can tune the tight kernel for speed, while ignoring everything else. OCaml is an excellent choice if you believe the problem space itself requires complicated abstractions where you are likely to make a mistake at the type-structural level of the program. If you use the type system cleverly, you can arrange such that you cannot make mistakes by accident on how to program the exchange correctly. OCaml is pretty fast to boot, and you can get decent asynchronous behaviour with core.async. Otoh, you won't be able to seamlessly scale in more than a single core and if this proves to be a bottleneck in your system, you have to write code all of a sudden to handle it. Java + LMAX/Disruptor has its share of problems. Go is excellent when it comes to multi-core scaling and computational processing power. But it severely lacks when it comes to robust behaviour. My bet would definitely be on either Erlang or OCaml. The former because there is a nice fit with the problem domain. The latter because it is a favorite among high-frequency-trading companies, and its abstraction features are a well-known advantage in that space. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Mon Jan 19 19:03:45 2015 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Mon, 19 Jan 2015 15:03:45 -0300 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? In-Reply-To: <3ad16342158c1164f4b67994470e137c.squirrel@mail.jschneider.net> References: <3ad16342158c1164f4b67994470e137c.squirrel@mail.jschneider.net> Message-ID: Any sufficiently advanced technology is indistinguishable from magic --Clarks three laws On Mon, Jan 19, 2015 at 2:14 PM, Jon Schneider wrote: > This is what erl does unless you start it with > > -connect_all false > > It is documented. > > Jon > > > > > > (minion3@REDACTED)2> nodes(). > > > > ['gru@REDACTED','minion2@REDACTED', > > > > 'minion1@REDACTED'] > > > > (minion3@REDACTED)3> > > > > > > > > How does this process work? What's going on behind the scenes? > > > > Thanks > > + Harit > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Jan 20 04:17:31 2015 From: mihai@REDACTED (Mihai Balea) Date: Mon, 19 Jan 2015 22:17:31 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> Message-ID: > On Jan 19, 2015, at 12:12 PM, Garrett Smith wrote: > > If you don't know what specific problem you're going to face enough to narrow your language choice, I suggest you stop worrying about language choice and start building your app with whatever language you're most proficient in. This is solid advice. Write your first version in whatever you feel most comfortable. Ideally something that will give you a quick turnaround. Then be prepared to scrap it and rewrite, once you figure out exactly what you need to address. Mihai From mihai@REDACTED Tue Jan 20 04:54:13 2015 From: mihai@REDACTED (Mihai Balea) Date: Mon, 19 Jan 2015 22:54:13 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> Message-ID: <38756B58-F433-4D47-814A-A72860458B0A@hates.ms> > Erlang is an excellent choice if you value predictable latency and highly robust operation, even in the event of considerable load on the system. I would tend to disagree with the first part. Any garbage collected language will not offer predictable latency - in a real time sense. Granted, the particulars of GC in Erlang make it so the variation in latency is small and localized. There is no ?stop the world? event, but each process is prone to minute GC delays. Depending on application, these may or may not matter. In the case of a high volume, low latency exchange, they will. I agree 100% with the second part. Out of the box robustness features in Erlang/OTP are second to none. > Here, your main vehicle for scaling would be to buy more cores. At some point time the general wisdom was the Beam VM would no longer scale linearly past 16 cores or so. Is that still the case in R17? Mihai From drormein@REDACTED Tue Jan 20 10:32:35 2015 From: drormein@REDACTED (Dror Mein) Date: Tue, 20 Jan 2015 09:32:35 +0000 (UTC) Subject: [erlang-questions] tidier Message-ID: <1539785936.3770384.1421746355533.JavaMail.yahoo@jws10605.mail.bf1.yahoo.com> does anyone know where to find the tidier application?there is a link to a webpage that i get and erl_error from.? -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Tue Jan 20 11:10:14 2015 From: davidnwelton@REDACTED (David Welton) Date: Tue, 20 Jan 2015 11:10:14 +0100 Subject: [erlang-questions] Graphical display of messages In-Reply-To: References: Message-ID: And here's something I hacked up to do sequence diagrams, thanks to a Javascript library: http://davidw.github.io/tracemap/gallery/sequence.html On Fri, Jan 16, 2015 at 12:38 PM, David Welton wrote: > I threw this together, for the heck of it: > > https://github.com/davidw/tracemap -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From johanmon@REDACTED Tue Jan 20 11:26:38 2015 From: johanmon@REDACTED (Johan Montelius) Date: Tue, 20 Jan 2015 11:26:38 +0100 Subject: [erlang-questions] floating point exception when accuracy exceeds limit Message-ID: This might be no news to anyone who is doing a lot of floating point arithmetics but it made me scratch my head. 7> (6.0e150*6.0e150). 3.6000000000000004e301 8> (6.0e154*6.0e154). ** exception error: an error occurred when evaluating an arithmetic expression in operator */2 called as 6.0e154 * 6.0e154 Hmm, would it not be better to answer with the highest possible accuracy? Johan -- Assoc. Prof. Johan Montelius Software and Computer Systems School of Information and Communication Technology KTH - Royal Institute of Technology From sverker.eriksson@REDACTED Tue Jan 20 12:09:00 2015 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 20 Jan 2015 12:09:00 +0100 Subject: [erlang-questions] floating point exception when accuracy exceeds limit In-Reply-To: References: Message-ID: <54BE374C.6030609@erix.ericsson.se> Largest possible floating-point number with double precision is 1.7976931348623157e308 http://en.wikipedia.org/wiki/Double-precision_floating-point_format /Sverker On 01/20/2015 11:26 AM, Johan Montelius wrote: > > This might be no news to anyone who is doing a lot of floating point > arithmetics but it made me scratch my head. > > > 7> (6.0e150*6.0e150). > 3.6000000000000004e301 > > 8> (6.0e154*6.0e154). > ** exception error: an error occurred when evaluating an arithmetic > expression > in operator */2 > called as 6.0e154 * 6.0e154 > > > Hmm, would it not be better to answer with the highest possible accuracy? > > Johan > > From eriksoe@REDACTED Tue Jan 20 12:10:42 2015 From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=) Date: Tue, 20 Jan 2015 12:10:42 +0100 Subject: [erlang-questions] floating point exception when accuracy exceeds limit In-Reply-To: References: Message-ID: What exactly do you mean by "accuracy"? In very few applications would it "be better" to get an answer which is orders of magnitude from the correct result, which is what you'd likely get if you handled all overflows with "use the maximum representable value". (For a better version of "No", wait till the sun reaches NZ; O'Keefe usually has something eloquent to say then. :-)) /Erik Den 20/01/2015 11.49 skrev "Johan Montelius" : > > This might be no news to anyone who is doing a lot of floating point > arithmetics but it made me scratch my head. > > > 7> (6.0e150*6.0e150). > 3.6000000000000004e301 > > 8> (6.0e154*6.0e154). > ** exception error: an error occurred when evaluating an arithmetic > expression > in operator */2 > called as 6.0e154 * 6.0e154 > > > Hmm, would it not be better to answer with the highest possible accuracy? > > Johan > > > -- > Assoc. Prof. Johan Montelius > Software and Computer Systems > School of Information and Communication Technology > KTH - Royal Institute of Technology > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Tue Jan 20 12:25:55 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 20 Jan 2015 12:25:55 +0100 Subject: [erlang-questions] "What's cooking in Erlang/OTP" Message-ID: <480BBDA9-BD27-4F8E-B26C-CF50998B66E6@gmail.com> Hello, What happened to that weekly digest of patches that are currently ongoing in Erlang? Now that we have an official community manager, shouldn't it be brought back? Regards, From tuncer.ayaz@REDACTED Tue Jan 20 12:26:51 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 20 Jan 2015 12:26:51 +0100 Subject: [erlang-questions] tidier In-Reply-To: <1539785936.3770384.1421746355533.JavaMail.yahoo@jws10605.mail.bf1.yahoo.com> References: <1539785936.3770384.1421746355533.JavaMail.yahoo@jws10605.mail.bf1.yahoo.com> Message-ID: On Tue, Jan 20, 2015 at 10:32 AM, Dror Mein wrote: > does anyone know where to find the tidier application? there is a > link to a webpage that i get and erl_error from. http://tidier.softlab.ntua.gr/ Looks like it fails to find erlyweb:out and therefore doesn't load. Kostis? From davidnwelton@REDACTED Tue Jan 20 12:28:43 2015 From: davidnwelton@REDACTED (David Welton) Date: Tue, 20 Jan 2015 12:28:43 +0100 Subject: [erlang-questions] "What's cooking in Erlang/OTP" In-Reply-To: <480BBDA9-BD27-4F8E-B26C-CF50998B66E6@gmail.com> References: <480BBDA9-BD27-4F8E-B26C-CF50998B66E6@gmail.com> Message-ID: It would be cool to have something like that, plus some other news, to keep track of what's going on with Erlang, for the casual observer. https://lobste.rs/s/bjcb9s/this_week_in -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From vladdu55@REDACTED Tue Jan 20 13:00:35 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 20 Jan 2015 13:00:35 +0100 Subject: [erlang-questions] embedded Java clients? Message-ID: Hi all, I am considering to make some improvements to jinterface and realized that I wasn't thinking about the possibility that the library is used in embedded devices (with the whole Internet of Things stuff going on). Embedded Java is quite restricted compared to the desktop SDK, so any improvement might cause problems... My questions are: - should jinterface target even embedded clients? Did it ever? - if yes, does OTP test against Java ME? - does anyone work with embedded Java clients, or knows about someone that does? Nowadays, Erlang can run on most embedded devices, so hopefully having embedded Java clients is something of the past. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From johanmon@REDACTED Tue Jan 20 11:57:33 2015 From: johanmon@REDACTED (Johan Montelius) Date: Tue, 20 Jan 2015 11:57:33 +0100 Subject: [erlang-questions] floating point exception when accuracy exceeds limit In-Reply-To: References: Message-ID: Ahh ignore, was thinking about 6.0e-154 Johan On Tue, 20 Jan 2015 11:26:38 +0100, Johan Montelius wrote: > > This might be no news to anyone who is doing a lot of floating point > arithmetics but it made me scratch my head. > > > 7> (6.0e150*6.0e150). > 3.6000000000000004e301 > > 8> (6.0e154*6.0e154). > ** exception error: an error occurred when evaluating an arithmetic > expression > in operator */2 > called as 6.0e154 * 6.0e154 > > > Hmm, would it not be better to answer with the highest possible accuracy? > > Johan > > -- Assoc. Prof. Johan Montelius Software and Computer Systems School of Information and Communication Technology KTH - Royal Institute of Technology From davidnwelton@REDACTED Tue Jan 20 13:36:48 2015 From: davidnwelton@REDACTED (David Welton) Date: Tue, 20 Jan 2015 13:36:48 +0100 Subject: [erlang-questions] embedded Java clients? In-Reply-To: References: Message-ID: > - does anyone work with embedded Java clients, or knows about someone that > does? It's not very 'embedded' in terms of its Java, but Android is pretty important for the folks I work with. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From lostcolony@REDACTED Tue Jan 20 14:14:41 2015 From: lostcolony@REDACTED (Christopher Phillips) Date: Tue, 20 Jan 2015 08:14:41 -0500 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? Message-ID: Essentially connecting nodes assumes you want a fully meshed topology, and is implemented as such. When your main gru node talks to minion1, the fact minion1 exists is registered (as evidenced by executing nodes() on gru; vice versa also applies). When gru then establishes connection with minion2, it communicates the existence of minion1 to minion2, etc. This can, as indicated, be disabled with connect_all being set to false, but that also prevents global registration of processes. You can also set the minion nodes to be hidden nodes; this changes some of the semantics necessary to interact with them. > > Date: Mon, 19 Jan 2015 15:03:45 -0300 > From: Thomas Elsgaard > To: Jon Schneider > Cc: erlang questions > Subject: Re: [erlang-questions] How does running nodes() on host knows > about all connected nodes? > Message-ID: > Zk169Ui8zfAVgmG8VyfYi2X9p7qMvWoZwA@REDACTED> > Content-Type: text/plain; charset="iso-8859-1" > > Any sufficiently advanced technology is indistinguishable from magic > > --Clarks three laws > > On Mon, Jan 19, 2015 at 2:14 PM, Jon Schneider > wrote: > > > This is what erl does unless you start it with > > > > -connect_all false > > > > It is documented. > > > > Jon > > > > > > > > > > (minion3@REDACTED)2> nodes(). > > > > > > ['gru@REDACTED','minion2@REDACTED', > > > > > > 'minion1@REDACTED'] > > > > > > (minion3@REDACTED)3> > > > > > > > > > > > > How does this process work? What's going on behind the scenes? > > > > > > Thanks > > > + Harit > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jan 20 14:28:05 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 20 Jan 2015 14:28:05 +0100 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: <38756B58-F433-4D47-814A-A72860458B0A@hates.ms> References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> <38756B58-F433-4D47-814A-A72860458B0A@hates.ms> Message-ID: On Tue, Jan 20, 2015 at 4:54 AM, Mihai Balea wrote: > I would tend to disagree with the first part. Any garbage collected > language will not offer predictable latency - in a real time sense. This might have been true back in the day, but it surely is not true anymore. First of all, you have to make the distinction between hard and soft realtime operation. The rule of hard realtime is that if you miss a deadline, the program is faulty. In soft realtime it is okay, as long as the average (or perhaps median) hits the deadline. In other words, soft realtime systems are allowed to fail occasionally, and Erlang has always been a soft realtime system. The first important point is that realtime garbage collectors do exist. They are insanely complex, but they do exist and can do very well. The other important point is that for some workloads, Erlang soundly beats non-GC'ed languages in the latency game, which is food for thought if you are of the opinion that GC'ed languages all have problematic latency. The Techempower benchmark of webservers, https://www.techempower.com/benchmarks/ shows a latency where the deviation of "cowboy", an Erlang web server, is on par with Ur/Web, and the maximal latency is *far* better than its competition. And do mind that the competition is sometimes written in C++. Also, note that I specifically wrote "predictable latency" over "low latency", which is a different beast. The goal here is that if the latency is 1ms, then the latency is probably going to be 1ms plus/minus a small variation. The C++ web servers can handle a request in perhaps 0.1ms, but then their variation is large and some requests might take 5, 10, or 134ms. Chances are that the problems you will face using Erlang are much smaller compared to a C++ solution, when all is being measured up. Of course, the C++ solution processes at a much higher volume, but doing so, it has worse predictable latency. Then again, there are parts of the Erlang/OTP solution which makes it somewhat of a GC/non-GC hybrid. ETS tables are not garbage collected. A terminating process doesn't need garbage collection, and you can sometimes arrange it such that the process has enough initial heap to never collect. Processes with small memory footprint have very predictable pause times since they are bounded by two-space copying traversal time. If you know what you are doing, it is entirely possible to avoid long pause times by a little attention to how you program the system. I suspect the reason boils down to head-of-line blocking due to cooperative scheduling. In most C++ solutions, what you are aiming for is to process each work unit as fast as possible, notwithstanding what code path you took. If there is a mistake in one path which imposes latency on the system, then your latency suffers on a global scale. Not so in Erlang (and Go 1.3.x+), where the process would simply be scheduled away, so only the slow code path suffers. Then, there is the question of high volume. Systems tend to operate differently under massive volume and load. There, a small mistake in a data structure is what is going to cost you the desired latency. Or a pathological situation, where a hash table ends up with too many conflicts. The trade-off is to use a more robust data structure, but this comes at a performance price. You got the predictable latency, but lost the very fast operation. The perhaps most glaring omission is to ignore quantities of "volume" and "low latency". To some people, low latency means FPGA implementations, because C is too slow. To some people, a webserver taking 20 reqs/s over the day on average is a loaded webserver. There are limits to all system designs, where language choice is but one. As for the 16 core "limit", it is a myth on a modern OTP 17.x. Of course, such measurements are dependent on the particular benchmark, and if you hit lock contention in the OTP subsystem. But most of the old limits have been lifted systematically and you are looking at at least 64 cores now, and perhaps closer to 128. That said, it depends. Currently, there is a bottleneck around the timer wheel, something which is being addressed for release 18.x. It is a contended lock which WhatsApp also worked around in their solutions. Perhaps somewhat enlightening, the single core performance can sometimes suffer from efficient multi-core support. Erlang currently leans such that fast multi-core performance is more important than the single executing thread. I think my main point still stands. In Erlang, the problem could be to get a desired low latency. But this is easy to test for early on in development, and eventually implement parts of the code as a NIF to get the desired speed. I don't see volume as a problem at all. In a solution written in "fast" languages like Java, C#, C or C++, the problems are far more subtle and will begin showing themselves long into the process of writing the software. And it will then be hard to change the solution because the investment is now sunk cost. Incidentally, this is why languages with fast prototyping properties are so powerful (OCaml specifically comes to mind here). -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Tue Jan 20 15:03:10 2015 From: mihai@REDACTED (Mihai Balea) Date: Tue, 20 Jan 2015 09:03:10 -0500 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? In-Reply-To: References: Message-ID: > On Jan 19, 2015, at 12:10 PM, Harit Himanshu wrote: > > > I have a server (gru) and 3 minions talking to gru. But none of the minions ever talked to each other. > > Still when I run nodes() on any of the minions tells me which other minions are connected. > > How does this process work? What's going on behind the scenes? When a node connects to another node in a running cluster, it gets a list of nodes already in the cluster. It then establishes connections to every one of them automatically. You end up with a fully connected mesh. That is the default behavior, and the reason why large erlang clusters become problematic (n*n connections). There are ways to override this behavior at a loss of some functionality. Mihai From mihai@REDACTED Tue Jan 20 15:17:22 2015 From: mihai@REDACTED (Mihai Balea) Date: Tue, 20 Jan 2015 09:17:22 -0500 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? In-Reply-To: References: Message-ID: <2C0E47DC-8C83-4E77-A9D2-C11FFC4714A5@hates.ms> > On Jan 20, 2015, at 9:03 AM, Mihai Balea wrote: > > When a node connects to another node in a running cluster, it gets a list of nodes already in the cluster. It then establishes connections to every one of them automatically. You end up with a fully connected mesh. That is the default behavior, and the reason why large erlang clusters become problematic (n*n connections). There are ways to override this behavior at a loss of some functionality. Just to be accurate, the number of connection is given by the formula n*(n-1)/2 From harit.subscriptions@REDACTED Tue Jan 20 18:18:25 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 20 Jan 2015 09:18:25 -0800 Subject: [erlang-questions] How does running nodes() on host knows about all connected nodes? In-Reply-To: <2C0E47DC-8C83-4E77-A9D2-C11FFC4714A5@hates.ms> References: <2C0E47DC-8C83-4E77-A9D2-C11FFC4714A5@hates.ms> Message-ID: That is very interesting to learn. Thanks a lot for explaining this to me. The follow up question then becomes, is it recommended to have such a mesh? (because I read word problematic). How do people in general connect to server? so they start with *-connect_all false* as mentioned? Thanks + Harit On Tue, Jan 20, 2015 at 6:17 AM, Mihai Balea wrote: > > > On Jan 20, 2015, at 9:03 AM, Mihai Balea wrote: > > > > When a node connects to another node in a running cluster, it gets a > list of nodes already in the cluster. It then establishes connections to > every one of them automatically. You end up with a fully connected mesh. > That is the default behavior, and the reason why large erlang clusters > become problematic (n*n connections). There are ways to override this > behavior at a loss of some functionality. > > Just to be accurate, the number of connection is given by the formula > n*(n-1)/2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From xiut.xu@REDACTED Tue Jan 20 21:37:26 2015 From: xiut.xu@REDACTED (xu xiut) Date: Tue, 20 Jan 2015 15:37:26 -0500 Subject: [erlang-questions] Is Erlang ideal for a global exchange? In-Reply-To: References: <8514848E-EED2-4EA1-9A02-C637732F9991@hates.ms> <54B1436C.7060008@gmail.com> <984D7F91-39CC-45FC-A30E-8FF5EB4B0D4D@hates.ms> <58A3CA17-A08B-44E1-B495-4896092458D6@hates.ms> <38756B58-F433-4D47-814A-A72860458B0A@hates.ms> Message-ID: There's a related thread about garbage collection and exchanges at https://www.reddit.com/r/programming/comments/12b1o6/why_program_in_erlang/c6trowm -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Tue Jan 20 22:19:19 2015 From: garry@REDACTED (Garry Hodgson) Date: Tue, 20 Jan 2015 16:19:19 -0500 Subject: [erlang-questions] tls triple handshake vulnerabilty In-Reply-To: References: <54B7E1B5.3080509@research.att.com> Message-ID: <54BEC657.4080206@research.att.com> thanks. i'll try that. On 1/20/15 11:24 AM, Ingela Andin wrote: > Hi! > > Well you could set {reuse_sessions, boolean()} to false, it could have > some performance penalty but destroys the prerequisites for the attack. > > Regards Ingela Erlang/OTP team - Ericsson AB > > On Thu, Jan 15, 2015 at 4:50 PM, Garry Hodgson > wrote: > > Are the erlang ssl libraries subject to the TLS triple handshake > vulnerability described at https://secure-resumption.com? > If so, are there configuration options that can mitigate the risk? > I've read through the erlang ssl docs, but don't understand the > subject well enough to tell. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Tue Jan 20 22:13:40 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Tue, 20 Jan 2015 22:13:40 +0100 Subject: [erlang-questions] "What's cooking in Erlang/OTP" In-Reply-To: <480BBDA9-BD27-4F8E-B26C-CF50998B66E6@gmail.com> References: <480BBDA9-BD27-4F8E-B26C-CF50998B66E6@gmail.com> Message-ID: <54BEC504.6040900@wanadoo.fr> Le 20/01/2015 12:25, Anthony Ramine a ?crit : > Hello, > > What happened to that weekly digest of patches that are currently ongoing in Erlang? Now that we have an official community manager, shouldn't it be brought back? > Hello, Note, that the README.md in https://github.com/erlang/otp still refer to "What's cooking in Erlang/OTP", as well to the 'pu' branch . And 'pu' branch did not changed since 1 Aug 2013 ! This can be puzzling for new comers. I suppose this page, at least, should be updated to explain the current "Contribution process" . I remember http://www.erlang.org/news/70 that was great , with a macro planning on next realeases. this should be given at each new releases MHO. Regards From watson.timothy@REDACTED Wed Jan 21 11:40:33 2015 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Jan 2015 10:40:33 +0000 Subject: [erlang-questions] Fwd: Question about registry semantics for the remote processes. In-Reply-To: References: Message-ID: Can anyone shed any light on this?? On 14 January 2015 at 10:57, Alexander V Vershilov < alexander.vershilov@REDACTED> wrote: > Hello. > > I want to understand the reasons for the current semantics > for saving remote processes in local registry. > > According to the specification for register function [1] it's > impossible to save remote process in local registry: > > Failure: badarg if PidOrPort is not an existing, > local process or port, if RegName is already in use, > if the process or port is already registered (already has a name), > or if RegName is the atom undefined. > > As a result it's not possible to construct expression Name ! Message, > where Name is evaluated to a atom stored in registry that points > to the remote process. > > Another approach is described in unified semantics paper (Section 2.3) > > However, for uniformity, in this semantics names can be registered > for remote processes (i.e., register(name,pid) does not fail if pid > is a remote process), and registering a local process at a remote > node is supported too (using the operation register(node,name,pid)). > As a consequence, when a message is sent to a remote node using the > syntax > {atom,node}!msg there is no guarantee that the process that should > receive the mes- > sage is located at node; thus it may be necessary to relay the message > to a process on yet another node > > So I have two questions: > > 1. Were there any technical reasons for forbidding remote pids in local > registry as we have in current erlang, or it's a historical reasons? > 2. If there are technical reasons for that, then why uniformed > specification > introduces such behaviour, or it provides some mechanisms to solve > those problems? > > One can argue that the reason is monitoring, and case when we are sending > to the remote process from registry that already dead, but I don't see any > difference with sending to remote Pid that already died, semantics of all > the functions and guarantees are the same. > > Thanks > > [1] http://www.erlang.org/doc/man/erlang.html#register-2 > [2] http://happy-testing.com/hans/papers/EW2010-UnifiedSemantics.pdf > > -- > Alexander > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Wed Jan 21 11:47:16 2015 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Wed, 21 Jan 2015 11:47:16 +0100 Subject: [erlang-questions] Kill a process after T seconds In-Reply-To: References: Message-ID: I think the problem is you don't kill spawned process but parent. The code should be my_spawn(Mod, Func, Args, Time) -> Pid = spawn(Mod, Func, Args), io:format("created new process: ~p.~n", [Pid]), timer:kill_after(Time * 1000, Pid), Pid. Note using timer:kill_after/2 instead of timer:kill_after/1 On Tue, Jan 13, 2015 at 7:55 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > The recent problem I worked on is > > Write a function my_spawn(Mod, Func, Args, Time) that behaves like > spawn(Mod, Func, Args) but with one difference. If the spawned process > lives for more than Time seconds, it should be killed. > > My attempt looks like > > my_spawn(Mod, Func, Args, Time) -> > Pid = spawn(Mod, Func, Args), > io:format("created new process: ~p.~n", [Pid]), > timer:kill_after(Time * 1000), > Pid. > > and when I run I see > > 1> c(error_handling). > > {ok,error_handling} > > 2> Pid1 = error_handling:my_spawn(area_server0, loop, [], 30). > > created new process: <0.39.0>. > > <0.39.0> > > 3> Pid1 ! {self(), {square, 10}}. > > {<0.32.0>,{square,10}} > > 4> receive X->X end. > > 100 > > 5> Pid1 ! {self(), {square, 20}}. > > {<0.32.0>,{square,20}} > > 6> receive Y->Y end. > > 400 > > ** exception error: killed > > 7> > > Looking for suggestions on the code or ways to write it better > > Thanks > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Wed Jan 21 13:45:42 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 21 Jan 2015 13:45:42 +0100 Subject: [erlang-questions] embedded Java clients? In-Reply-To: References: Message-ID: Hi! This is what information I gathered, it looks not so bad. If anyone knows better, please let me know. - There are many Java ME variants, some very old (for very small devices, like smartcards). I believe that these old small devices won't run jinterface anyway, but need heavy optimizations, so we can ignore them. The others have the libraries that we need most (the collections framework). - Some of the older ME variants require source code compatibility with older Java specifications. JInterface has been updated past those for some time now and since nobody complained as far as I know, I suppose that nobody uses it in that context and we're safe for now. - No problems for Android, it's sufficiently recent. - The only way that I can think of for making sure that unsupported libraries aren't starting to get used is by building against all the targets. This is resource intensive (both developer time and hardware setup) and I would guess that it won't make it very far from the bottom of OTP's priority list. So using jinterface in embedded projects should come with a disclaimer about that. In conclusion, we're safe for now. Expect some thoughts about future evolution in the not so far future :-) best regards, Vlad On Tue, Jan 20, 2015 at 1:00 PM, Vlad Dumitrescu wrote: > Hi all, > > I am considering to make some improvements to jinterface and realized that > I wasn't thinking about the possibility that the library is used in > embedded devices (with the whole Internet of Things stuff going on). > Embedded Java is quite restricted compared to the desktop SDK, so any > improvement might cause problems... > > My questions are: > - should jinterface target even embedded clients? Did it ever? > - if yes, does OTP test against Java ME? > - does anyone work with embedded Java clients, or knows about someone that > does? > > Nowadays, Erlang can run on most embedded devices, so hopefully having > embedded Java clients is something of the past. > > best regards, > Vlad > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aronisstav@REDACTED Wed Jan 21 17:10:37 2015 From: aronisstav@REDACTED (Stavros Aronis) Date: Wed, 21 Jan 2015 17:10:37 +0100 Subject: [erlang-questions] Dialyzer failing to find simple type error In-Reply-To: References: Message-ID: Well, this was what I ended up trying to improve in my master thesis back in 2010... (Mainly for Tobias:) The attempted solution was to take all the type constraints that Dialyzer generates, convert them to disjunctive normal form and then carefully assemble an 'intersection type', able to infer things like bar(1) -> 'foo';(2) -> 'bar'. Still far from things like though. For the provided examples, intersection types together with standard call refinement would produce the desired warnings (I had a brief moment of nostalgia trying it out with just erts in the PLT on: https://github.com/aronisstav/otp/tree/intersection): $ dialyzer test2.erl [...] test2.erl:5: Function main/0 has no local return test2.erl:7: The call test2:foo('bar') will never return since it differs in the 1st argument from the success typing arguments: ('foo') [...] done in 0m0.11s done (warnings were emitted) $ typer test2.erl %% File: "test2.erl" %% ----------------- -spec main() -> none(). -spec foo('foo') -> 'ok'. -spec bar(1) -> 'foo'; (2) -> 'bar'. Unfortunately, this extension made the analysis too slow (my dead-stupid conversion to disjunctive normal form leads to a lot of duplication of work), so it never went into OTP and as the code has been evolving significantly I would need to redo everything from scratch... Other than that, I fully agree with Tobias. Stavros On Thu, Jan 1, 2015 at 11:57 PM, Tobias Lindahl wrote: > Hi Philip, > > You are confusing Dialyzer with a static type checker (which it isn't [1]). > > In one of your examples, there is no problem, and in the other one, the > analysis would have to be pretty strong (and have a different type domain) > to find the problem. > > In test.erl, you have the success typings (written as specs) > > -spec bar() -> 1 | 0. > -spec foo(1) -> 'ok' > > So applying foo(bar()) _can_ succeed, which makes it perfectly fine in > Dialyzer's eyes. > > Similarly, in test2.erl you have the success typings > > -spec bar(any()) -> 'foo' | 'bar'. > -spec foo('foo') -> 'ok'. > > So applying > > ok = foo(bar(1)), > ok = foo(bar(2)). > > once again _can_ succeed (based on the success typings) so Dialyzer > remains quiet. > > Arguably, if Dialyzer's analysis was strong enough to infer the type > > -spec bar (1) -> 'foo'; > () -> 'bar', > > it could warn, but the above typing is way beyond Dialyzer. > > Tobias > > [1] > http://www.it.uu.se/research/group/hipe/dialyzer/publications/succ_types.pdf > > > 2015-01-01 17:24 GMT+01:00 Philip M?ller : > >> Hi there, >> >> a few days ago I wanted to demonstrate the dialyzer to a friend who is >> unfamiliar with erlang/otp. >> >> Unfortunately, my simplest demo case did not work. I made a github >> project for it at [1] which documents the steps taken. >> >> For me, the first case (test.erl) looks like dialyzer is ignoring >> -Wunmatched_returns completely. The second case (test2.erl) I don't >> understand at all. >> >> Can someone explain what's going on there? Or should I file a bug report? >> >> Best regards >> Philip >> >> [1]: https://github.com/exterm/dialyzer-fail-example >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Wed Jan 21 14:59:18 2015 From: radek@REDACTED (Rad Gruchalski) Date: Wed, 21 Jan 2015 08:59:18 -0500 Subject: [erlang-questions] Cowboy server, hackney client - SSL problems Message-ID: <22D6885CDB914F2291B946E72A07CA96@gruchalski.com> Hi everyone, This is my first question on this mailing list. I have hit a wall while writing a bit of software which does the following: I have a REST-like server running on cowboy which runs on SSL, I?m using a self signed certificate generated by my own certificate authority, the settings I?m using in cowboy are: [ { port, ? }, { cacertfile, absolute-path-to-the-public-cacert }, { certfile, absolute-path-to-cert.pem }, { keyfile, absolute-path-to-key.pem } ] Cowboy starts fine, any request coming from CURL or Chrome browser is working fine, the clients are served, no issues whatsoever. One of the parts of this software is a set of unit tests for the REST-like API. I am intending on using hackney for this. What happens is that when hackney client hits the API, cowboy fails with the following error message: [error] SSL: certify: ssl_alert.erl:92:Fatal error: bad certificate The error happens when hackney uses SSL options as cowboy server and with no SSL options. I have verified my certificates in the following way: added it as a trusted cert in Chrome and Chrome does not complain about anything regarding the certificate anymore. I?ve also done the following: openssl s_server -accept 8080 -cert ...cert.pem -key ...key.pem -CAfile ?_ca.crt Using default temp DH parameters Using default temp ECDH parameters ACCEPT -----BEGIN SSL SESSION PARAMETERS----- MHUCAQECAgMBBAIAOQQgzGrVgGBoyIRc9v3w3bqmiFQS3t6RjUfmaaa6AyCLsMgE MFf7Tw9pR21yhGRTzuhEr8tXfOnlWKkl08eRS3bhld2jhHm3PgqB/0hinTw/f4CT rqEGAgRUvyw9ogQCAgEspAYEBAEAAAA= -----END SSL SESSION PARAMETERS----- Shared ciphers:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:DHE-RSA-SEED-SHA:DHE-DSS-SEED-SHA:SEED-SHA:RC4-SHA:RC4-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA:EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC4-MD5 CIPHER is DHE-RSA-AES256-SHA Secure Renegotiation IS supported openssl s_client -connect localhost:8080 -cert ...cert.pem -key ...key.pem -CAfile ?_ca.crt CONNECTED(00000003) depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED verify error:num=18:self signed certificate verify return:1 depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED verify return:1 --- Certificate chain 0 s:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED i:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED --- Server certificate -----BEGIN CERTIFICATE----- MIIFhTCCA22gAwIBAgIJANOPgonG2vS4MA0GCSqGSIb3DQEBCwUAMHkxEjAQBgNV BAMTCWxvY2FsaG9zdDEYMBYGA1UECBMPUmhlaW5sYW5kLVBmYWx6MQswCQYDVQQG EwJERTEgMB4GCSqGSIb3DQEJARYRZGV2QGdvc3NpcGVybC5jb20xGjAYBgNVBAoU EWRldkBnb3NzaXBlcmwuY29tMB4XDTE1MDEyMTAxMjkyN1oXDTE2MDEyMTAxMjky N1oweTESMBAGA1UEAxMJbG9jYWxob3N0MRgwFgYDVQQIEw9SaGVpbmxhbmQtUGZh bHoxCzAJBgNVBAYTAkRFMSAwHgYJKoZIhvcNAQkBFhFkZXZAZ29zc2lwZXJsLmNv bTEaMBgGA1UEChQRZGV2QGdvc3NpcGVybC5jb20wggIiMA0GCSqGSIb3DQEBAQUA A4ICDwAwggIKAoICAQDSPEXcaq4gdKyB6nGmac91sLNW2ZfBqJOWmkCIpYQnGB27 EUQTsdxqTtDkfEXlNjf6o4NupytDMqx7lRdVHh+Cqv38S8/Sb9FtyYtsxab4X9hv vf063O455MKVGCeQGqOTmmQTfybCsiQAa8UYK/chS8wQeBLAIAAaVOcNtmEhbUpb OaOkwInrjfK9lemD5J8G3z1oUDoiuxwoepyrEWGsmDEWLQKWNJmD6RLeHANH1/UQ V0PNWqwwYPrkEp9hEgau25/NHrglE9OW1SJmL79Cy3DKvLGxwaH1U0K9vh4rEW3A Vc36/TCVSpWXkxMUUDYHFihmR2oxyXSgs6/XKWSeV+xJD7VogljVJxl1IzAYcjlV EbYT4KNqZaqAdSeriRAMSJ5LlZ+7/uknOfdqKcAwUUwdYKKdb1IHpiRmGjFso0zF icMdKudNLZu854PIkSslh2/SJUhj2fiNwZ8QaGrnklJSEUE6jgjPBcAWbkY6M0Xm QUON8LcuDGivh1HPOzgiHiWnkd2zGqwAg1u5vsLcraspXBPfgQc+v+oek5xi5OVL K4YvB8iVf1dg8NQ1Xm3S+aPthi/eI/6UxSx3hSV2fPfydJ8JHoeRx2YWV3Q3dClG iJJLg36pOoFmO6nMwDDIZpSvPwZsfa/S9A69ZuQMv7ra6o8rzfXo7ZtSvy2X0QID AQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCoinw+YKNs 0amglVRGcgDJ4kQlz4HN9Z7phmc5MoEUzeo1vSUH/dvKu9tfgzdArRndXIVMOhqM UiOJvfIFua+IZIO2VWqSgIiyuJ1NYYk3trceVu9BZFb8GC0zymG54kXfzcCejDxZ K8hm3eSnTqHJWSFwqGW8OGDzoxfJJCtn6hivZJcyvz6lV0Gp+GxkKHqtDZxQpjCl 4I7qyz9i7KOfARtca3VTNwQfaMWZSBZ9A7DoYEFvpIGu0faNmy4zbIM7eIKNhBid v4xqkQHG8f/XTRhNYebBgCzmittKlQc61NodCGnQJXTcS/xZO8NVweAMmqzaR3A9 BAn9PZ1b/2hScDSAGtJEon3EToqzb5g2ijbKknshbNTe+Nt3J+Riyll115LZRSxg qwFnPv56WRvL1wJLfYW/S2q9Vwa/RzTiPpPBHPdJw7TETh8Z3UshjSeDVDGmdiQC V+vFxdMAVtvV7krfuwLa04NQ3mG+WrWfnbPUli7OzZ7NlUxG3Koc0QahABrIahlo ZDe1yV1OxeSJZByBcM57N04LVDkK29TuqaQsieW/A80g9EN1Q6Y3AJOLhhyZ1Euq bDg24BJ0P6FiuBz/WFtDRmjwfxhmPiZICfBqpQeltgKsungE4Lg8sMuxPFc7pJpm hIedZIKnykXzT2wkzRuBz9F9EqyYNRA7BQ== -----END CERTIFICATE----- subject=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED issuer=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED --- No client certificate CA names sent --- SSL handshake has read 2244 bytes and written 264 bytes --- New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA Server public key is 4096 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1 Cipher : DHE-RSA-AES256-SHA Session-ID: CC6AD5806068C8845CF6FDF0DDBAA6885412DEDE918D47E669A6BA03208BB0C8 Session-ID-ctx: Master-Key: 57FB4F0F69476D72846453CEE844AFCB577CE9E558A925D3C7914B76E195DDA38479B73E0A81FF48629D3C3F7F8093AE Key-Arg : None Start Time: 1421814845 Timeout : 300 (sec) Verify return code: 18 (self signed certificate) ? ssl:versions(). [{ssl_app,"5.3.8"}, {supported,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}, {available,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}] I am trying to understand why Erlang has a problem with it. I read that Erlang had a problem with sha256 signed certs but I am not sure if this is still a problem? I am using OTP 17.4 for these tests. The error happens in some unidentified place, I tried modifying OTP to log the error message in a place where there?s only reference to ?BAD_CERTIFICATE (via call to path_validation_alert function in ssl_handshake.erl) but I get no log - the modification can be seen here: https://github.com/gossiperl/otp/blob/b446bcc3ece0367c96b54af96577b23e4fa43ee4/lib/ssl/src/ssl_handshake.erl#L433 and I am 100% that the installation of Erlang I?m using is the one with the modified code. Of course it could be an issue with my certificate. I would appreciate any pointers. Kind regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) +4917685656526 Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Wed Jan 21 17:56:42 2015 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 21 Jan 2015 16:56:42 +0000 Subject: [erlang-questions] eunit timeout setting doesn't work? Message-ID: I have some smoke tests, implemented in the 'smoke' module, and I run them as follows: Tests = [smoke], eunit:test(Tests, [verbose]). And everything works. Except that when I point the tests at a remote server, they start taking a lot longer to run, so eunit kills them after 5 seconds. I've tried: Tests = [smoke], eunit:test({timeout, 600, Tests}, [verbose]). ...and it doesn't seem to make any difference. What can I do extend the allowed time for each of my tests to something more than 5 seconds? I found http://stackoverflow.com/q/1847004/8446, which suggests wrapping each test in a {timeout, but I don't want to use that, because I'm not using _assertWhatever in my tests. They're *cough* not exactly "unit" tests... Learn You Some Erlang suggests that all_test_() -> {timeout, 600, [fun first/0, fun second/0]}. ...ought to work. But it doesn't. I still get a timeout after approx 5 seconds. What am I doing wrong? From vincent.dephily@REDACTED Wed Jan 21 18:05:24 2015 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Wed, 21 Jan 2015 18:05:24 +0100 Subject: [erlang-questions] Figuring out proper ssl certificate settings with 17.3 Message-ID: <4306941.X6676PlrAe@moltowork> Hi, we're trying to upgrade from 17.1 to 17.4 and are hitting authentication failures, apparently due to the changes that introduced the partial_chain option. First some background: * We want to authenticate both server and client * We manage certificate ourselves, and don't care about the OS-supplied CAs. * The signing hierarchy is TopCA -> ServerCA -> Account1ServerCert -> Account2ServerCert -> ... -> ClientCA -> Account1ClientCert -> Account1Device1Cert -> Account1Device2Cert -> ... -> Account2ClientCert -> ... * The server and test client are in erlang, the real clients are in various languages (and we can't afford to update them). What we used to do, works fine in 17.1: server: {cert, Account1ServerCert}, {cacerts, [ClientCA]} client: {cert, Account1Device1Cert}, {cacerts, [ServerCA]} (in some cases we used the 'file' version of the option, but that doesn't change the semantics AFAIK). But that gives us an "unknown ca" error with 17.4. The minimum list of cacerts that seems to work is [ServerCA,TopCA] for the server and [clientCA,TopCA] for the client, but: * Having to specify the TopCA is an administration pain (lots of configs to change, and need to take the file out of the vault). * I'd rather not send unnecessary bytes over the wire. Ideally I don't want to send any root/intermediate CA, only the leaf cert. * Huh ? Why does ClientCA not need to be in ther server's cacerts (and vice- versa) ? I guess some confusion comes from the fact that the cacerts option is used for two different things (authenticating the cert received by the peer, and sending intermediate CAs to the peer to help it authenticate me). Maybe the option could be split in two ? AFAIU, the new requirement for TopCA to be present comes from a tightening of the check mandated by the RFC : the certificate chain (which one ? the one sent by the client or the one set up locally) needs to end with a self-signed cert. I'll pass on the weirdness of that requirement (I'll trust whichever CA I decide, wether it's self-signed or not, damnit), as it looks like partial_chain is expressely designed to bypass it. But I wasn't able to write one that worked *and* didn't look like a big security hole. Could somebody write an example partial_chain fun that works for my usecase ? Ideally, the docs could do with some clarifications : * The spec for partial_chain has a typo (at least a missing '}') Again, and example in the docs wouldn't go amiss. * It's not clear which chain it acts uppon * Is the semantic of cacertfile really different for the server and the client ? From my knowledge of TLS, it should be symetrical (even if the common real-world case is that the client doesn't authenticate itself), so the option should be in the common section (next to cacerts with a note that one is equivalent to the other). Thanks in advance. -- Vincent de Phily From bchesneau@REDACTED Wed Jan 21 18:14:56 2015 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 21 Jan 2015 18:14:56 +0100 Subject: [erlang-questions] Cowboy server, hackney client - SSL problems In-Reply-To: <22D6885CDB914F2291B946E72A07CA96@gruchalski.com> References: <22D6885CDB914F2291B946E72A07CA96@gruchalski.com> Message-ID: How are you using curl against your custom certificate? Hackney is very strict by default unless you told it to be not and check only against valid cerificates. There are too way to bypass this protection: - use the option `insecure` ewhen you do a request - use your own SSL options. Feel free to join me directly if you need it. - benoit On Wed, Jan 21, 2015 at 2:59 PM, Rad Gruchalski wrote: > Hi everyone, > > This is my first question on this mailing list. I have hit a wall while > writing a bit of software which does the following: I have a REST-like > server running on cowboy which runs on SSL, I?m using a self signed > certificate generated by my own certificate authority, the settings I?m > using in cowboy are: > > [ { port, ? }, > { cacertfile, absolute-path-to-the-public-cacert }, > { certfile, absolute-path-to-cert.pem }, > { keyfile, absolute-path-to-key.pem } ] > > Cowboy starts fine, any request coming from CURL or Chrome browser is > working fine, the clients are served, no issues whatsoever. > > One of the parts of this software is a set of unit tests for the REST-like > API. I am intending on using hackney for this. What happens is that when > hackney client hits the API, cowboy fails with the following error message: > > [error] SSL: certify: ssl_alert.erl:92:Fatal error: bad certificate > > The error happens when hackney uses SSL options as cowboy server and with > no SSL options. > > I have verified my certificates in the following way: added it as a > trusted cert in Chrome and Chrome does not complain about anything > regarding the certificate anymore. I?ve also done the following: > > openssl s_server -accept 8080 -cert ...cert.pem -key ...key.pem > -CAfile ?_ca.crt > Using default temp DH parameters > Using default temp ECDH parameters > ACCEPT > -----BEGIN SSL SESSION PARAMETERS----- > MHUCAQECAgMBBAIAOQQgzGrVgGBoyIRc9v3w3bqmiFQS3t6RjUfmaaa6AyCLsMgE > MFf7Tw9pR21yhGRTzuhEr8tXfOnlWKkl08eRS3bhld2jhHm3PgqB/0hinTw/f4CT > rqEGAgRUvyw9ogQCAgEspAYEBAEAAAA= > -----END SSL SESSION PARAMETERS----- > Shared > ciphers:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:DHE-RSA-SEED-SHA:DHE-DSS-SEED-SHA:SEED-SHA:RC4-SHA:RC4-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA:EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC4-MD5 > CIPHER is DHE-RSA-AES256-SHA > Secure Renegotiation IS supported > > openssl s_client -connect localhost:8080 -cert ...cert.pem -key > ...key.pem -CAfile ?_ca.crt > CONNECTED(00000003) > depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > verify error:num=18:self signed certificate > verify return:1 > depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > verify return:1 > --- > Certificate chain > 0 s:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > i:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > --- > Server certificate > -----BEGIN CERTIFICATE----- > MIIFhTCCA22gAwIBAgIJANOPgonG2vS4MA0GCSqGSIb3DQEBCwUAMHkxEjAQBgNV > BAMTCWxvY2FsaG9zdDEYMBYGA1UECBMPUmhlaW5sYW5kLVBmYWx6MQswCQYDVQQG > EwJERTEgMB4GCSqGSIb3DQEJARYRZGV2QGdvc3NpcGVybC5jb20xGjAYBgNVBAoU > EWRldkBnb3NzaXBlcmwuY29tMB4XDTE1MDEyMTAxMjkyN1oXDTE2MDEyMTAxMjky > N1oweTESMBAGA1UEAxMJbG9jYWxob3N0MRgwFgYDVQQIEw9SaGVpbmxhbmQtUGZh > bHoxCzAJBgNVBAYTAkRFMSAwHgYJKoZIhvcNAQkBFhFkZXZAZ29zc2lwZXJsLmNv > bTEaMBgGA1UEChQRZGV2QGdvc3NpcGVybC5jb20wggIiMA0GCSqGSIb3DQEBAQUA > A4ICDwAwggIKAoICAQDSPEXcaq4gdKyB6nGmac91sLNW2ZfBqJOWmkCIpYQnGB27 > EUQTsdxqTtDkfEXlNjf6o4NupytDMqx7lRdVHh+Cqv38S8/Sb9FtyYtsxab4X9hv > vf063O455MKVGCeQGqOTmmQTfybCsiQAa8UYK/chS8wQeBLAIAAaVOcNtmEhbUpb > OaOkwInrjfK9lemD5J8G3z1oUDoiuxwoepyrEWGsmDEWLQKWNJmD6RLeHANH1/UQ > V0PNWqwwYPrkEp9hEgau25/NHrglE9OW1SJmL79Cy3DKvLGxwaH1U0K9vh4rEW3A > Vc36/TCVSpWXkxMUUDYHFihmR2oxyXSgs6/XKWSeV+xJD7VogljVJxl1IzAYcjlV > EbYT4KNqZaqAdSeriRAMSJ5LlZ+7/uknOfdqKcAwUUwdYKKdb1IHpiRmGjFso0zF > icMdKudNLZu854PIkSslh2/SJUhj2fiNwZ8QaGrnklJSEUE6jgjPBcAWbkY6M0Xm > QUON8LcuDGivh1HPOzgiHiWnkd2zGqwAg1u5vsLcraspXBPfgQc+v+oek5xi5OVL > K4YvB8iVf1dg8NQ1Xm3S+aPthi/eI/6UxSx3hSV2fPfydJ8JHoeRx2YWV3Q3dClG > iJJLg36pOoFmO6nMwDDIZpSvPwZsfa/S9A69ZuQMv7ra6o8rzfXo7ZtSvy2X0QID > AQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCoinw+YKNs > 0amglVRGcgDJ4kQlz4HN9Z7phmc5MoEUzeo1vSUH/dvKu9tfgzdArRndXIVMOhqM > UiOJvfIFua+IZIO2VWqSgIiyuJ1NYYk3trceVu9BZFb8GC0zymG54kXfzcCejDxZ > K8hm3eSnTqHJWSFwqGW8OGDzoxfJJCtn6hivZJcyvz6lV0Gp+GxkKHqtDZxQpjCl > 4I7qyz9i7KOfARtca3VTNwQfaMWZSBZ9A7DoYEFvpIGu0faNmy4zbIM7eIKNhBid > v4xqkQHG8f/XTRhNYebBgCzmittKlQc61NodCGnQJXTcS/xZO8NVweAMmqzaR3A9 > BAn9PZ1b/2hScDSAGtJEon3EToqzb5g2ijbKknshbNTe+Nt3J+Riyll115LZRSxg > qwFnPv56WRvL1wJLfYW/S2q9Vwa/RzTiPpPBHPdJw7TETh8Z3UshjSeDVDGmdiQC > V+vFxdMAVtvV7krfuwLa04NQ3mG+WrWfnbPUli7OzZ7NlUxG3Koc0QahABrIahlo > ZDe1yV1OxeSJZByBcM57N04LVDkK29TuqaQsieW/A80g9EN1Q6Y3AJOLhhyZ1Euq > bDg24BJ0P6FiuBz/WFtDRmjwfxhmPiZICfBqpQeltgKsungE4Lg8sMuxPFc7pJpm > hIedZIKnykXzT2wkzRuBz9F9EqyYNRA7BQ== > -----END CERTIFICATE----- > subject=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > issuer=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress= > dev@REDACTED/O=dev@REDACTED > --- > No client certificate CA names sent > --- > SSL handshake has read 2244 bytes and written 264 bytes > --- > New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA > Server public key is 4096 bit > Secure Renegotiation IS supported > Compression: NONE > Expansion: NONE > SSL-Session: > Protocol : TLSv1 > Cipher : DHE-RSA-AES256-SHA > Session-ID: > CC6AD5806068C8845CF6FDF0DDBAA6885412DEDE918D47E669A6BA03208BB0C8 > Session-ID-ctx: > Master-Key: > 57FB4F0F69476D72846453CEE844AFCB577CE9E558A925D3C7914B76E195DDA38479B73E0A81FF48629D3C3F7F8093AE > Key-Arg : None > Start Time: 1421814845 > Timeout : 300 (sec) > Verify return code: 18 (self signed certificate) > ? > > ssl:versions(). > > [{ssl_app,"5.3.8"}, > {supported,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}, > {available,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}] > > I am trying to understand why Erlang has a problem with it. I read that > Erlang had a problem with sha256 signed certs but I am not sure if this is > still a problem? I am using OTP 17.4 for these tests. The error happens in > some unidentified place, I tried modifying OTP to log the error message in > a place where there?s only reference to ?BAD_CERTIFICATE (via call to > path_validation_alert function in ssl_handshake.erl) but I get no log - the > modification can be seen here: > https://github.com/gossiperl/otp/blob/b446bcc3ece0367c96b54af96577b23e4fa43ee4/lib/ssl/src/ssl_handshake.erl#L433 and > I am 100% that the installation of Erlang I?m using is the one with the > modified code. Of course it could be an issue with my certificate. > > I would appreciate any pointers. > > Kind regards, > Radek Gruchalski > radek@REDACTED > de.linkedin.com/in/radgruchalski/ > +4917685656526 > > > *Confidentiality:*This communication is intended for the above-named > person and may be confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Wed Jan 21 19:54:15 2015 From: radek@REDACTED (Rad Gruchalski) Date: Wed, 21 Jan 2015 13:54:15 -0500 Subject: [erlang-questions] Cowboy server, hackney client - SSL problems In-Reply-To: References: <22D6885CDB914F2291B946E72A07CA96@gruchalski.com> Message-ID: Hi Benoit, The insecure option does the job for me. Thank you. I?m not concerned about strict certificate validation for the unit test and currently have no ability to test with a non self-signed cert. I?m still quite new to the erlang scene and I will definitely dive deeper into the SSL validation topic. The problem is that there?s no in-depth knowledge on this subject easily available, most of it is rather difficult to find. Your ssl_verify_hostname project is a very good start. Kind regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) +4917685656526 Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Wednesday, 21 January 2015 at 12:14, Benoit Chesneau wrote: > How are you using curl against your custom certificate? Hackney is very strict by default unless you told it to be not and check only against valid cerificates. There are too way to bypass this protection: > > - use the option `insecure` ewhen you do a request > - use your own SSL options. > > Feel free to join me directly if you need it. > > - benoit > > > On Wed, Jan 21, 2015 at 2:59 PM, Rad Gruchalski wrote: > > Hi everyone, > > > > This is my first question on this mailing list. I have hit a wall while writing a bit of software which does the following: I have a REST-like server running on cowboy which runs on SSL, I?m using a self signed certificate generated by my own certificate authority, the settings I?m using in cowboy are: > > > > [ { port, ? }, > > { cacertfile, absolute-path-to-the-public-cacert }, > > { certfile, absolute-path-to-cert.pem }, > > { keyfile, absolute-path-to-key.pem } ] > > > > Cowboy starts fine, any request coming from CURL or Chrome browser is working fine, the clients are served, no issues whatsoever. > > > > One of the parts of this software is a set of unit tests for the REST-like API. I am intending on using hackney for this. What happens is that when hackney client hits the API, cowboy fails with the following error message: > > > > [error] SSL: certify: ssl_alert.erl:92:Fatal error: bad certificate > > > > The error happens when hackney uses SSL options as cowboy server and with no SSL options. > > > > I have verified my certificates in the following way: added it as a trusted cert in Chrome and Chrome does not complain about anything regarding the certificate anymore. I?ve also done the following: > > > > openssl s_server -accept 8080 -cert ...cert.pem -key ...key.pem -CAfile ?_ca.crt > > Using default temp DH parameters > > Using default temp ECDH parameters > > ACCEPT > > -----BEGIN SSL SESSION PARAMETERS----- > > MHUCAQECAgMBBAIAOQQgzGrVgGBoyIRc9v3w3bqmiFQS3t6RjUfmaaa6AyCLsMgE > > MFf7Tw9pR21yhGRTzuhEr8tXfOnlWKkl08eRS3bhld2jhHm3PgqB/0hinTw/f4CT > > rqEGAgRUvyw9ogQCAgEspAYEBAEAAAA= > > -----END SSL SESSION PARAMETERS----- > > Shared ciphers:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DES-CBC3-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:DHE-RSA-SEED-SHA:DHE-DSS-SEED-SHA:SEED-SHA:RC4-SHA:RC4-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA:EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC4-MD5 > > CIPHER is DHE-RSA-AES256-SHA > > Secure Renegotiation IS supported > > > > openssl s_client -connect localhost:8080 -cert ...cert.pem -key ...key.pem -CAfile ?_ca.crt > > CONNECTED(00000003) > > depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > verify error:num=18:self signed certificate > > verify return:1 > > depth=0 /CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > verify return:1 > > --- > > Certificate chain > > 0 s:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > i:/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > --- > > Server certificate > > -----BEGIN CERTIFICATE----- > > MIIFhTCCA22gAwIBAgIJANOPgonG2vS4MA0GCSqGSIb3DQEBCwUAMHkxEjAQBgNV > > BAMTCWxvY2FsaG9zdDEYMBYGA1UECBMPUmhlaW5sYW5kLVBmYWx6MQswCQYDVQQG > > EwJERTEgMB4GCSqGSIb3DQEJARYRZGV2QGdvc3NpcGVybC5jb20xGjAYBgNVBAoU > > EWRldkBnb3NzaXBlcmwuY29tMB4XDTE1MDEyMTAxMjkyN1oXDTE2MDEyMTAxMjky > > N1oweTESMBAGA1UEAxMJbG9jYWxob3N0MRgwFgYDVQQIEw9SaGVpbmxhbmQtUGZh > > bHoxCzAJBgNVBAYTAkRFMSAwHgYJKoZIhvcNAQkBFhFkZXZAZ29zc2lwZXJsLmNv > > bTEaMBgGA1UEChQRZGV2QGdvc3NpcGVybC5jb20wggIiMA0GCSqGSIb3DQEBAQUA > > A4ICDwAwggIKAoICAQDSPEXcaq4gdKyB6nGmac91sLNW2ZfBqJOWmkCIpYQnGB27 > > EUQTsdxqTtDkfEXlNjf6o4NupytDMqx7lRdVHh+Cqv38S8/Sb9FtyYtsxab4X9hv > > vf063O455MKVGCeQGqOTmmQTfybCsiQAa8UYK/chS8wQeBLAIAAaVOcNtmEhbUpb > > OaOkwInrjfK9lemD5J8G3z1oUDoiuxwoepyrEWGsmDEWLQKWNJmD6RLeHANH1/UQ > > V0PNWqwwYPrkEp9hEgau25/NHrglE9OW1SJmL79Cy3DKvLGxwaH1U0K9vh4rEW3A > > Vc36/TCVSpWXkxMUUDYHFihmR2oxyXSgs6/XKWSeV+xJD7VogljVJxl1IzAYcjlV > > EbYT4KNqZaqAdSeriRAMSJ5LlZ+7/uknOfdqKcAwUUwdYKKdb1IHpiRmGjFso0zF > > icMdKudNLZu854PIkSslh2/SJUhj2fiNwZ8QaGrnklJSEUE6jgjPBcAWbkY6M0Xm > > QUON8LcuDGivh1HPOzgiHiWnkd2zGqwAg1u5vsLcraspXBPfgQc+v+oek5xi5OVL > > K4YvB8iVf1dg8NQ1Xm3S+aPthi/eI/6UxSx3hSV2fPfydJ8JHoeRx2YWV3Q3dClG > > iJJLg36pOoFmO6nMwDDIZpSvPwZsfa/S9A69ZuQMv7ra6o8rzfXo7ZtSvy2X0QID > > AQABoxAwDjAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCoinw+YKNs > > 0amglVRGcgDJ4kQlz4HN9Z7phmc5MoEUzeo1vSUH/dvKu9tfgzdArRndXIVMOhqM > > UiOJvfIFua+IZIO2VWqSgIiyuJ1NYYk3trceVu9BZFb8GC0zymG54kXfzcCejDxZ > > K8hm3eSnTqHJWSFwqGW8OGDzoxfJJCtn6hivZJcyvz6lV0Gp+GxkKHqtDZxQpjCl > > 4I7qyz9i7KOfARtca3VTNwQfaMWZSBZ9A7DoYEFvpIGu0faNmy4zbIM7eIKNhBid > > v4xqkQHG8f/XTRhNYebBgCzmittKlQc61NodCGnQJXTcS/xZO8NVweAMmqzaR3A9 > > BAn9PZ1b/2hScDSAGtJEon3EToqzb5g2ijbKknshbNTe+Nt3J+Riyll115LZRSxg > > qwFnPv56WRvL1wJLfYW/S2q9Vwa/RzTiPpPBHPdJw7TETh8Z3UshjSeDVDGmdiQC > > V+vFxdMAVtvV7krfuwLa04NQ3mG+WrWfnbPUli7OzZ7NlUxG3Koc0QahABrIahlo > > ZDe1yV1OxeSJZByBcM57N04LVDkK29TuqaQsieW/A80g9EN1Q6Y3AJOLhhyZ1Euq > > bDg24BJ0P6FiuBz/WFtDRmjwfxhmPiZICfBqpQeltgKsungE4Lg8sMuxPFc7pJpm > > hIedZIKnykXzT2wkzRuBz9F9EqyYNRA7BQ== > > -----END CERTIFICATE----- > > subject=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > issuer=/CN=localhost/ST=Rheinland-Pfalz/C=DE/emailAddress=dev@REDACTED/O=dev@REDACTED (http://dev@REDACTED/O=dev@REDACTED) > > --- > > No client certificate CA names sent > > --- > > SSL handshake has read 2244 bytes and written 264 bytes > > --- > > New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA > > Server public key is 4096 bit > > Secure Renegotiation IS supported > > Compression: NONE > > Expansion: NONE > > SSL-Session: > > Protocol : TLSv1 > > Cipher : DHE-RSA-AES256-SHA > > Session-ID: CC6AD5806068C8845CF6FDF0DDBAA6885412DEDE918D47E669A6BA03208BB0C8 > > Session-ID-ctx: > > Master-Key: 57FB4F0F69476D72846453CEE844AFCB577CE9E558A925D3C7914B76E195DDA38479B73E0A81FF48629D3C3F7F8093AE > > Key-Arg : None > > Start Time: 1421814845 > > Timeout : 300 (sec) > > Verify return code: 18 (self signed certificate) > > ? > > > > ssl:versions(). > > > > [{ssl_app,"5.3.8"}, > > {supported,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}, > > {available,['tlsv1.2','tlsv1.1',tlsv1,sslv3]}] > > > > > > I am trying to understand why Erlang has a problem with it. I read that Erlang had a problem with sha256 signed certs but I am not sure if this is still a problem? I am using OTP 17.4 for these tests. The error happens in some unidentified place, I tried modifying OTP to log the error message in a place where there?s only reference to ?BAD_CERTIFICATE (via call to path_validation_alert function in ssl_handshake.erl) but I get no log - the modification can be seen here: https://github.com/gossiperl/otp/blob/b446bcc3ece0367c96b54af96577b23e4fa43ee4/lib/ssl/src/ssl_handshake.erl#L433 and I am 100% that the installation of Erlang I?m using is the one with the modified code. Of course it could be an issue with my certificate. > > > > I would appreciate any pointers. > > > > > > > > > > > > > > > > > > > > > > Kind regards, > > Radek Gruchalski > > radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) > > de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) > > +4917685656526 (tel:%2B4917685656526) > > > > Confidentiality: > > This communication is intended for the above-named person and may be confidential and/or legally privileged. > > If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From samjbarney@REDACTED Wed Jan 21 23:58:57 2015 From: samjbarney@REDACTED (Samuel Barney) Date: Wed, 21 Jan 2015 15:58:57 -0700 Subject: [erlang-questions] Multiple Networking interfaces on one node Message-ID: If I have a server with multiple network interfaces, is it possible to communicate over all of them, or is a node limited to just one? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jan 22 05:48:40 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 22 Jan 2015 17:48:40 +1300 Subject: [erlang-questions] floating point exception when accuracy exceeds limit In-Reply-To: References: Message-ID: <4E33037D-3384-4D32-997B-72C80FC77CE6@cs.otago.ac.nz> On 20/01/2015, at 11:26 pm, Johan Montelius wrote: > > This might be no news to anyone who is doing a lot of floating point arithmetics but it made me scratch my head. It?s something we teach in second year: IEEE double precision floating point numbers have DBL_MIN = 2.22...E-308 and DBL_MAX = 1.79...E+308. This means that the number 3.6E309 that you are trying to compute cannot be represented. This has nothing to do with accuracy. In most programming languages, you would get +Infinity as the answer. f% gsi # Scheme > (expt 6.0e154 2) +inf.0 f% irb # Rugy irb(main):001:0> x = 6.0e154 6.0e+154 irb(main):002:0> print x*x Infinity nil f% gst # GNU Smalltalk st> 6.0e154 squared Inf Erlang reports an error instead of returning ?Infinity. It would be a little more helpful if it *said* that it was an overflow error. Now some machines DO have floating point formats with wider ranges. For example, the SPARC architecture includes 128-bit floats (LDBL_MIN = 3.36...E-4392, LDBL_MAX = 1.18?E+4932) and the x86 architecture has 80-bit floats (with nearly the same range but less accuracy). But this is not portable. 64-bit floats are the most you can *rely* on getting. There is a handful of things *every* programmer has to worry about when using floating point arithmetic: - domain errors like divide by zero or square root of a negative number or acos(x) where |x| > 1 - underflow (commonly reported as 0, but this can cause division by 0 problems in expressions that are mathematically non-zero) - overflow - catastrophic cancellation (subtracting two numbers that are close, so that the result has very few meaningful bits) - adding numbers that are very different in magnitude, so that all or most of the information from the smaller one is lost - roundoff (if *all* you have to worry about is roundoff, you are either a lucky programmer or a good one ^_^) If you have to take the product of a lot of numbers, one common approach that avoids overflow is to take the product of the signs and exp(the sum of the logarithms). Would that be any use in your case? From ct.radu.001@REDACTED Thu Jan 22 11:32:30 2015 From: ct.radu.001@REDACTED (CT Radu) Date: Thu, 22 Jan 2015 12:32:30 +0200 Subject: [erlang-questions] cowboy_router weird availability error Message-ID: Hello everyone, I'm experiencing a weird behavior related to cowboy. I have an application that uses cowboy for the webinterface. Prior to this month we used 0.6.1 and recently I started to update it to 1.0.1 I get the following error only when I start the application on fedora 20 (using R16B03-1) [error] <0.943.0>Undefined - Undefined:Undefined:Undefined - CRASH REPORT Process <0.943.0> with 0 neighbours exited with reason: call to undefined function cowboy_router:compile([{'_',[{[<<"/wsock">>]... The same code, compiled the same R16B03-1 on centos 6, when started, properly finds the cowboy_router module and executes the cowboy_router:compile code. Cowboy is included in app.src in the applications section. Also (I think this is redundant) I've started it in the top application module (the one with the suffix _app.erl). Still, I cannot start my application on fedora 20... This application is packed as a release using relx. Any ideas where I could look or what I may have missed ? Many thanks, Tiberiu -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Jan 22 12:45:58 2015 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 22 Jan 2015 11:45:58 +0000 Subject: [erlang-questions] eunit timeout setting doesn't work? In-Reply-To: References: Message-ID: On 21 January 2015 at 16:56, Roger Lipscombe wrote: > Learn You Some Erlang suggests that > > all_test_() -> > {timeout, 600, [fun first/0, fun second/0]}. > > ...ought to work. But it doesn't. I still get a timeout after approx 5 seconds. It turns out that it works fine if there's only *one* test in the set, but not if there's more than one. This is surely a bug? all_test_() -> {timeout, 600, [fun slow/0]}. From michael.santos@REDACTED Thu Jan 22 16:30:14 2015 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 22 Jan 2015 10:30:14 -0500 Subject: [erlang-questions] Multiple Networking interfaces on one node In-Reply-To: References: Message-ID: <20150122153014.GA10578@brk> On Wed, Jan 21, 2015 at 03:58:57PM -0700, Samuel Barney wrote: > If I have a server with multiple network interfaces, is it possible to > communicate over all of them, or is a node limited to just one? Sure. Are the interfaces on the same network? Then bonding the interfaces might work. Different networks? Set up the routing table. Neither of these requires anything from erlang. gen_tcp:connect/3 and gen_udp:open/2 allow binding to a specific address: {ifaddr, ip_address()}: Same as {ip, ip_address()}. If the host has several network interfaces, this option specifies which one to use. From boozelclark@REDACTED Thu Jan 22 16:54:36 2015 From: boozelclark@REDACTED (Chris Clark) Date: Thu, 22 Jan 2015 17:54:36 +0200 Subject: [erlang-questions] Stopping a supervisor running under a supervisor In-Reply-To: References: Message-ID: Thank you all very much for your assistance. With the help of the members of the mailing list I managed to get this issue solved. The children are transient. My problem was that the workers (w1, w2 ...) and the serv were being killed and they were logging the errors. I managed to work around this by first sending a shutdown command to the workers and then using a timer:apply after to stop the supervisors in my tree after some time essentially giving the workers and serv a time-out to shutdown or be killed. Thank you all for your assistance. On Fri, Jan 16, 2015 at 2:07 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > I'd go with a solution where you mark the children as 'transient' and then > build code inside them to stop gracefully. That way, if the termination is > due to a 'normal' exit, there should be no indication of anything in the > log files. > > On Tue Jan 06 2015 at 4:32:01 PM Chris Clark > wrote: > >> Hi Sean >> >> supervisor:terminate_child/2 does work however it causes errors in my >> logs so i am looking for a way to gracefully terminate the children. >> >> Thanks >> >> On Tue, Jan 6, 2015 at 4:20 PM, Sean Cribbs wrote: >> >>> Did you try supervisor:terminate_child/2? >>> >>> On Mon, Jan 5, 2015 at 11:44 PM, Chris Clark >>> wrote: >>> >>>> Hi Antonio >>>> >>>> Thanks for the assistance. From what I have read a supervisor should >>>> stop if it is sent a shutdown signal using exit(Pid, shutdown). but it >>>> will only accept that signal from its parent process in an OTP application. >>>> I tried creating a function in the parent supervisor(my_supersup) >>>> stop_child(Pid) that then sends the exit(pid, shutdown) signal and then >>>> call that from the shell but that doesn't seem to stop it either. >>>> >>>> Thanks, >>>> >>>> On Tue, Dec 30, 2014 at 11:47 PM, Antonio Dias < >>>> antonio.r.dias@REDACTED> wrote: >>>> >>>>> Hi, i'm fairly new to Erlang also, but can't you send a 'stop' message >>>>> and use that to stop the process? >>>>> >>>>> ADias >>>>> >>>>> On Thu, Dec 11, 2014 at 10:40 AM, Chris Clark >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> I am fairly new to erlang and have a question regarding properly >>>>>> stopping a supervisor that resides directly under another supervisor. >>>>>> >>>>>> My application consist of a top level supervisor (my_supersup) which >>>>>> is a simple_one_for_one supervisor of other supervisors my_sup. Each my_sup >>>>>> supervises a gen_server, serv and another supervisor that is also a >>>>>> simple_one_for_one supervisor of some workers >>>>>> >>>>>> my_supersup >>>>>> (simple_one_for_one, transient) >>>>>> | >>>>>> |_______ ... >>>>>> | >>>>>> my_sup (one_for_all, transient) >>>>>> |________________ >>>>>> | | >>>>>> | serv >>>>>> worker_sup >>>>>> (simple_one_for_one, transient) >>>>>> |___________ ... >>>>>> | | >>>>>> w1 w2 >>>>>> >>>>>> How do shutdown an instance of my_sup gracefully without killing it. >>>>>> At present I can do it with supervisor:terminate_child and >>>>>> supervisor:delete_child but I would like them to shutdown rather than be >>>>>> killed if possible? >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> Sean Cribbs >>> Sr. Software Engineer >>> Basho Technologies, Inc. >>> http://basho.com/ >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 22 17:33:57 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Jan 2015 17:33:57 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory Message-ID: Dear List, I'm having some troubles in pinpointing why a node is crashing due to memory issues. For info, when it crashes, it does not produce a crash dump. However I've monitored live and I've seen the .beam process eat up all memory until it abruptly exits. The system is a big router that relays data coming from TCP connections, into other TCP connections. I'm using cowboy as the HTTP server that initiates the long-lived TCP connections. I've done all the obvious: - Checked the States of my gen_servers and processes. - Checked my processes mailboxes (the ones with the longest queue have 1 item in the inbox). - My ETS table memory is constant (see below). I put the system under controlled load, and I can see with length(processes()). that my process count is stable, always around 120,000. I check the processes that are using most memory with this call: MostMemory = fun(N) -> lists:sublist( lists:sort( fun({_, _, V1}, {_, _, V2}) -> V1 >= V2 end, [try [{memory, Mem}, {registered_name, RegName}] = erlang:process_info(Pid, [memory, registered_name]), {Pid, RegName, Mem} catch _:_ -> {Pid, undefined, 0} end || Pid <- processes(), Pid =/= self()] ), N) end. Which always returns very similar numbers: 1> MostMemory(20). [{<0.96.0>,[],5180448}, {<0.78.0>,tls_connection_sup,4525096}, {<0.6.0>,error_logger,743776}, {<0.7.0>,application_controller,372592}, {<0.77.0>,ssl_manager,284640}, {<0.11.0>,kernel_sup,176712}, {<0.26.0>,code_server,176272}, {<0.33.0>,[],143064}, {<0.419.0>,[],142896}, {<0.420.0>,[],142896}, {<0.421.0>,[],142896}, {<0.422.0>,[],142896}, {<0.423.0>,[],142896}, {<0.424.0>,[],142896}, {<0.425.0>,[],142896}, {<0.426.0>,[],142896}, {<0.427.0>,[],142896}, {<0.428.0>,[],142896}, {<0.429.0>,[],142896}, {<0.430.0>,[],142896}] See the last processes there with all identical memory? These are the processes handling the connections, and they stay stable with the same identical number throughout all test. I get the pid of the .beam process, and I check its reported RES memory with top -p beam-pid-here. I get my erlang memory with this simple call (I just convert everything to GB, thanks to Ferd and his article https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole): [{K,V / math:pow(1024,3)} || {K,V} <- erlang:memory()]. This is what I get (at random time intervals): - BEAM process RES memory:* 2.751 GB* - Erlang memory: [{total,2.11871287971735}, {processes,1.6582859307527542}, {processes_used,1.6581560596823692}, {system,0.4604269489645958}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.29880597442388535}, {code,0.009268132038414478}, {ets,0.004808835685253143}] - BEAM process RES memory:* 3.039 GB* - Erlang memory: [{total,2.2570599243044853}, {processes,1.7243007272481918}, {processes_used,1.7241046279668808}, {system,0.5327591970562935}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.37129393219947815}, {code,0.009268132038414478}, {ets,0.004808835685253143}] - BEAM process RES memory:* 3.630 GB* - Erlang memory: [{total,2.677028402686119}, {processes,2.1421403884887695}, {processes_used,2.142106533050537}, {system,0.5348880141973495}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.37329262495040894}, {code,0.009268132038414478}, {ets,0.004808835685253143}] - BEAM process RES memory:* 3.807 GB* - Erlang memory: [{total,2.9233806803822517}, {processes,2.277688652276993}, {processes_used,2.277618482708931}, {system,0.6456920281052589}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.48407071083784103}, {code,0.009268132038414478}, {ets,0.004808835685253143}] - BEAM process RES memory:* 4.026 GB* - Erlang memory: [{total,2.8762372359633446}, {processes,2.100425034761429}, {processes_used,2.1003194376826286}, {system,0.7758122012019157}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.6143399104475975}, {code,0.009268132038414478}, {ets,0.004808835685253143}] - BEAM process RES memory:* 4.136 GB* - Erlang memory: [{total,2.9030912443995476}, {processes,2.028559662401676}, {processes_used,2.0283572375774384}, {system,0.8745315819978714}, {atom,4.000673070549965e-4}, {atom_used,3.847004845738411e-4}, {binary,0.7129654437303543}, {code,0.00929991528391838}, {ets,0.004809550940990448}] - BEAM process RES memory:* 4.222 GB* - Erlang memory: [{total,2.785604253411293}, {processes,1.875294029712677}, {processes_used,1.8752291351556778}, {system,0.910310223698616}, {atom,4.000673070549965e-4}, {atom_used,3.847004845738411e-4}, {binary,0.7487552836537361}, {code,0.00929991528391838}, {ets,0.004809550940990448}] As you can see, at the beginning both the BEAM RES memory and the total Erlang memory increase, but after a while it becomes clear that the BEAM process memory keeps increasing while the memory reported as used by Erlang stabilizes, and even decreases. Erlang reported memory never surpasses 3 GB. At this point I tried forcing a Garbage Collection: [erlang:garbage_collect(Pid) || Pid <- processes()] After that, we went back to: - BEAM process RES memory:* 3.336 GB* - Erlang memory: [{total,1.9107630401849747}, {processes,1.5669479593634605}, {processes_used,1.5668926388025284}, {system,0.34381508082151413}, {atom,4.000673070549965e-4}, {atom_used,3.847004845738411e-4}, {binary,0.18235664814710617}, {code,0.00929991528391838}, {ets,0.004809550940990448}] However after that, I let the system go and it kept on having the same behavior (and increasing the BEAM memory). What puzzles me is that you can clearly see that: - The total memory used by processes is increasing, however the top processes always use the same amount of memory (and the process count is always stable). - Binary consumption also increases, but in proportion with process memory (and my data is <64K so I don't anticipate it being an issue of Refc-binaries not being garbage collected). I already hibernate most of the long-term open connections. I also added a periodic garbage collector on the main router, since it touches all the binaries that go through it, to ensure that all Refc-binaries that hold a reference to the router are garbage collected. So I tried the hard approach, and I've set fullsweep_after to 0 as a system flag (passed in as an environment variable -env ERL_FULLSWEEP_AFTER 0). After this, I could see notable improvements: - BEAM process RES memory:* 2.049 GB* - Erlang memory: [{total,1.597476489841938}, {processes,1.2037805244326591}, {processes_used,1.2036690935492516}, {system,0.39369596540927887}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.2321353331208229}, {code,0.009268132038414478}, {ets,0.004821933805942535}] - BEAM process RES memory:* 1.919 GB* - Erlang memory: [{total,1.549286112189293}, {processes,1.1740453317761421}, {processes_used,1.1739420965313911}, {system,0.3752407804131508}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.2134672999382019}, {code,0.009268132038414478}, {ets,0.004821933805942535}] - BEAM process RES memory:* 2.004 GB* - Erlang memory: [{total,1.6023957282304764}, {processes,1.2192133665084839}, {processes_used,1.219102293252945}, {system,0.3831823617219925}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.22155668586492538}, {code,0.009268132038414478}, {ets,0.004821933805942535}] - BEAM process RES memory:* 2.456 GB* - Erlang memory: [{total,1.7860298827290535}, {processes,1.4158401936292648}, {processes_used,1.4157484397292137}, {system,0.37018968909978867}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.20867645740509033}, {code,0.009268132038414478}, {ets,0.004821933805942535}] - BEAM process RES memory:* 2.455 GB* - Erlang memory: [{total,1.8919306173920631}, {processes,1.4726912006735802}, {processes_used,1.4726523533463478}, {system,0.41923941671848297}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.25766071677207947}, {code,0.009268132038414478}, {ets,0.004821933805942535}] However, the down size to this is obviously that the CPU load increased almost of a point. I also have a GC "guardian" similar to the one that Fred implemented in Heroku's logplex: https://github.com/heroku/logplex/blob/master/src/logplex_leak.erl But this obviously is a guard, not a solution per se. Can anyone give me some pointers on how I can process to identify what is going on? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Jan 22 18:00:37 2015 From: rvirding@REDACTED (Robert Virding) Date: Thu, 22 Jan 2015 18:00:37 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: One thing you can see is that the size of the binary data is growing. This space contains the large binaries (> 64 bytes) which are sent in messages between processes. While this means that the messages become (much) smaller and faster to send it takes a much longer time to detect that they are no longer alive and can be reclaimed. Basically it takes until all the processes they have passed through does a full garbage collection. Setting fullsweep_after to 0 and doing explicit garbage collects speeds up reclaiming the binaries. You could be much more selective in which processes you set fullsweep_after to 0 and which ones you explicitly garbage collect. I don't know if the is *the* problem but it is *a* problem you have. Robert On 22 January 2015 at 17:33, Roberto Ostinelli wrote: > Dear List, > I'm having some troubles in pinpointing why a node is crashing due to > memory issues. > For info, when it crashes, it does not produce a crash dump. However I've > monitored live and I've seen the .beam process eat up all memory until it > abruptly exits. > > The system is a big router that relays data coming from TCP connections, > into other TCP connections. > I'm using cowboy as the HTTP server that initiates the long-lived TCP > connections. > > I've done all the obvious: > > - Checked the States of my gen_servers and processes. > - Checked my processes mailboxes (the ones with the longest queue have > 1 item in the inbox). > - My ETS table memory is constant (see below). > > I put the system under controlled load, and I can see with > length(processes()). that my process count is stable, always around > 120,000. > > I check the processes that are using most memory with this call: > > MostMemory = fun(N) -> > lists:sublist( > lists:sort( > fun({_, _, V1}, {_, _, V2}) -> V1 >= V2 end, > [try > [{memory, Mem}, {registered_name, RegName}] = > erlang:process_info(Pid, [memory, registered_name]), > {Pid, RegName, Mem} > catch _:_ -> > {Pid, undefined, 0} > end || Pid <- processes(), Pid =/= self()] > ), N) > end. > > Which always returns very similar numbers: > > 1> MostMemory(20). > [{<0.96.0>,[],5180448}, > {<0.78.0>,tls_connection_sup,4525096}, > {<0.6.0>,error_logger,743776}, > {<0.7.0>,application_controller,372592}, > {<0.77.0>,ssl_manager,284640}, > {<0.11.0>,kernel_sup,176712}, > {<0.26.0>,code_server,176272}, > {<0.33.0>,[],143064}, > {<0.419.0>,[],142896}, > {<0.420.0>,[],142896}, > {<0.421.0>,[],142896}, > {<0.422.0>,[],142896}, > {<0.423.0>,[],142896}, > {<0.424.0>,[],142896}, > {<0.425.0>,[],142896}, > {<0.426.0>,[],142896}, > {<0.427.0>,[],142896}, > {<0.428.0>,[],142896}, > {<0.429.0>,[],142896}, > {<0.430.0>,[],142896}] > > See the last processes there with all identical memory? These are the > processes handling the connections, and they stay stable with the same > identical number throughout all test. > > I get the pid of the .beam process, and I check its reported RES memory > with top -p beam-pid-here. > I get my erlang memory with this simple call (I just convert everything to > GB, thanks to Ferd and his article > https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole): > > [{K,V / math:pow(1024,3)} || {K,V} <- erlang:memory()]. > > This is what I get (at random time intervals): > > - BEAM process RES memory:* 2.751 GB* > - Erlang memory: > [{total,2.11871287971735}, > {processes,1.6582859307527542}, > {processes_used,1.6581560596823692}, > {system,0.4604269489645958}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.29880597442388535}, > {code,0.009268132038414478}, > {ets,0.004808835685253143}] > > - BEAM process RES memory:* 3.039 GB* > - Erlang memory: > [{total,2.2570599243044853}, > {processes,1.7243007272481918}, > {processes_used,1.7241046279668808}, > {system,0.5327591970562935}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.37129393219947815}, > {code,0.009268132038414478}, > {ets,0.004808835685253143}] > > - BEAM process RES memory:* 3.630 GB* > - Erlang memory: > [{total,2.677028402686119}, > {processes,2.1421403884887695}, > {processes_used,2.142106533050537}, > {system,0.5348880141973495}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.37329262495040894}, > {code,0.009268132038414478}, > {ets,0.004808835685253143}] > > - BEAM process RES memory:* 3.807 GB* > - Erlang memory: > [{total,2.9233806803822517}, > {processes,2.277688652276993}, > {processes_used,2.277618482708931}, > {system,0.6456920281052589}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.48407071083784103}, > {code,0.009268132038414478}, > {ets,0.004808835685253143}] > > > - BEAM process RES memory:* 4.026 GB* > - Erlang memory: > [{total,2.8762372359633446}, > {processes,2.100425034761429}, > {processes_used,2.1003194376826286}, > {system,0.7758122012019157}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.6143399104475975}, > {code,0.009268132038414478}, > {ets,0.004808835685253143}] > > > - BEAM process RES memory:* 4.136 GB* > - Erlang memory: > [{total,2.9030912443995476}, > {processes,2.028559662401676}, > {processes_used,2.0283572375774384}, > {system,0.8745315819978714}, > {atom,4.000673070549965e-4}, > {atom_used,3.847004845738411e-4}, > {binary,0.7129654437303543}, > {code,0.00929991528391838}, > {ets,0.004809550940990448}] > > > - BEAM process RES memory:* 4.222 GB* > - Erlang memory: > [{total,2.785604253411293}, > {processes,1.875294029712677}, > {processes_used,1.8752291351556778}, > {system,0.910310223698616}, > {atom,4.000673070549965e-4}, > {atom_used,3.847004845738411e-4}, > {binary,0.7487552836537361}, > {code,0.00929991528391838}, > {ets,0.004809550940990448}] > > > As you can see, at the beginning both the BEAM RES memory and the total > Erlang memory increase, but after a while it becomes clear that the BEAM > process memory keeps increasing while the memory reported as used by Erlang > stabilizes, and even decreases. > Erlang reported memory never surpasses 3 GB. > > At this point I tried forcing a Garbage Collection: > > [erlang:garbage_collect(Pid) || Pid <- processes()] > > After that, we went back to: > > - BEAM process RES memory:* 3.336 GB* > - Erlang memory: > [{total,1.9107630401849747}, > {processes,1.5669479593634605}, > {processes_used,1.5668926388025284}, > {system,0.34381508082151413}, > {atom,4.000673070549965e-4}, > {atom_used,3.847004845738411e-4}, > {binary,0.18235664814710617}, > {code,0.00929991528391838}, > {ets,0.004809550940990448}] > > However after that, I let the system go and it kept on having the same > behavior (and increasing the BEAM memory). > > What puzzles me is that you can clearly see that: > > - The total memory used by processes is increasing, however the top > processes always use the same amount of memory (and the process count is > always stable). > - Binary consumption also increases, but in proportion with process > memory (and my data is <64K so I don't anticipate it being an issue of > Refc-binaries not being garbage collected). > > I already hibernate most of the long-term open connections. > I also added a periodic garbage collector on the main router, since it > touches all the binaries that go through it, to ensure that all > Refc-binaries that hold a reference to the router are garbage collected. > > So I tried the hard approach, and I've set fullsweep_after to 0 as a > system flag (passed in as an environment variable -env > ERL_FULLSWEEP_AFTER 0). > > After this, I could see notable improvements: > > - BEAM process RES memory:* 2.049 GB* > - Erlang memory: > [{total,1.597476489841938}, > {processes,1.2037805244326591}, > {processes_used,1.2036690935492516}, > {system,0.39369596540927887}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.2321353331208229}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > > - BEAM process RES memory:* 1.919 GB* > - Erlang memory: > [{total,1.549286112189293}, > {processes,1.1740453317761421}, > {processes_used,1.1739420965313911}, > {system,0.3752407804131508}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.2134672999382019}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > > - BEAM process RES memory:* 2.004 GB* > - Erlang memory: > [{total,1.6023957282304764}, > {processes,1.2192133665084839}, > {processes_used,1.219102293252945}, > {system,0.3831823617219925}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.22155668586492538}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > > - BEAM process RES memory:* 2.456 GB* > - Erlang memory: > [{total,1.7860298827290535}, > {processes,1.4158401936292648}, > {processes_used,1.4157484397292137}, > {system,0.37018968909978867}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.20867645740509033}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > > - BEAM process RES memory:* 2.455 GB* > - Erlang memory: > [{total,1.8919306173920631}, > {processes,1.4726912006735802}, > {processes_used,1.4726523533463478}, > {system,0.41923941671848297}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.25766071677207947}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > > > However, the down size to this is obviously that the CPU load increased > almost of a point. > > I also have a GC "guardian" similar to the one that Fred implemented in > Heroku's logplex: > https://github.com/heroku/logplex/blob/master/src/logplex_leak.erl > > But this obviously is a guard, not a solution per se. > > Can anyone give me some pointers on how I can process to identify what is > going on? > > Thank you, > r. > > > > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 22 18:11:07 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Jan 2015 18:11:07 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Thank you Robert. I'm going to try a selective fullsweep_after. Could this also justify the process memory increase (which is more significant)? On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding wrote: > One thing you can see is that the size of the binary data is growing. This > space contains the large binaries (> 64 bytes) which are sent in messages > between processes. While this means that the messages become (much) smaller > and faster to send it takes a much longer time to detect that they are no > longer alive and can be reclaimed. Basically it takes until all the > processes they have passed through does a full garbage collection. Setting > fullsweep_after to 0 and doing explicit garbage collects speeds up > reclaiming the binaries. > > You could be much more selective in which processes you set > fullsweep_after to 0 and which ones you explicitly garbage collect. > > I don't know if the is *the* problem but it is *a* problem you have. > > Robert > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 22 18:38:56 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Jan 2015 18:38:56 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Here's something I've tried which is successful in avoiding the memory increase for binaries. Inside a loop, I used to have: [...] <> = Data, loop(Body, Rest); Now I force a binary copy to ensure that the reference to the original full binary is easily removed: [...] <> = Data, Body = binary:copy(Body0), Rest = binary:copy(Rest0), loop(Body, Rest); This seems to have stabilized the memory usage reported by erlang:memory/0. However: - I believe this can only work if the copied binary are *heap* and not *ref-c*, is this correct? - Unfortunately, the BEAM process reported RES memory sill keeps growing. Any other ideas? r. On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli wrote: > Thank you Robert. > I'm going to try a selective fullsweep_after. > > Could this also justify the process memory increase (which is more > significant)? > > > > On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding > wrote: > >> One thing you can see is that the size of the binary data is growing. >> This space contains the large binaries (> 64 bytes) which are sent in >> messages between processes. While this means that the messages become >> (much) smaller and faster to send it takes a much longer time to detect >> that they are no longer alive and can be reclaimed. Basically it takes >> until all the processes they have passed through does a full garbage >> collection. Setting fullsweep_after to 0 and doing explicit garbage >> collects speeds up reclaiming the binaries. >> >> You could be much more selective in which processes you set >> fullsweep_after to 0 and which ones you explicitly garbage collect. >> >> I don't know if the is *the* problem but it is *a* problem you have. >> >> Robert >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From valentin@REDACTED Thu Jan 22 18:48:15 2015 From: valentin@REDACTED (Valentin Micic) Date: Thu, 22 Jan 2015 19:48:15 +0200 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: If you going to copy, you may just as well use lists instead of binary? V/ On 22 Jan 2015, at 7:38 PM, Roberto Ostinelli wrote: > Here's something I've tried which is successful in avoiding the memory increase for binaries. > > Inside a loop, I used to have: > > [...] > <> = Data, > loop(Body, Rest); > > Now I force a binary copy to ensure that the reference to the original full binary is easily removed: > > [...] > <> = Data, > Body = binary:copy(Body0), > Rest = binary:copy(Rest0), > loop(Body, Rest); > > This seems to have stabilized the memory usage reported by erlang:memory/0. > > However: > I believe this can only work if the copied binary are heap and not ref-c, is this correct? > Unfortunately, the BEAM process reported RES memory sill keeps growing. > Any other ideas? > r. > > > > > On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli wrote: > Thank you Robert. > I'm going to try a selective fullsweep_after. > > Could this also justify the process memory increase (which is more significant)? > > > > On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding wrote: > One thing you can see is that the size of the binary data is growing. This space contains the large binaries (> 64 bytes) which are sent in messages between processes. While this means that the messages become (much) smaller and faster to send it takes a much longer time to detect that they are no longer alive and can be reclaimed. Basically it takes until all the processes they have passed through does a full garbage collection. Setting fullsweep_after to 0 and doing explicit garbage collects speeds up reclaiming the binaries. > > You could be much more selective in which processes you set fullsweep_after to 0 and which ones you explicitly garbage collect. > > I don't know if the is *the* problem but it is *a* problem you have. > > Robert > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Jan 22 18:56:04 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 22 Jan 2015 18:56:04 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: It's an interesting point, though as stated I'm using cowboy. Don't tell me I have to revive misultin, I won't ^^_ Do you have an alternative suggestion? Best, r. On Thu, Jan 22, 2015 at 6:48 PM, Valentin Micic wrote: > If you going to copy, you may just as well use lists instead of binary? > > V/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Thu Jan 22 19:14:54 2015 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Thu, 22 Jan 2015 19:14:54 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <675B6342-59C2-46E6-8AAF-28B2FBFAC681@gmail.com> How about regular respawning of processes? Sockets can easily be transfered to new processes. I?m not sure how difficult that would be to achieve with cowboy however. Sergej On 22 Jan 2015, at 18:56, Roberto Ostinelli wrote: > It's an interesting point, though as stated I'm using cowboy. Don't tell me I have to revive misultin, I won't ^^_ > > Do you have an alternative suggestion? > > Best, > r. > > On Thu, Jan 22, 2015 at 6:48 PM, Valentin Micic wrote: > If you going to copy, you may just as well use lists instead of binary? > > V/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabrek@REDACTED Thu Jan 22 19:41:12 2015 From: mabrek@REDACTED (Anton Lebedevich) Date: Thu, 22 Jan 2015 21:41:12 +0300 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <54C14448.3070804@gmail.com> Hello. Do you have any external monitoring system in place (like statsderl or exometer + graphite) that watches erlang memory consumption (process, binary, ... total)? What is the long-term growth trend? Is it linear or similar to capacitor charge (like http://hades.mech.northwestern.edu/index.php/File:RC_charge_voltage.gif) >From the data provided it looks like some subpopulation of processes is growing their heaps. It might happen for mid- or small-sized processes since top processes always have the same size. Can you measure [memory, message_queue_len, total_heap_size, heap_size, stack_size] for all processes several times (with the same interval as you measured total memory growth) and compare histograms of their memory consumption? Or upload that data somewhere and I'll play with it in R to find what's changing inside. Regards, Anton Lebedevich. PS forgot to reply to the list From ingela.andin@REDACTED Thu Jan 22 21:45:08 2015 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 22 Jan 2015 21:45:08 +0100 Subject: [erlang-questions] Fwd: Figuring out proper ssl certificate settings with 17.3 In-Reply-To: References: <4306941.X6676PlrAe@moltowork> Message-ID: Hi! There was a bug in R16 that if the client sends only its peer-cert and the server happens to own the intermediate CA cert that signed the client cert it concludes that the intermediate CA cert is the trusted cert and everything works. But according to the TLS protocol the client must send the whole chain only the self signed ROOT may be excluded from the chain. However pkix-X509 certificate path validation says it should be possible to put the trust in an intermediate cert, and that is why we have the partial_chain to allow path validation on only part of the chain, so that means the server that accepts a partial chain could know of the intermediate cert but not own the ROOT, as it has decided to trust the intermediate CA. So a TLS client that can not build its whole chain is incorrectly configured. E.i. the partial chain function lets you shorten the chain by accepting an intermediate cert sent to you by the peer as trusted. If you want to handle incorrect clients by building the chain to the client certificate on the server side , if possible, you need to do that in the verify_fun when it fails and then call public_key:pkix_path_validation again with the chain that you built. As the self-signed root cert may be left out of the chain client software will guess that if the signer of a cert is not found it must be the self-signed root, however if the client is not correctly configured this could be a wrong guess. See also comment inlined 2015-01-21 18:05 GMT+01:00 Vincent de Phily < vincent.dephily@REDACTED>: > Hi, > > we're trying to upgrade from 17.1 to 17.4 and are hitting authentication > failures, apparently due to the changes that introduced the partial_chain > option. > > First some background: > * We want to authenticate both server and client > * We manage certificate ourselves, and don't care about the OS-supplied > CAs. > * The signing hierarchy is > TopCA -> ServerCA -> Account1ServerCert > -> Account2ServerCert > -> ... > -> ClientCA -> Account1ClientCert > -> Account1Device1Cert > -> Account1Device2Cert > -> ... > -> Account2ClientCert > -> ... > * The server and test client are in erlang, the real clients are in various > languages (and we can't afford to update them). > > > What we used to do, works fine in 17.1: > server: {cert, Account1ServerCert}, {cacerts, [ClientCA]} > client: {cert, Account1Device1Cert}, {cacerts, [ServerCA]} > (in some cases we used the 'file' version of the option, but that doesn't > change the semantics AFAIK). > > But that gives us an "unknown ca" error with 17.4. > > The minimum list of cacerts that seems to work is [ServerCA,TopCA] for the > server and [clientCA,TopCA] for the client, but: > * Having to specify the TopCA is an administration pain (lots of configs to > change, and need to take the file out of the vault). > * I'd rather not send unnecessary bytes over the wire. Ideally I don't > want to > send any root/intermediate CA, only the leaf cert. > * Huh ? Why does ClientCA not need to be in ther server's cacerts (and > vice- > versa) ? > > I guess some confusion comes from the fact that the cacerts option is used > for > two different things (authenticating the cert received by the peer, and > sending intermediate CAs to the peer to help it authenticate me). Maybe the > option could be split in two ? > > This is documented and you will find the same behaviour in openSSL. Client side: *{cacertfile, path()}*The path to a file containing PEM encoded CA certificates. The CA certificates are used during server authentication and when building the client certificate chain. Sever side: *{cacertfile, path()}* The path to a file containing PEM encoded CA certificates. The CA certificates are used to build the server certificate chain, and for client authentication. Also the CAs are used in the list of acceptable client CAs passed to the client when a certificate is requested. May be omitted if there is no need to verify the client and if there are not any intermediate CAs for the server certificate. AFAIU, the new requirement for TopCA to be present comes from a tightening > of > the check mandated by the RFC : the certificate chain (which one ? the one > sent by the client or the one set up locally) needs to end with a > self-signed > cert. > > I'll pass on the weirdness of that requirement (I'll trust whichever CA I > decide, wether it's self-signed or not, damnit), as it looks like > partial_chain is expressely designed to bypass it. But I wasn't able to > write > one that worked *and* didn't look like a big security hole. Could somebody > write an example partial_chain fun that works for my usecase ? > > Ideally, the docs could do with some clarifications : > * The spec for partial_chain has a typo (at least a missing '}') > Again, and example in the docs wouldn't go amiss. > * It's not clear which chain it acts uppon > * Is the semantic of cacertfile really different for the server and the > client ? From my knowledge of TLS, it should be symetrical (even if the > common real-world case is that the client doesn't authenticate itself), > so > the option should be in the common section (next to cacerts with a note > that > one is equivalent to the other). > > > Thanks in advance. > Hope that helps. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From bm@REDACTED Thu Jan 22 23:42:04 2015 From: bm@REDACTED (Bernd May) Date: Thu, 22 Jan 2015 23:42:04 +0100 Subject: [erlang-questions] cipher negotiation problem in SSL application when using PFS only Message-ID: <54C17CBC.2060801@dv-team.de> Hi List, I have spent the past two days fiddling with erlang for the first time in my life mainly because I want to use rabbitMQ in what seems to be a yet more unusual way. During that fiddling I have encountered an error that I have a hard time debugging. Here is the scenario: Whenever I try to configure RabbitMQ to use SSL with PFS ciphers (that is DHE and ECDHE) only I receive the following error: =ERROR REPORT==== 21-Jan-2015::20:16:30 === SSL: hello: tls_handshake.erl:116:Fatal error: insufficient security Now I thought this might be some sort of error due to the source of rabbitMQ so to rule that out I wrote a small erlang program to test ssl according to the manual here http://www.erlang.org/doc/apps/ssl/using_ssl.html . Unfortunately to no avail. Whenever I set the ciphers option to only include PFS ciphers I receive the same error. If I include aes256,sha256 for example, the connection setup works, using that cipher. Some searching around the web and on the OpenSSL site about the error code got me the conclusion that there is something wrong with the cipher negotiation between client and server. So on to wireshark and let's see what happens there - nope the client sends the correct cipher set, whether I use openssl s_client or a sample erlang ssl program. The server hower simply sends the above tls alert and closes the connection. This led me to the conclusion that the server cipher list is either empty or the matching between my sent list and the serverlist does not work. (Yes i tested a simple openssl s_client to s_server connection with the PFS cipher only - that works flawlessly) Unfortunately I have so far been unsuccessfull in debugging said part of the connection setup. I have tried my way around the dbg application but since I am more or less a newbee to erlang my attempts have been in vain. Which is why I finally write to the list: * How can I further debug this problem? Maybe a suggestion on the server code I used so far, posted here http://pastebin.com/ZfAtnSbU * Anyone got a hint why this does not work? Please have a look at the pastebin. I am running R17.4 and R16B03, on Ubuntu 14.04. Both incur the described problem. Many thanks in advance. -- Bernd May -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From nayibor@REDACTED Fri Jan 23 03:44:24 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Fri, 23 Jan 2015 02:44:24 +0000 Subject: [erlang-questions] unregistering a process during takeover Message-ID: hello . i am a bit of a noob at erlang and was having some problems and needed some help . i am trying out distributed erlang and i was trying to perform a fail-over and takeover . the fail over works fine but i am having some issues with the takeover . i created a release and i was able to run three nodes a(prim.),b(sec.),c(sec.) . the fail over works fine but the takeover gives me some errors when i try it . i get this error when i try restarting the primary node however with the application having failed over to node b and running well . =INFO REPORT==== 23-Jan-2015::02:20:57 === global: Name conflict terminating {m8ball_server,<2910.62.0>} =INFO REPORT==== 23-Jan-2015::02:20:57 === global: Name conflict terminating {m8ball_sup,<2910.61.0>} i know its got to do with supervisor:start_link({global,?MODULE}, ?MODULE, []). --top level supervisor gen_server:start_link({global, ?MODULE}, ?MODULE, [], []).--main gen server which i use in the top level supervisor and the main genserver i wanted to know how to use the start_phase(Phase, StartType, PhaseArgs) and re_register_name(Name, Pid) in application callback file and .app file to create a sort of synchronized start up procedure(code samples) or if there is an alternate way of tackling this problem of . *ps:lyse .great book!!. my first foray into fp,erlang .learning a lot .* *thanks !!* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Fri Jan 23 07:14:05 2015 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 23 Jan 2015 07:14:05 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: On Thu, Jan 22, 2015 at 6:38 PM, Roberto Ostinelli wrote: > Here's something I've tried which is successful in avoiding the memory > increase for binaries. > > Inside a loop, I used to have: > > [...] > <> = Data, > loop(Body, Rest); > > Now I force a binary copy to ensure that the reference to the original > full binary is easily removed: > > [...] > <> = Data, > Body = binary:copy(Body0), > Rest = binary:copy(Rest0), > loop(Body, Rest); > > This seems to have stabilized the memory usage reported by erlang:memory/0 > . > > However: > > - I believe this can only work if the copied binary are *heap* and not > *ref-c*, is this correct? > > Binary copy makes a new copy of the binary regardless of size, but it is only useful on ref-c binaries, and should be used to avoid keeping references to large binaries, when you keep and use a small part of the original binary. But in this case you make a copy of Data (-2 bytes) and thus will double up the memory until Data is gc'ed and refc reaches zero. So I do not see the point of the above nor of making it a list which will only explode your memory even more. > - Unfortunately, the BEAM process reported RES memory sill keeps > growing. > > Any other ideas? > r. > > > > > On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli > wrote: > >> Thank you Robert. >> I'm going to try a selective fullsweep_after. >> >> Could this also justify the process memory increase (which is more >> significant)? >> >> >> >> On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding >> wrote: >> >>> One thing you can see is that the size of the binary data is growing. >>> This space contains the large binaries (> 64 bytes) which are sent in >>> messages between processes. While this means that the messages become >>> (much) smaller and faster to send it takes a much longer time to detect >>> that they are no longer alive and can be reclaimed. Basically it takes >>> until all the processes they have passed through does a full garbage >>> collection. Setting fullsweep_after to 0 and doing explicit garbage >>> collects speeds up reclaiming the binaries. >>> >>> You could be much more selective in which processes you set >>> fullsweep_after to 0 and which ones you explicitly garbage collect. >>> >>> I don't know if the is *the* problem but it is *a* problem you have. >>> >>> Robert >>> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jan 23 07:35:27 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 23 Jan 2015 07:35:27 +0100 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: References: Message-ID: > On 23 Jan 2015, at 03:44, Nuku Ameyibor wrote: > > =INFO REPORT==== 23-Jan-2015::02:20:57 === > global: Name conflict terminating {m8ball_server,<2910.62.0>} This suggests that you?re connecting the nodes *after* starting your applications, so that the application processes have time to globally register themselves first. Global detects the name conflict when trying to merge the registries of the two nodes. You can for example use the ?distributed? option, described here: http://www.erlang.org/doc/apps/kernel/application.html#load-1 and here: http://www.erlang.org/doc/man/kernel_app.html BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Fri Jan 23 10:09:19 2015 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Fri, 23 Jan 2015 03:09:19 -0600 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: In our case, a key insight is something Robert mentioned earlier. To paraphrase it, "Your memory is not going to be reclaimed till *every* process that touched it has either been GC'd, or sleeps with the fishes". And our code had a few router processes that, quite literally, did nothing but pass binaries on from point A to point B - no reason to worry about *those* right? (hint: wrong) In the short run, walking the process chain and doing full-sweeps on *every* process that might (!!!) touch the binary. (we did that) In the longer run, see if you can swap out the routing processes with versions that respawn themselves in some form or the other (we did that too. eventually. was harder) Cheers On Fri, Jan 23, 2015 at 12:14 AM, Dan Gudmundsson wrote: > > > On Thu, Jan 22, 2015 at 6:38 PM, Roberto Ostinelli > wrote: > >> Here's something I've tried which is successful in avoiding the memory >> increase for binaries. >> >> Inside a loop, I used to have: >> >> [...] >> <> = Data, >> loop(Body, Rest); >> >> Now I force a binary copy to ensure that the reference to the original >> full binary is easily removed: >> >> [...] >> <> = Data, >> Body = binary:copy(Body0), >> Rest = binary:copy(Rest0), >> loop(Body, Rest); >> >> This seems to have stabilized the memory usage reported by >> erlang:memory/0. >> >> However: >> >> - I believe this can only work if the copied binary are *heap* and >> not *ref-c*, is this correct? >> >> Binary copy makes a new copy of the binary regardless of size, but it is > only useful on ref-c binaries, > and should be used to avoid keeping references to large binaries, when you > keep and use a small part > of the original binary. But in this case you make a copy of Data (-2 > bytes) and thus will double up the memory until Data is gc'ed and refc > reaches zero. So I do not see the point of the above nor of making it a > list which will > only explode your memory even more. > > >> - Unfortunately, the BEAM process reported RES memory sill keeps >> growing. >> >> Any other ideas? >> r. >> >> >> >> >> On Thu, Jan 22, 2015 at 6:11 PM, Roberto Ostinelli >> wrote: >> >>> Thank you Robert. >>> I'm going to try a selective fullsweep_after. >>> >>> Could this also justify the process memory increase (which is more >>> significant)? >>> >>> >>> >>> On Thu, Jan 22, 2015 at 6:00 PM, Robert Virding >>> wrote: >>> >>>> One thing you can see is that the size of the binary data is growing. >>>> This space contains the large binaries (> 64 bytes) which are sent in >>>> messages between processes. While this means that the messages become >>>> (much) smaller and faster to send it takes a much longer time to detect >>>> that they are no longer alive and can be reclaimed. Basically it takes >>>> until all the processes they have passed through does a full garbage >>>> collection. Setting fullsweep_after to 0 and doing explicit garbage >>>> collects speeds up reclaiming the binaries. >>>> >>>> You could be much more selective in which processes you set >>>> fullsweep_after to 0 and which ones you explicitly garbage collect. >>>> >>>> I don't know if the is *the* problem but it is *a* problem you have. >>>> >>>> Robert >>>> >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Google+ | Blog | Twitter | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Fri Jan 23 10:40:22 2015 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 23 Jan 2015 10:40:22 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Hello Roberto, I cannot find any information about which version of Erlang/OTP that you are running, nor which setting you have given to VM at startup? On Thu, Jan 22, 2015 at 5:33 PM, Roberto Ostinelli wrote: > > I get my erlang memory with this simple call (I just convert everything to > GB, thanks to Ferd and his article > https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole): > > Have you also read the part about the memory fragmentation issues that Mr Hebert described? Specifically running recon_alloc:memory(allocated,current) and recon_alloc:memory(usage, current) and checking those values during the test. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ct.radu.001@REDACTED Fri Jan 23 11:33:40 2015 From: ct.radu.001@REDACTED (CT Radu) Date: Fri, 23 Jan 2015 12:33:40 +0200 Subject: [erlang-questions] cowboy_router weird availability error In-Reply-To: References: Message-ID: Solved. The problem was a prepackaged cowboy (ebin dir) that was shipped as it is and packed in the release. The code wanted 1.0.x, the release was packed with 0.6.1. Sorry for the spam. Regards, Tiberiu 2015-01-22 12:32 GMT+02:00 CT Radu : > Hello everyone, > > I'm experiencing a weird behavior related to cowboy. > > I have an application that uses cowboy for the webinterface. > Prior to this month we used 0.6.1 and recently I started to update it to > 1.0.1 > > I get the following error only when I start the application on fedora 20 > (using R16B03-1) > > [error] <0.943.0>Undefined - Undefined:Undefined:Undefined - CRASH REPORT > Process <0.943.0> with 0 neighbours exited with reason: call to undefined > function cowboy_router:compile([{'_',[{[<<"/wsock">>]... > > The same code, compiled the same R16B03-1 on centos 6, when started, > properly finds > the cowboy_router module and executes the cowboy_router:compile code. > > Cowboy is included in app.src in the applications section. > Also (I think this is redundant) I've started it in the top application > module (the one with the suffix _app.erl). > > Still, I cannot start my application on fedora 20... > This application is packed as a release using relx. > > Any ideas where I could look or what I may have missed ? > > Many thanks, > Tiberiu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 13:26:27 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 13:26:27 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: <54C14448.3070804@gmail.com> References: <54C14448.3070804@gmail.com> Message-ID: Thank you Anton, I do not have such system yet in place, as this isn't a production environment (yet). Will definitely add one asap. Thanks, r. On Thu, Jan 22, 2015 at 7:41 PM, Anton Lebedevich wrote: > Hello. > > Do you have any external monitoring system in place (like statsderl or > exometer + graphite) that watches erlang memory consumption (process, > binary, ... total)? What is the long-term growth trend? Is it linear > or similar to capacitor charge (like > http://hades.mech.northwestern.edu/index.php/File:RC_charge_voltage.gif) > > From the data provided it looks like some subpopulation of processes > is growing their heaps. It might happen for mid- or small-sized > processes since top processes always have the same size. > Can you measure [memory, message_queue_len, total_heap_size, > heap_size, stack_size] for all processes several times (with the same > interval as you measured total memory growth) and compare histograms > of their memory consumption? Or upload that data somewhere and I'll > play with it in R to find what's changing inside. > > > Regards, > Anton Lebedevich. > > PS forgot to reply to the list > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 13:27:59 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 13:27:59 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Correct, however just after this call the process gets hibernated so Data (I assume) is getting gc'ed right away). On Fri, Jan 23, 2015 at 7:14 AM, Dan Gudmundsson wrote: > > Binary copy makes a new copy of the binary regardless of size, but it is > only useful on ref-c binaries, > and should be used to avoid keeping references to large binaries, when you > keep and use a small part > of the original binary. But in this case you make a copy of Data (-2 > bytes) and thus will double up the memory until Data is gc'ed and refc > reaches zero. So I do not see the point of the above nor of making it a > list which will > only explode your memory even more. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 13:30:08 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 13:30:08 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Thank you Mahesh. I even have a fullsweep_after 0 in the router process, even though it is literally never touching the data: the router API will lookup an element in a router ETS table, and will immediately send the data to that process - without being sent to the router itself. The swapping seems an interesting option and has also been suggested here above. Not sure how easily it can be done. On Fri, Jan 23, 2015 at 10:09 AM, Mahesh Paolini-Subramanya < mahesh@REDACTED> wrote: > In our case, a key insight is something Robert mentioned earlier. > To paraphrase it, "Your memory is not going to be reclaimed till *every* > process that touched it has either been GC'd, or sleeps with the fishes". > And our code had a few router processes that, quite literally, did nothing > but pass binaries on from point A to point B - no reason to worry about > *those* right? (hint: wrong) > > In the short run, walking the process chain and doing full-sweeps on > *every* process that might (!!!) touch the binary. (we did that) > In the longer run, see if you can swap out the routing processes with > versions that respawn themselves in some form or the other (we did that > too. eventually. was harder) > > Cheers > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Jan 23 13:55:40 2015 From: tony@REDACTED (Tony Rogvall) Date: Fri, 23 Jan 2015 13:55:40 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <27ABB33C-BA9E-44D4-8E40-6B03103C1AA4@rogvall.se> > On 23 jan 2015, at 13:27, Roberto Ostinelli wrote: > > Correct, however just after this call the process gets hibernated so Data (I assume) is getting gc'ed right away). > hibernate will garbage collect before going to sleep. > On Fri, Jan 23, 2015 at 7:14 AM, Dan Gudmundsson > wrote: > > Binary copy makes a new copy of the binary regardless of size, but it is only useful on ref-c binaries, > and should be used to avoid keeping references to large binaries, when you keep and use a small part > of the original binary. But in this case you make a copy of Data (-2 bytes) and thus will double up the memory until Data is gc'ed and refc reaches zero. So I do not see the point of the above nor of making it a list which will > only explode your memory even more. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From nayibor@REDACTED Fri Jan 23 12:09:23 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Fri, 23 Jan 2015 11:09:23 +0000 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: References: Message-ID: hi ulf , thanks. the takeover is working okay now !!! . config a is this [{kernel, [{distributed, [{m8ball, 5000, [a@REDACTED, {b@REDACTED, c@REDACTED}]}]}, {sync_nodes_mandatory, [b@REDACTED, c@REDACTED]}, {sync_nodes_timeout, 30000} ] } ]. i start each node with bin\werl.exe -sname c -config C:\wamp\www\erlang\proj\m8ball\apps\m8ball_1_0_0\config\c.config there are 3 nodes .their conf files are similar with sync_nodes_mandatory being slightly different . a small question . the first round of failover and takeover seems to work fine . but after the first round (start node a,b,c . kill a b does failover .kill b c does failover . restart a,b a does takeover from c killing c application on c .i restart shell on c ) * after the first round when i try to do a failover onto b by killing a it doesnt work .* was wondering how come since the first round of failover take over worked fine . On Fri, Jan 23, 2015 at 6:35 AM, Ulf Wiger wrote: > > On 23 Jan 2015, at 03:44, Nuku Ameyibor wrote: > > =INFO REPORT==== 23-Jan-2015::02:20:57 === > global: Name conflict terminating {m8ball_server,<2910.62.0>} > > > This suggests that you?re connecting the nodes *after* starting your > applications, so that the application processes have time to globally > register themselves first. Global detects the name conflict when trying to > merge the registries of the two nodes. > > You can for example use the ?distributed? option, described here: > http://www.erlang.org/doc/apps/kernel/application.html#load-1 > > and here: > > http://www.erlang.org/doc/man/kernel_app.html > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.jackson@REDACTED Fri Jan 23 12:19:48 2015 From: anthony.jackson@REDACTED (Anthony Jackson) Date: Fri, 23 Jan 2015 13:19:48 +0200 Subject: [erlang-questions] XMERL Atr String Escaping Message-ID: <4DFAAF76-D737-4868-9015-5055D514E2E1@patternmatched.com> Hi It would seem that XMERL doesn?t escape greater than characters in Attribute strings. Is this a bug, or is this intentional? Function to look at is ./xmerl-1.3.6/src/xmerl_lib.erl -> export_attribute/2 12> f(Packet), Packet = [{ussd,[ {'STRING',"this has > escape"}],["this has > escape", {cookie,[],[]}]}]. [{ussd,[{'STRING',"this has > escape"}], ["this has > escape",{cookie,[],[]}]}] 13> lists:flatten(xmerl:export_simple(Packet, xmerl_xml)). " escape\">this has > escape? -- Anthony From vladdu55@REDACTED Fri Jan 23 14:08:07 2015 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 23 Jan 2015 14:08:07 +0100 Subject: [erlang-questions] XMERL Atr String Escaping In-Reply-To: <4DFAAF76-D737-4868-9015-5055D514E2E1@patternmatched.com> References: <4DFAAF76-D737-4868-9015-5055D514E2E1@patternmatched.com> Message-ID: Hi! I guess that it's not escaped because it's not required to be. Only & and < must be, according to the spec (http://www.w3.org/TR/xml11/#syntax). > must be escaped in text when part of the string "]]>". Do you have a parser that chokes on that? regards, Vlad On Fri, Jan 23, 2015 at 12:19 PM, Anthony Jackson < anthony.jackson@REDACTED> wrote: > Hi > > It would seem that XMERL doesn?t escape greater than characters in > Attribute strings. > Is this a bug, or is this intentional? > > Function to look at is ./xmerl-1.3.6/src/xmerl_lib.erl -> > export_attribute/2 > > 12> f(Packet), Packet = [{ussd,[ {'STRING',"this has > escape"}],["this > has > escape", {cookie,[],[]}]}]. > [{ussd,[{'STRING',"this has > escape"}], > ["this has > escape",{cookie,[],[]}]}] > 13> lists:flatten(xmerl:export_simple(Packet, xmerl_xml)). > " escape\">this has > > escape? > > -- > Anthony > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 14:33:04 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 14:33:04 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: Hi Lucas, I'm using 17.4 on ubuntu LTS 14.04. The settings I use are: +K true +P 2000000 +Q 1000000 Please find here below an additional set of data with the changes I've stated here above (binary:copy/0 and the fullsweep_after 0 in the router). Of the recon library I'm using the following calls: recon_alloc:memory(allocated, current). recon_alloc:memory(used, current). recon_alloc:memory(usage, current). All of this data is taken at random intervals of time. - BEAM process RES memory:* 2.732 GB* - Erlang memory: [{total,1.7860298827290535}, {processes,1.4158401936292648}, {processes_used,1.4157484397292137}, {system,0.37018968909978867}, {atom,4.000673070549965e-4}, {atom_used,3.846092149615288e-4}, {binary,0.20867645740509033}, {code,0.009268132038414478}, {ets,0.004821933805942535}] - recon_alloc: allocated: 3015796080 (2.808 GB) used: 2161850416 usage: 0.7187714029859935 - BEAM process RES memory:* 2.813 GB* - Erlang memory: [{total,2.026990756392479}, {processes,1.6270370781421661}, {processes_used,1.6269719526171684}, {system,0.3999536782503128}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.23845425993204117}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3098895728 (2.886 GB) used: 2176172480 usage: 0.7023218278482198 - BEAM process RES memory:* 3.029 GB* - Erlang memory: [{total,2.351852521300316}, {processes,1.9361207410693169}, {processes_used,1.9360847249627113}, {system,0.415731780230999}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.2539009377360344}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3337524592 (3.108 GB) used: 2525365352 usage: 0.7548030747173055 - BEAM process RES memory:* 3.099 GB* - Erlang memory: [{total,2.0704088881611824}, {processes,1.6625376418232918}, {processes_used,1.6624245047569275}, {system,0.4078712463378906}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.24636883288621902}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3400623472 (3.167 GB) used: 2222575336 usage: 0.6552131374790817 - BEAM process RES memory:* 3.132 GB* - Erlang memory: [{total,2.367126949131489}, {processes,1.9388784170150757}, {processes_used,1.938723236322403}, {system,0.4282485321164131}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.2667432576417923}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3435644272 (3.200 GB) used: 2541469864 usage: 0.7397146313173368 - BEAM process RES memory:* 3.307 GB* - Erlang memory: [{total,2.379016488790512}, {processes,1.9780860394239426}, {processes_used,1.9779272973537445}, {system,0.4009304493665695}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.23943009227514267}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3619329392 (3.371 GB) used: 2554804000 usage: 0.704330124232303 - BEAM process RES memory:* 3.351 GB* - Erlang memory: [{total,2.607522390782833}, {processes,2.168950654566288}, {processes_used,2.1688189953565598}, {system,0.4385717362165451}, {atom,4.000673070549965e-4}, {atom_used,3.8593634963035583e-4}, {binary,0.2771267145872116}, {code,0.009311830624938011}, {ets,0.004802137613296509}] - recon_alloc: allocated: 3669321072 (3.417 GB) used: 2799919616 usage: 0.7629933213977411 - BEAM process RES memory:* 3.469 GB* - Erlang memory: [{total,2.2596140429377556}, {processes,1.8098593652248383}, {processes_used,1.8098137602210045}, {system,0.44975467771291733}, {atom,4.000673070549965e-4}, {atom_used,3.86836938560009e-4}, {binary,0.2881493419408798}, {code,0.009375222958624363}, {ets,0.00480380654335022}] - recon_alloc: allocated: 3789014384 (3.528 GB) used: 2425613912 usage: 0.6401929098614897 - BEAM process RES memory:* 3.660 GB* - Erlang memory: [{total,2.692381367087364}, {processes,2.2282255738973618}, {processes_used,2.228082850575447}, {system,0.46415579319000244}, {atom,4.000673070549965e-4}, {atom_used,3.86836938560009e-4}, {binary,0.30247989296913147}, {code,0.009375222958624363}, {ets,0.00480380654335022}] - recon_alloc: allocated: 3993933168 (3.719 GB) used: 2890714704 usage: 0.7233507625681828 - BEAM process RES memory:* 3.667 GB* - Erlang memory: [{total,2.4165985733270645}, {processes,1.9264011159539223}, {processes_used,1.9263720959424973}, {system,0.49019745737314224}, {atom,4.000673070549965e-4}, {atom_used,3.86836938560009e-4}, {binary,0.3284950777888298}, {code,0.009375222958624363}, {ets,0.00480380654335022}] - recon_alloc: allocated: 4001830256 (3.727 GB) used: 2594872464 usage: 0.6483950197811689 It looks like the memory allocated has some mismatch with the memory reported by the OS, but maybe this is just a timing issue (since top provides an average during a period of time)? Anyway, the BEAM process keeps on increasing. Also, it looks like memory usage bounces anywhere from 64-72% and I don't know if that's a good figure or not. This is what I get from a check on memory fragmentation: 1> recon_alloc:fragmentation(current). [{{eheap_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.6252789717100151}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,391670536}, {mbcs_carriers_size,626393264}]}, {{eheap_alloc,2}, [{sbcs_usage,0.6168021896258503}, {mbcs_usage,0.6893887270883688}, {sbcs_block_size,371384}, {sbcs_carriers_size,602112}, {mbcs_block_size,370926112}, {mbcs_carriers_size,538050736}]}, {{eheap_alloc,3}, [{sbcs_usage,0.9991333400321544}, {mbcs_usage,0.7006580932915004}, {sbcs_block_size,5091008}, {sbcs_carriers_size,5095424}, {mbcs_block_size,324091688}, {mbcs_carriers_size,462553264}]}, {{eheap_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.6924985776876923}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,305976264}, {mbcs_carriers_size,441843888}]}, {{eheap_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.6397496430493375}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,207536944}, {mbcs_carriers_size,324403376}]}, {{eheap_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.6660125315617468}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,166472816}, {mbcs_carriers_size,249954480}]}, {{eheap_alloc,6}, [{sbcs_usage,0.9980070153061225}, {mbcs_usage,0.6770963446791575}, {sbcs_block_size,600912}, {sbcs_carriers_size,602112}, {mbcs_block_size,169065768}, {mbcs_carriers_size,249692336}]}, {{eheap_alloc,7}, [{sbcs_usage,0.997382155987395}, {mbcs_usage,0.6925225022824623}, {sbcs_block_size,972296}, {sbcs_carriers_size,974848}, {mbcs_block_size,167834424}, {mbcs_carriers_size,242352304}]}, {{ll_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.7387279228243809}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,112706224}, {mbcs_carriers_size,152567976}]}, {{binary_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7614033191804973}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,58507096}, {mbcs_carriers_size,76841136}]}, {{binary_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.7179048085736076}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,46131288}, {mbcs_carriers_size,64258224}]}, {{binary_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.6128136272843548}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,27651200}, {mbcs_carriers_size,45121712}]}, {{binary_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.6492102167567332}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,32186648}, {mbcs_carriers_size,49578160}]}, {{binary_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.7715637758298944}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,56051664}, {mbcs_carriers_size,72646832}]}, {{binary_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.7514879308127178}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,49077272}, {mbcs_carriers_size,65306800}]}, {{binary_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.6757064168943689}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25706456}, {mbcs_carriers_size,38043824}]}, {{binary_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.7016146506167494}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25956408}, {mbcs_carriers_size,36995248}]}, {{fix_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.7409146075507547}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,22748888}, {mbcs_carriers_size,30703792}]}, {{fix_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.8581522751225302}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,33547232}, {mbcs_carriers_size,39092400}]}, {{fix_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.8630094940716118}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,26497664}, {mbcs_carriers_size,...}]}, {{driver_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7615924083651237}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,...}, {...}]}, {{fix_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.8947156992151927}, {sbcs_block_size,0}, {sbcs_carriers_size,...}, {...}|...]}, {{fix_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.7166474590927997}, {sbcs_block_size,...}, {...}|...]}, {{ll_alloc,7},[{sbcs_usage,1.0},{mbcs_usage,...},{...}|...]}, {{driver_alloc,2},[{sbcs_usage,...},{...}|...]}, {{driver_alloc,4},[{...}|...]}, {{ll_alloc,...},[...]}, {{...},...}, {...}|...] Unless I'm mistaken reading this data, everything looks fine there, with normal usages. What can I do else to address this issue? Thank you for your help. r. On Fri, Jan 23, 2015 at 10:40 AM, Lukas Larsson wrote: > Hello Roberto, > > I cannot find any information about which version of Erlang/OTP that you > are running, nor which setting you have given to VM at startup? > > On Thu, Jan 22, 2015 at 5:33 PM, Roberto Ostinelli > wrote: > >> >> I get my erlang memory with this simple call (I just convert everything >> to GB, thanks to Ferd and his article >> https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole): >> >> > Have you also read the part about the memory fragmentation issues that Mr > Hebert described? Specifically running > recon_alloc:memory(allocated,current) and recon_alloc:memory(usage, > current) and checking those values during the test. > > Lukas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.jackson@REDACTED Fri Jan 23 14:52:31 2015 From: anthony.jackson@REDACTED (Anthony Jackson) Date: Fri, 23 Jan 2015 15:52:31 +0200 Subject: [erlang-questions] XMERL Atr String Escaping In-Reply-To: References: <4DFAAF76-D737-4868-9015-5055D514E2E1@patternmatched.com> Message-ID: Hey Yes I am integrating with with an MNO and it would seem it breaks their XML decoder. I would be curious to know what they use, as its also written in Erlang. Looks like there isn?t a way to tweak the xmerl:export_simple/2 function. I have to obfuscate the gt character before the export, and post process the XML string.. regex to the rescue? :| -- Anthony On 23 Jan 2015, at 15:08, Vlad Dumitrescu wrote: > Hi! > > I guess that it's not escaped because it's not required to be. Only & and < must be, according to the spec (http://www.w3.org/TR/xml11/#syntax). > must be escaped in text when part of the string "]]>". > > Do you have a parser that chokes on that? > > regards, > Vlad > > > On Fri, Jan 23, 2015 at 12:19 PM, Anthony Jackson wrote: > Hi > > It would seem that XMERL doesn?t escape greater than characters in Attribute strings. > Is this a bug, or is this intentional? > > Function to look at is ./xmerl-1.3.6/src/xmerl_lib.erl -> export_attribute/2 > > 12> f(Packet), Packet = [{ussd,[ {'STRING',"this has > escape"}],["this has > escape", {cookie,[],[]}]}]. > [{ussd,[{'STRING',"this has > escape"}], > ["this has > escape",{cookie,[],[]}]}] > 13> lists:flatten(xmerl:export_simple(Packet, xmerl_xml)). > " escape\">this has > escape? > > -- > Anthony > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jan 23 14:58:05 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 23 Jan 2015 14:58:05 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Fri, Jan 23, 2015 at 1:26 PM, Roberto Ostinelli wrote: > I do not have such system yet in place, as this isn't a production > environment (yet). > > Will definitely add one asap. > In my experience, instrumentation of the running production/development system is one of the most important things to add early on. The views of the system it gives and the depth at which it explains what is going on will help in so many debugging scenarios you can't fathom it. My hunch is you are holding on to binary values for too long in some processes, or are storing a ref-c binary in ETS which keeps the underlying binary alive. The two major ways to handle this has already been mentioned. If you are picking apart a large binary, and you need to reference *binary* data in it, then it may be beneficial to break that binary data off: <> = Packet, CPayload = binary:copy(Payload), ... recurse_on(Rest). but as Dan says, copying `Rest` in this case is a bad idea. You just copy the very binary you are running over and processing. Also note that it is important you are picking out binary data. If you had something like <> then the integer conversion already excises the data from the binary, so you don't have to keep the binary around anymore. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 15:09:50 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 15:09:50 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: Hi Jesper, thank you for your input. My hunch is you are holding on to binary values for too long in some > processes, or are storing a ref-c binary in ETS which keeps the underlying > binary alive. The two major ways to handle this has already been mentioned. > I'm not storing binaries in a ETS. Also, I don't see anywhere I'm holding binary values for too long. > If you are picking apart a large binary, and you need to reference > *binary* data in it, then it may be beneficial to break that binary data > off: > > <> = Packet, > CPayload = binary:copy(Payload), > ... > recurse_on(Rest). > > but as Dan says, copying `Rest` in this case is a bad idea. You just copy > the very binary you are running over and processing. Also note that it is > important you are picking out binary data. If you had something like > > <> > > then the integer conversion already excises the data from the binary, so > you don't have to keep the binary around anymore. > Ok, as per this suggestion I removed copying the Rest. Thanks, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jan 23 15:26:36 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 23 Jan 2015 15:26:36 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Fri, Jan 23, 2015 at 3:09 PM, Roberto Ostinelli wrote: > I'm not storing binaries in a ETS. Also, I don't see anywhere I'm holding > binary values for too long. It is only a hunch. Clearly, something is using more memory than you expect, and I assume you have made calculations which shows that this memory usage is excessive given the number of processes and the specific profile of the system. Running 20000 SSL sockets will have some overhead due to SSL, so it is important to figure out if the memory usage is normal or unexpected in the first place. Also, check the operating system. If you are out of memory, and being killed by the OS, then there should be a log line in the kernels log about it. It is highly suspicious you get killed suddenly with no message at all. If Erlang terminates, you can arrange that it crash-dumps so you can go inspect the system afterwards. Alternatively, you can force it to terminate with a SIGUSR1 signal which will have it crash-dump. Luckily, you are in development mode, so being mean to the system is an option :) Other typical OS limitations are resources such as file descriptors. You need to verify that you are not being killed by the OS and debug from the bottom-up. Otherwise you will end up making a wrong conclusion somewhere along the way. Your goal should be to verify that what you *expect* is *reality*. So seek those conclusions. Another common thing I do, is to run mem_sup on the machine and then add in an alert for processes which take up more than 5% of the memory of the system. If such a process occurs, you go inspect it with process_info and figure out if it has a rather small backtrace. In that case, you log the backtrace fully and if it is too long, you log only the topmost parts of the call chain. All of this is possible by installing a custom alarm_handler. It is yet another of those things I like to have in production systems, because it quickly finds and reports odd processes that are doing nasty things to the system. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Jan 23 17:22:28 2015 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 23 Jan 2015 17:22:28 +0100 Subject: [erlang-questions] cipher negotiation problem in SSL application when using PFS only In-Reply-To: <54C17CBC.2060801@dv-team.de> References: <54C17CBC.2060801@dv-team.de> Message-ID: Hi! The alert you get means that the client and server had no cipher suites in common. You can use io:format("~p", [ssl:cipher_suites(openssl)]). io:format("~p", [ssl:cipher_suites(erlang)]). (io:format as the lists returned might be long and truncated) to find out what cipher suites that are available on your erlang node. It can differ depending on what openSSL crypto library that the crypto application is linked to. Regards Ingela Erlang/OTP team - Ericsson AB 2015-01-22 23:42 GMT+01:00 Bernd May : > Hi List, > > I have spent the past two days fiddling with erlang for the first time > in my life mainly because I want to use rabbitMQ in what seems to be a > yet more unusual way. During that fiddling I have encountered an error > that I have a hard time debugging. Here is the scenario: > > Whenever I try to configure RabbitMQ to use SSL with PFS ciphers (that > is DHE and ECDHE) only I receive the following error: > > =ERROR REPORT==== 21-Jan-2015::20:16:30 === > SSL: hello: tls_handshake.erl:116:Fatal error: insufficient security > > Now I thought this might be some sort of error due to the source of > rabbitMQ so to rule that out I wrote a small erlang program to test ssl > according to the manual here > http://www.erlang.org/doc/apps/ssl/using_ssl.html . Unfortunately to no > avail. Whenever I set the ciphers option to only include PFS ciphers I > receive the same error. If I include aes256,sha256 for example, the > connection setup works, using that cipher. > > Some searching around the web and on the OpenSSL site about the error > code got me the conclusion that there is something wrong with the cipher > negotiation between client and server. So on to wireshark and let's see > what happens there - nope the client sends the correct cipher set, > whether I use openssl s_client or a sample erlang ssl program. The > server hower simply sends the above tls alert and closes the connection. > This led me to the conclusion that the server cipher list is either > empty or the matching between my sent list and the serverlist does not > work. (Yes i tested a simple openssl s_client to s_server connection > with the PFS cipher only - that works flawlessly) > > Unfortunately I have so far been unsuccessfull in debugging said part of > the connection setup. I have tried my way around the dbg application but > since I am more or less a newbee to erlang my attempts have been in > vain. Which is why I finally write to the list: > > * How can I further debug this problem? Maybe a suggestion on the server > code I used so far, posted here http://pastebin.com/ZfAtnSbU > * Anyone got a hint why this does not work? Please have a look at the > pastebin. > > I am running R17.4 and R16B03, on Ubuntu 14.04. Both incur the described > problem. > > Many thanks in advance. > > -- > Bernd May > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 17:25:05 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 17:25:05 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: > > It is only a hunch. Clearly, something is using more memory than you > expect, and I assume you have made calculations which shows that this > memory usage is excessive given the number of processes and the specific > profile of the system. Running 20000 SSL sockets will have some overhead > due to SSL, so it is important to figure out if the memory usage is normal > or unexpected in the first place. > The main question here is: why is there such a difference between what the BEAM uses, and what Erlang VM uses. In another (bigger) test I am now seeing erlang:memory reporting a total of 10 GB, while the BEAM eats up to 15 GB. > Also, check the operating system. If you are out of memory, and being > killed by the OS, then there should be a log line in the kernels log about > it. It is highly suspicious you get killed suddenly with no message at all. > If Erlang terminates, you can arrange that it crash-dumps so you can go > inspect the system afterwards. Alternatively, you can force it to terminate > with a SIGUSR1 signal which will have it crash-dump. Luckily, you are in > development mode, so being mean to the system is an option :) > > Other typical OS limitations are resources such as file descriptors. You > need to verify that you are not being killed by the OS and debug from the > bottom-up. Otherwise you will end up making a wrong conclusion somewhere > along the way. Your goal should be to verify that what you *expect* is > *reality*. So seek those conclusions. > File descriptors are ok. I have 2M available, only using 220k. > Another common thing I do, is to run mem_sup on the machine and then add > in an alert for processes which take up more than 5% of the memory of the > system. If such a process occurs, you go inspect it with process_info and > figure out if it has a rather small backtrace. In that case, you log the > backtrace fully and if it is too long, you log only the topmost parts of > the call chain. All of this is possible by installing a custom > alarm_handler. > Doing this. No process is taking such a huge amount of memory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri Jan 23 18:03:16 2015 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 23 Jan 2015 18:03:16 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: Hello Roberto On Fri, Jan 23, 2015 at 5:25 PM, Roberto Ostinelli wrote: > It is only a hunch. Clearly, something is using more memory than you >> expect, and I assume you have made calculations which shows that this >> memory usage is excessive given the number of processes and the specific >> profile of the system. Running 20000 SSL sockets will have some overhead >> due to SSL, so it is important to figure out if the memory usage is normal >> or unexpected in the first place. >> > > The main question here is: why is there such a difference between what the > BEAM uses, and what Erlang VM uses. > > In another (bigger) test I am now seeing erlang:memory reporting a total > of 10 GB, while the BEAM eats up to 15 GB. > > With a recon_alloc:memory(usage, current) value of 65 - 70%, this difference is expected. What is happening is that the memory allocators of the VM is not finding any good slots to put new heaps in, so it keeps requesting more memory from the OS. erlang:memory reports used memory, which is not the same as the memory requested from the OS. recon_alloc:memory(allocated, current) reports something very close to what has actually been requested from the OS, which is why this value is much closer to what you see with top. It would be interesting to see if you get the same results if you start the VM with "+Muacul 0", or if you get better utlization if you use that. If you want to know what it is that you are disabling you can read this: https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CarrierMigration.md . Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From seriy.pr@REDACTED Fri Jan 23 18:29:06 2015 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Fri, 23 Jan 2015 20:29:06 +0300 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory Message-ID: Hi, Roberto. Looks like you have some sort of multiblock allocator fragmentation. As you can see in `recon_alloc:fragmentation(current).` output, your mbcs_usage ~0.7 is correlating with `recon_alloc:memory(usage, current).` So, problem isn't in refcounted binaries, but in multiblock allocator fragmantation. Also, note that worst utilized allocators is allocators for erlang process heaps (eheap_alloc), allocators for refc binaries (binary_alloc) are on 2'nd place. Erlang MBCS allocate large chunks of memory (carrier) and then return chunks of it (so called blocks) by-demand [ busy1 | busy2 | busy3 | free ....... ] When some erlang sybsystem releases used memory (eg, process dies or binary released), this space is marked as free on carrier and pointer to it is placed to binary tree for being reused [ busy1 | busy2 | busy3 | free1 | busy4 | free2 ] *--------------------^ ^ a b*-------------------------------| \ / free_blocks_tree Whole carrier will be deallocated and released to OS only when all blocks on it are free. There is no such thing like carrier defragmentation. But, AFAIK, erlang may terurn (shrink) to OS free blocks from carrier's tail (eg, free2 from pic). Plus 2 free neighbours will be merged to one (eg, if busy4 will be freed, ti will be merged to single block with free1 and free2). As you can see, carrier on my "picture" has some holes. It's memory utilization is ~ `sum(busy) / (sum(free) + sum(busy))`. Also, if beam asks free_blocks_tree for a free block of certain size, but it has only bigger one, it splits it to 2 blocks, leading to additional fragmentation. There are some strategies to improve this situation: 1) make less alloc/dealloc operations (you may try to make larger min_heap_size for new processes to reduce GC frequency) 2) change `free_blocks_tree` sorting order so it will be address-ordered and blocks from tail may be returned to OS faster 3) tune mbcs_block_size and mbcs_carrier_size to fit better to your most common data's size to reduce fragmentation by helping free blocks reusage. Which one fits your needs I don't know. You may consult this doc http://erlang.org/doc/man/erts_alloc.html for more details. Hi Lucas, > I'm using 17.4 on ubuntu LTS 14.04. The settings I use are: > +K true > +P 2000000 > +Q 1000000 > Please find here below an additional set of data with the changes I've > stated here above (binary:copy/0 and the fullsweep_after 0 in the router). > Of the recon library I'm using the following calls: > recon_alloc:memory(allocated, current). > recon_alloc:memory(used, current). > recon_alloc:memory(usage, current). > > All of this data is taken at random intervals of time. > - BEAM process RES memory:* 2.732 GB* > - Erlang memory: > [{total,1.7860298827290535}, > {processes,1.4158401936292648}, > {processes_used,1.4157484397292137}, > {system,0.37018968909978867}, > {atom,4.000673070549965e-4}, > {atom_used,3.846092149615288e-4}, > {binary,0.20867645740509033}, > {code,0.009268132038414478}, > {ets,0.004821933805942535}] > - recon_alloc: > allocated: 3015796080 (2.808 GB) > used: 2161850416 > usage: 0.7187714029859935 > > - BEAM process RES memory:* 2.813 GB* > - Erlang memory: > [{total,2.026990756392479}, > {processes,1.6270370781421661}, > {processes_used,1.6269719526171684}, > {system,0.3999536782503128}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.23845425993204117}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3098895728 (2.886 GB) > used: 2176172480 > usage: 0.7023218278482198 > > - BEAM process RES memory:* 3.029 GB* > - Erlang memory: > [{total,2.351852521300316}, > {processes,1.9361207410693169}, > {processes_used,1.9360847249627113}, > {system,0.415731780230999}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.2539009377360344}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3337524592 (3.108 GB) > used: 2525365352 > usage: 0.7548030747173055 > > - BEAM process RES memory:* 3.099 GB* > - Erlang memory: > [{total,2.0704088881611824}, > {processes,1.6625376418232918}, > {processes_used,1.6624245047569275}, > {system,0.4078712463378906}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.24636883288621902}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3400623472 (3.167 GB) > used: 2222575336 > usage: 0.6552131374790817 > > - BEAM process RES memory:* 3.132 GB* > - Erlang memory: > [{total,2.367126949131489}, > {processes,1.9388784170150757}, > {processes_used,1.938723236322403}, > {system,0.4282485321164131}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.2667432576417923}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3435644272 (3.200 GB) > used: 2541469864 > usage: 0.7397146313173368 > > - BEAM process RES memory:* 3.307 GB* > - Erlang memory: > [{total,2.379016488790512}, > {processes,1.9780860394239426}, > {processes_used,1.9779272973537445}, > {system,0.4009304493665695}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.23943009227514267}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3619329392 (3.371 GB) > used: 2554804000 > usage: 0.704330124232303 > > - BEAM process RES memory:* 3.351 GB* > - Erlang memory: > [{total,2.607522390782833}, > {processes,2.168950654566288}, > {processes_used,2.1688189953565598}, > {system,0.4385717362165451}, > {atom,4.000673070549965e-4}, > {atom_used,3.8593634963035583e-4}, > {binary,0.2771267145872116}, > {code,0.009311830624938011}, > {ets,0.004802137613296509}] > - recon_alloc: > allocated: 3669321072 (3.417 GB) > used: 2799919616 > usage: 0.7629933213977411 > > - BEAM process RES memory:* 3.469 GB* > - Erlang memory: > [{total,2.2596140429377556}, > {processes,1.8098593652248383}, > {processes_used,1.8098137602210045}, > {system,0.44975467771291733}, > {atom,4.000673070549965e-4}, > {atom_used,3.86836938560009e-4}, > {binary,0.2881493419408798}, > {code,0.009375222958624363}, > {ets,0.00480380654335022}] > - recon_alloc: > allocated: 3789014384 (3.528 GB) > used: 2425613912 > usage: 0.6401929098614897 > > - BEAM process RES memory:* 3.660 GB* > - Erlang memory: > [{total,2.692381367087364}, > {processes,2.2282255738973618}, > {processes_used,2.228082850575447}, > {system,0.46415579319000244}, > {atom,4.000673070549965e-4}, > {atom_used,3.86836938560009e-4}, > {binary,0.30247989296913147}, > {code,0.009375222958624363}, > {ets,0.00480380654335022}] > - recon_alloc: > allocated: 3993933168 (3.719 GB) > used: 2890714704 > usage: 0.7233507625681828 > > - BEAM process RES memory:* 3.667 GB* > - Erlang memory: > [{total,2.4165985733270645}, > {processes,1.9264011159539223}, > {processes_used,1.9263720959424973}, > {system,0.49019745737314224}, > {atom,4.000673070549965e-4}, > {atom_used,3.86836938560009e-4}, > {binary,0.3284950777888298}, > {code,0.009375222958624363}, > {ets,0.00480380654335022}] > - recon_alloc: > allocated: 4001830256 (3.727 GB) > used: 2594872464 > usage: 0.6483950197811689 > > It looks like the memory allocated has some mismatch with the memory > reported by the OS, but maybe this is just a timing issue (since top > provides an average during a period of time)? > Anyway, the BEAM process keeps on increasing. Also, it looks like memory > usage bounces anywhere from 64-72% and I don't know if that's a good figure > or not. > This is what I get from a check on memory fragmentation: > 1> recon_alloc:fragmentation(current). > [{{eheap_alloc,1}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6252789717100151}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,391670536}, > {mbcs_carriers_size,626393264}]}, > {{eheap_alloc,2}, > [{sbcs_usage,0.6168021896258503}, > {mbcs_usage,0.6893887270883688}, > {sbcs_block_size,371384}, > {sbcs_carriers_size,602112}, > {mbcs_block_size,370926112}, > {mbcs_carriers_size,538050736}]}, > {{eheap_alloc,3}, > [{sbcs_usage,0.9991333400321544}, > {mbcs_usage,0.7006580932915004}, > {sbcs_block_size,5091008}, > {sbcs_carriers_size,5095424}, > {mbcs_block_size,324091688}, > {mbcs_carriers_size,462553264}]}, > {{eheap_alloc,4}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6924985776876923}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,305976264}, > {mbcs_carriers_size,441843888}]}, > {{eheap_alloc,5}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6397496430493375}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,207536944}, > {mbcs_carriers_size,324403376}]}, > {{eheap_alloc,8}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6660125315617468}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,166472816}, > {mbcs_carriers_size,249954480}]}, > {{eheap_alloc,6}, > [{sbcs_usage,0.9980070153061225}, > {mbcs_usage,0.6770963446791575}, > {sbcs_block_size,600912}, > {sbcs_carriers_size,602112}, > {mbcs_block_size,169065768}, > {mbcs_carriers_size,249692336}]}, > {{eheap_alloc,7}, > [{sbcs_usage,0.997382155987395}, > {mbcs_usage,0.6925225022824623}, > {sbcs_block_size,972296}, > {sbcs_carriers_size,974848}, > {mbcs_block_size,167834424}, > {mbcs_carriers_size,242352304}]}, > {{ll_alloc,0}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7387279228243809}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,112706224}, > {mbcs_carriers_size,152567976}]}, > {{binary_alloc,1}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7614033191804973}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,58507096}, > {mbcs_carriers_size,76841136}]}, > {{binary_alloc,4}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7179048085736076}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,46131288}, > {mbcs_carriers_size,64258224}]}, > {{binary_alloc,7}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6128136272843548}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,27651200}, > {mbcs_carriers_size,45121712}]}, > {{binary_alloc,5}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6492102167567332}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,32186648}, > {mbcs_carriers_size,49578160}]}, > {{binary_alloc,2}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7715637758298944}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,56051664}, > {mbcs_carriers_size,72646832}]}, > {{binary_alloc,3}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7514879308127178}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,49077272}, > {mbcs_carriers_size,65306800}]}, > {{binary_alloc,6}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.6757064168943689}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,25706456}, > {mbcs_carriers_size,38043824}]}, > {{binary_alloc,8}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7016146506167494}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,25956408}, > {mbcs_carriers_size,36995248}]}, > {{fix_alloc,4}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7409146075507547}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,22748888}, > {mbcs_carriers_size,30703792}]}, > {{fix_alloc,2}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.8581522751225302}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,33547232}, > {mbcs_carriers_size,39092400}]}, > {{fix_alloc,3}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.8630094940716118}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,26497664}, > {mbcs_carriers_size,...}]}, > {{driver_alloc,1}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7615924083651237}, > {sbcs_block_size,0}, > {sbcs_carriers_size,0}, > {mbcs_block_size,...}, > {...}]}, > {{fix_alloc,1}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.8947156992151927}, > {sbcs_block_size,0}, > {sbcs_carriers_size,...}, > {...}|...]}, > {{fix_alloc,5}, > [{sbcs_usage,1.0}, > {mbcs_usage,0.7166474590927997}, > {sbcs_block_size,...}, > {...}|...]}, > {{ll_alloc,7},[{sbcs_usage,1.0},{mbcs_usage,...},{...}|...]}, > {{driver_alloc,2},[{sbcs_usage,...},{...}|...]}, > {{driver_alloc,4},[{...}|...]}, > {{ll_alloc,...},[...]}, > {{...},...}, > {...}|...] > > Unless I'm mistaken reading this data, everything looks fine there, with > normal usages. > What can I do else to address this issue? > Thank you for your help. > r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jan 23 18:49:11 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 23 Jan 2015 18:49:11 +0100 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: References: Message-ID: <34DFA5F6-2237-45F5-9E90-AA3C1636E7B2@feuerlabs.com> Not sure based on what you?ve described. A possibility is that you?ve forgotten to start the application on b. The next step, though, would be to also use a boot script. You could use rebar, relx, OTP systools or my personal pet project, setup [1]. Using a boot script, your applications are started automatically, or in the case of distributed apps, at least prepared to start, so that failover/takeover can work. BR, Ulf W [1] http://github.com/uwiger/setup https://github.com/uwiger/setup/wiki/Example%3A-generating-boot-scripts-from-a-running-system > On 23 Jan 2015, at 12:09, Nuku Ameyibor wrote: > > hi ulf , > thanks. > the takeover is working okay now !!! . > config a is this > > [{kernel, > [{distributed, [{m8ball, > 5000, > [a@REDACTED, {b@REDACTED, c@REDACTED}]}]}, > {sync_nodes_mandatory, [b@REDACTED, c@REDACTED]}, > {sync_nodes_timeout, 30000} > ] > } > ]. > > i start each node with > bin\werl.exe -sname c -config C:\wamp\www\erlang\proj\m8ball\apps\m8ball_1_0_0\config\c.config > > there are 3 nodes .their conf files are similar with sync_nodes_mandatory being slightly different . > > a small question . > the first round of failover and takeover seems to work fine . > but after the first round (start node a,b,c . kill a b does failover .kill b c does failover . restart a,b a does takeover from c killing c application on c .i restart shell on c ) > > after the first round when i try to do a failover onto b by killing a it doesnt work . > was wondering how come since the first round of failover take over worked fine . > > > > > On Fri, Jan 23, 2015 at 6:35 AM, Ulf Wiger > wrote: > >> On 23 Jan 2015, at 03:44, Nuku Ameyibor > wrote: >> >> =INFO REPORT==== 23-Jan-2015::02:20:57 === >> global: Name conflict terminating {m8ball_server,<2910.62.0>} > > This suggests that you?re connecting the nodes *after* starting your applications, so that the application processes have time to globally register themselves first. Global detects the name conflict when trying to merge the registries of the two nodes. > > You can for example use the ?distributed? option, described here: > http://www.erlang.org/doc/apps/kernel/application.html#load-1 > > and here: > > http://www.erlang.org/doc/man/kernel_app.html > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Jan 23 19:07:03 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 19:07:03 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Fri, Jan 23, 2015 at 6:03 PM, Lukas Larsson wrote: > With a recon_alloc:memory(usage, current) value of 65 - 70%, this > difference is expected. What is happening is that the memory allocators of > the VM is not finding any good slots to put new heaps in, so it keeps > requesting more memory from the OS. erlang:memory reports used memory, > which is not the same as the memory requested from the OS. > recon_alloc:memory(allocated, current) reports something very close to what > has actually been requested from the OS, which is why this value is much > closer to what you see with top. > > It would be interesting to see if you get the same results if you start > the VM with "+Muacul 0", or if you get better utlization if you use that. > If you want to know what it is that you are disabling you can read this: > https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CarrierMigration.md > . > > Lukas > Thank you Lukas I will try this option. I have an update though, which is interesting. I am using lager as the logger, and in the logger I was printing out the body of the original request that I received. Since the requests are short lived (and the processes die after the logging), I imagined that the data would be freed. Instead, it looks like for some reason it is not. I have disabled the logging and the BEAM size usage is much more stable (even though it periodically goes through the forced GC that I've set up when thresholds are hit). I'll keep you posted on the "+Muacul 0" option. Thanks, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Fri Jan 23 19:40:12 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 19:40:12 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <80A20EAE-006B-4492-83A8-72CDF86D8AD5@widetag.com> Thank you Cepren, Yes it looks like I have also some fragmentation issue. Will try some options there as I was suggested. Thanks, r. > On 23/gen/2015, at 18:29, ?????? ???????? wrote: > > Hi, Roberto. > > Looks like you have some sort of multiblock allocator fragmentation. > As you can see in `recon_alloc:fragmentation(current).` output, your mbcs_usage ~0.7 is correlating with `recon_alloc:memory(usage, current).` > > So, problem isn't in refcounted binaries, but in multiblock allocator fragmantation. > Also, note that worst utilized allocators is allocators for erlang process heaps (eheap_alloc), allocators for refc binaries (binary_alloc) are on 2'nd place. > > Erlang MBCS allocate large chunks of memory (carrier) and then return chunks of it (so called blocks) by-demand > [ busy1 | busy2 | busy3 | free ....... ] > > When some erlang sybsystem releases used memory (eg, process dies or binary released), this space is marked as free on carrier and pointer to it is placed to binary tree for being reused > > [ busy1 | busy2 | busy3 | free1 | busy4 | free2 ] > > *--------------------^ ^ > a b*-------------------------------| > \ / > free_blocks_tree > > Whole carrier will be deallocated and released to OS only when all blocks on it are free. There is no such thing like carrier defragmentation. But, AFAIK, erlang may terurn (shrink) to OS free blocks from carrier's tail (eg, free2 from pic). Plus 2 free neighbours will be merged to one (eg, if busy4 will be freed, ti will be merged to single block with free1 and free2). > > As you can see, carrier on my "picture" has some holes. It's memory utilization is ~ `sum(busy) / (sum(free) + sum(busy))`. > > Also, if beam asks free_blocks_tree for a free block of certain size, but it has only bigger one, it splits it to 2 blocks, leading to additional fragmentation. > > There are some strategies to improve this situation: > 1) make less alloc/dealloc operations (you may try to make larger min_heap_size for new processes to reduce GC frequency) > 2) change `free_blocks_tree` sorting order so it will be address-ordered and blocks from tail may be returned to OS faster > 3) tune mbcs_block_size and mbcs_carrier_size to fit better to your most common data's size to reduce fragmentation by helping free blocks reusage. > > Which one fits your needs I don't know. You may consult this doc http://erlang.org/doc/man/erts_alloc.html for more details. > >> Hi Lucas, >> I'm using 17.4 on ubuntu LTS 14.04. The settings I use are: >> +K true >> +P 2000000 >> +Q 1000000 >> Please find here below an additional set of data with the changes I've >> stated here above (binary:copy/0 and the fullsweep_after 0 in the router). >> Of the recon library I'm using the following calls: >> recon_alloc:memory(allocated, current). >> recon_alloc:memory(used, current). >> recon_alloc:memory(usage, current). >> >> All of this data is taken at random intervals of time. >> - BEAM process RES memory:* 2.732 GB* >> - Erlang memory: >> [{total,1.7860298827290535}, >> {processes,1.4158401936292648}, >> {processes_used,1.4157484397292137}, >> {system,0.37018968909978867}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.846092149615288e-4}, >> {binary,0.20867645740509033}, >> {code,0.009268132038414478}, >> {ets,0.004821933805942535}] >> - recon_alloc: >> allocated: 3015796080 (2.808 GB) >> used: 2161850416 >> usage: 0.7187714029859935 >> >> - BEAM process RES memory:* 2.813 GB* >> - Erlang memory: >> [{total,2.026990756392479}, >> {processes,1.6270370781421661}, >> {processes_used,1.6269719526171684}, >> {system,0.3999536782503128}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.23845425993204117}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3098895728 (2.886 GB) >> used: 2176172480 >> usage: 0.7023218278482198 >> >> - BEAM process RES memory:* 3.029 GB* >> - Erlang memory: >> [{total,2.351852521300316}, >> {processes,1.9361207410693169}, >> {processes_used,1.9360847249627113}, >> {system,0.415731780230999}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2539009377360344}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3337524592 (3.108 GB) >> used: 2525365352 >> usage: 0.7548030747173055 >> >> - BEAM process RES memory:* 3.099 GB* >> - Erlang memory: >> [{total,2.0704088881611824}, >> {processes,1.6625376418232918}, >> {processes_used,1.6624245047569275}, >> {system,0.4078712463378906}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.24636883288621902}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3400623472 (3.167 GB) >> used: 2222575336 >> usage: 0.6552131374790817 >> >> - BEAM process RES memory:* 3.132 GB* >> - Erlang memory: >> [{total,2.367126949131489}, >> {processes,1.9388784170150757}, >> {processes_used,1.938723236322403}, >> {system,0.4282485321164131}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2667432576417923}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3435644272 (3.200 GB) >> used: 2541469864 >> usage: 0.7397146313173368 >> >> - BEAM process RES memory:* 3.307 GB* >> - Erlang memory: >> [{total,2.379016488790512}, >> {processes,1.9780860394239426}, >> {processes_used,1.9779272973537445}, >> {system,0.4009304493665695}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.23943009227514267}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3619329392 (3.371 GB) >> used: 2554804000 >> usage: 0.704330124232303 >> >> - BEAM process RES memory:* 3.351 GB* >> - Erlang memory: >> [{total,2.607522390782833}, >> {processes,2.168950654566288}, >> {processes_used,2.1688189953565598}, >> {system,0.4385717362165451}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2771267145872116}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3669321072 (3.417 GB) >> used: 2799919616 >> usage: 0.7629933213977411 >> >> - BEAM process RES memory:* 3.469 GB* >> - Erlang memory: >> [{total,2.2596140429377556}, >> {processes,1.8098593652248383}, >> {processes_used,1.8098137602210045}, >> {system,0.44975467771291733}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.2881493419408798}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 3789014384 (3.528 GB) >> used: 2425613912 >> usage: 0.6401929098614897 >> >> - BEAM process RES memory:* 3.660 GB* >> - Erlang memory: >> [{total,2.692381367087364}, >> {processes,2.2282255738973618}, >> {processes_used,2.228082850575447}, >> {system,0.46415579319000244}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.30247989296913147}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 3993933168 (3.719 GB) >> used: 2890714704 >> usage: 0.7233507625681828 >> >> - BEAM process RES memory:* 3.667 GB* >> - Erlang memory: >> [{total,2.4165985733270645}, >> {processes,1.9264011159539223}, >> {processes_used,1.9263720959424973}, >> {system,0.49019745737314224}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.3284950777888298}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 4001830256 (3.727 GB) >> used: 2594872464 >> usage: 0.6483950197811689 >> >> It looks like the memory allocated has some mismatch with the memory >> reported by the OS, but maybe this is just a timing issue (since top >> provides an average during a period of time)? >> Anyway, the BEAM process keeps on increasing. Also, it looks like memory >> usage bounces anywhere from 64-72% and I don't know if that's a good figure >> or not. >> This is what I get from a check on memory fragmentation: >> 1> recon_alloc:fragmentation(current). >> [{{eheap_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6252789717100151}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,391670536}, >> {mbcs_carriers_size,626393264}]}, >> {{eheap_alloc,2}, >> [{sbcs_usage,0.6168021896258503}, >> {mbcs_usage,0.6893887270883688}, >> {sbcs_block_size,371384}, >> {sbcs_carriers_size,602112}, >> {mbcs_block_size,370926112}, >> {mbcs_carriers_size,538050736}]}, >> {{eheap_alloc,3}, >> [{sbcs_usage,0.9991333400321544}, >> {mbcs_usage,0.7006580932915004}, >> {sbcs_block_size,5091008}, >> {sbcs_carriers_size,5095424}, >> {mbcs_block_size,324091688}, >> {mbcs_carriers_size,462553264}]}, >> {{eheap_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6924985776876923}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,305976264}, >> {mbcs_carriers_size,441843888}]}, >> {{eheap_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6397496430493375}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,207536944}, >> {mbcs_carriers_size,324403376}]}, >> {{eheap_alloc,8}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6660125315617468}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,166472816}, >> {mbcs_carriers_size,249954480}]}, >> {{eheap_alloc,6}, >> [{sbcs_usage,0.9980070153061225}, >> {mbcs_usage,0.6770963446791575}, >> {sbcs_block_size,600912}, >> {sbcs_carriers_size,602112}, >> {mbcs_block_size,169065768}, >> {mbcs_carriers_size,249692336}]}, >> {{eheap_alloc,7}, >> [{sbcs_usage,0.997382155987395}, >> {mbcs_usage,0.6925225022824623}, >> {sbcs_block_size,972296}, >> {sbcs_carriers_size,974848}, >> {mbcs_block_size,167834424}, >> {mbcs_carriers_size,242352304}]}, >> {{ll_alloc,0}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7387279228243809}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,112706224}, >> {mbcs_carriers_size,152567976}]}, >> {{binary_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7614033191804973}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,58507096}, >> {mbcs_carriers_size,76841136}]}, >> {{binary_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7179048085736076}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,46131288}, >> {mbcs_carriers_size,64258224}]}, >> {{binary_alloc,7}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6128136272843548}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,27651200}, >> {mbcs_carriers_size,45121712}]}, >> {{binary_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6492102167567332}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,32186648}, >> {mbcs_carriers_size,49578160}]}, >> {{binary_alloc,2}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7715637758298944}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,56051664}, >> {mbcs_carriers_size,72646832}]}, >> {{binary_alloc,3}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7514879308127178}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,49077272}, >> {mbcs_carriers_size,65306800}]}, >> {{binary_alloc,6}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6757064168943689}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,25706456}, >> {mbcs_carriers_size,38043824}]}, >> {{binary_alloc,8}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7016146506167494}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,25956408}, >> {mbcs_carriers_size,36995248}]}, >> {{fix_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7409146075507547}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,22748888}, >> {mbcs_carriers_size,30703792}]}, >> {{fix_alloc,2}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8581522751225302}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,33547232}, >> {mbcs_carriers_size,39092400}]}, >> {{fix_alloc,3}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8630094940716118}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,26497664}, >> {mbcs_carriers_size,...}]}, >> {{driver_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7615924083651237}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,...}, >> {...}]}, >> {{fix_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8947156992151927}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,...}, >> {...}|...]}, >> {{fix_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7166474590927997}, >> {sbcs_block_size,...}, >> {...}|...]}, >> {{ll_alloc,7},[{sbcs_usage,1.0},{mbcs_usage,...},{...}|...]}, >> {{driver_alloc,2},[{sbcs_usage,...},{...}|...]}, >> {{driver_alloc,4},[{...}|...]}, >> {{ll_alloc,...},[...]}, >> {{...},...}, >> {...}|...] >> >> Unless I'm mistaken reading this data, everything looks fine there, with >> normal usages. >> What can I do else to address this issue? >> Thank you for your help. >> r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Jan 23 19:47:29 2015 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 23 Jan 2015 10:47:29 -0800 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <54C29741.9090204@gmail.com> On 01/22/2015 08:33 AM, Roberto Ostinelli wrote: > Dear List, > I'm having some troubles in pinpointing why a node is crashing due to memory issues. > For info, when it crashes, it does not produce a crash dump. However I've monitored live and I've seen the .beam process eat up all memory until it abruptly exits. > > The system is a big router that relays data coming from TCP connections, into other TCP connections. > I'm using cowboy as the HTTP server that initiates the long-lived TCP connections. > You have an Erlang architectural problem due to how your processes hold onto old binary references based on your other posts (you are using long-lived processes which process binaries quick enough that they cause excessive memory consumption). For an architectural problem fullsweep_after, hibernate, and calling erlang:garbage_collect/0 are not solutions, they are just delaying the inevitable death of your Erlang node due to the throughput overloading your source code (in addition to the Erlang VM memory tweaks described at https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole). Those things are what you think about once you get source code (an architecture) that works. You need to make sure you use temporary Erlang processes when excessive binary garbage is created, so that GC is done quickly (the Erlang node will not be overloaded then, accumulating memory). If you don't believe me, this is already proven in CloudI. The 1.4.0 loadtests shows a situation similar to your scenario: many HTTP cowboy connections sending data into other TCP connections (http://cloudi.org/faq.html#5_LoadTesting). The cowboy connections are handled by a CloudI service called cloudi_service_http_cowboy and I use Tsung to send HTTP traffic (20000 connections doing 10000 requests/second) to a single TCP connection bottleneck because it is most useful to test bottlenecks (in CloudI, this is a service that is configured with a process count and thread count of 1). The service process count and the thread count (used by non-Erlang (external) services) can easily be increased to increase the number of TCP connections used by an external service (where the service instance total TCP connections count is count_process * count_thread). These TCP connections are what the CloudI API uses in OS processes to facilitate external services, so I am describing CloudI details that relate to your problem. If you were to solve this problem using CloudI's service abstraction you would be using either cloudi_service_tcp or your own internal CloudI service (an Erlang or Elixir service) to facilitate your own TCP connections with your specific configuration and processing you need (so that depends on business logic). The reason CloudI doesn't have problems in internal services when creating lots of binary garbage is due to using temporary processes for service request processing. That is controlled by a service configuration option called 'request_pid_uses' (after processing X service requests, create a new temporary process... the setting defaults to 1 so each service request uses a new temporary process while still keeping the service's processing transactional: http://cloudi.org/api.html#2_services_add_config_opts). To see this benefit in a simpler way, there are some results that show how an Erlang service can facilitate large throughput based on how many service processes are used: https://github.com/CloudI/CloudI/blob/develop/src/tests/request_rate/results/results_v1_4_0/results.txt#L26-L46 . That testing is separate from the Tsung/HTTP loadtests and is focused on service request traffic between two CloudI services, both internal Erlang services, showing the maximum throughput facilitated with specific service configuration options. The higher throughput is using a 'lazy' destination refresh method to avoid contention on process lookup, so using cached data (the 'lazy' setting is set on the sending service in its service configuration: http://cloudi.org/api.html#1_Intro_dest). Reusing CloudI is easier and simpler than experiencing these NIH-syndrome problems you have described. From roberto.ostinelli@REDACTED Fri Jan 23 19:40:12 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 19:40:12 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <80A20EAE-006B-4492-83A8-72CDF86D8AD5@widetag.com> Thank you Cepren, Yes it looks like I have also some fragmentation issue. Will try some options there as I was suggested. Thanks, r. > On 23/gen/2015, at 18:29, ?????? ???????? wrote: > > Hi, Roberto. > > Looks like you have some sort of multiblock allocator fragmentation. > As you can see in `recon_alloc:fragmentation(current).` output, your mbcs_usage ~0.7 is correlating with `recon_alloc:memory(usage, current).` > > So, problem isn't in refcounted binaries, but in multiblock allocator fragmantation. > Also, note that worst utilized allocators is allocators for erlang process heaps (eheap_alloc), allocators for refc binaries (binary_alloc) are on 2'nd place. > > Erlang MBCS allocate large chunks of memory (carrier) and then return chunks of it (so called blocks) by-demand > [ busy1 | busy2 | busy3 | free ....... ] > > When some erlang sybsystem releases used memory (eg, process dies or binary released), this space is marked as free on carrier and pointer to it is placed to binary tree for being reused > > [ busy1 | busy2 | busy3 | free1 | busy4 | free2 ] > > *--------------------^ ^ > a b*-------------------------------| > \ / > free_blocks_tree > > Whole carrier will be deallocated and released to OS only when all blocks on it are free. There is no such thing like carrier defragmentation. But, AFAIK, erlang may terurn (shrink) to OS free blocks from carrier's tail (eg, free2 from pic). Plus 2 free neighbours will be merged to one (eg, if busy4 will be freed, ti will be merged to single block with free1 and free2). > > As you can see, carrier on my "picture" has some holes. It's memory utilization is ~ `sum(busy) / (sum(free) + sum(busy))`. > > Also, if beam asks free_blocks_tree for a free block of certain size, but it has only bigger one, it splits it to 2 blocks, leading to additional fragmentation. > > There are some strategies to improve this situation: > 1) make less alloc/dealloc operations (you may try to make larger min_heap_size for new processes to reduce GC frequency) > 2) change `free_blocks_tree` sorting order so it will be address-ordered and blocks from tail may be returned to OS faster > 3) tune mbcs_block_size and mbcs_carrier_size to fit better to your most common data's size to reduce fragmentation by helping free blocks reusage. > > Which one fits your needs I don't know. You may consult this doc http://erlang.org/doc/man/erts_alloc.html for more details. > >> Hi Lucas, >> I'm using 17.4 on ubuntu LTS 14.04. The settings I use are: >> +K true >> +P 2000000 >> +Q 1000000 >> Please find here below an additional set of data with the changes I've >> stated here above (binary:copy/0 and the fullsweep_after 0 in the router). >> Of the recon library I'm using the following calls: >> recon_alloc:memory(allocated, current). >> recon_alloc:memory(used, current). >> recon_alloc:memory(usage, current). >> >> All of this data is taken at random intervals of time. >> - BEAM process RES memory:* 2.732 GB* >> - Erlang memory: >> [{total,1.7860298827290535}, >> {processes,1.4158401936292648}, >> {processes_used,1.4157484397292137}, >> {system,0.37018968909978867}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.846092149615288e-4}, >> {binary,0.20867645740509033}, >> {code,0.009268132038414478}, >> {ets,0.004821933805942535}] >> - recon_alloc: >> allocated: 3015796080 (2.808 GB) >> used: 2161850416 >> usage: 0.7187714029859935 >> >> - BEAM process RES memory:* 2.813 GB* >> - Erlang memory: >> [{total,2.026990756392479}, >> {processes,1.6270370781421661}, >> {processes_used,1.6269719526171684}, >> {system,0.3999536782503128}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.23845425993204117}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3098895728 (2.886 GB) >> used: 2176172480 >> usage: 0.7023218278482198 >> >> - BEAM process RES memory:* 3.029 GB* >> - Erlang memory: >> [{total,2.351852521300316}, >> {processes,1.9361207410693169}, >> {processes_used,1.9360847249627113}, >> {system,0.415731780230999}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2539009377360344}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3337524592 (3.108 GB) >> used: 2525365352 >> usage: 0.7548030747173055 >> >> - BEAM process RES memory:* 3.099 GB* >> - Erlang memory: >> [{total,2.0704088881611824}, >> {processes,1.6625376418232918}, >> {processes_used,1.6624245047569275}, >> {system,0.4078712463378906}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.24636883288621902}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3400623472 (3.167 GB) >> used: 2222575336 >> usage: 0.6552131374790817 >> >> - BEAM process RES memory:* 3.132 GB* >> - Erlang memory: >> [{total,2.367126949131489}, >> {processes,1.9388784170150757}, >> {processes_used,1.938723236322403}, >> {system,0.4282485321164131}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2667432576417923}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3435644272 (3.200 GB) >> used: 2541469864 >> usage: 0.7397146313173368 >> >> - BEAM process RES memory:* 3.307 GB* >> - Erlang memory: >> [{total,2.379016488790512}, >> {processes,1.9780860394239426}, >> {processes_used,1.9779272973537445}, >> {system,0.4009304493665695}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.23943009227514267}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3619329392 (3.371 GB) >> used: 2554804000 >> usage: 0.704330124232303 >> >> - BEAM process RES memory:* 3.351 GB* >> - Erlang memory: >> [{total,2.607522390782833}, >> {processes,2.168950654566288}, >> {processes_used,2.1688189953565598}, >> {system,0.4385717362165451}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.8593634963035583e-4}, >> {binary,0.2771267145872116}, >> {code,0.009311830624938011}, >> {ets,0.004802137613296509}] >> - recon_alloc: >> allocated: 3669321072 (3.417 GB) >> used: 2799919616 >> usage: 0.7629933213977411 >> >> - BEAM process RES memory:* 3.469 GB* >> - Erlang memory: >> [{total,2.2596140429377556}, >> {processes,1.8098593652248383}, >> {processes_used,1.8098137602210045}, >> {system,0.44975467771291733}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.2881493419408798}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 3789014384 (3.528 GB) >> used: 2425613912 >> usage: 0.6401929098614897 >> >> - BEAM process RES memory:* 3.660 GB* >> - Erlang memory: >> [{total,2.692381367087364}, >> {processes,2.2282255738973618}, >> {processes_used,2.228082850575447}, >> {system,0.46415579319000244}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.30247989296913147}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 3993933168 (3.719 GB) >> used: 2890714704 >> usage: 0.7233507625681828 >> >> - BEAM process RES memory:* 3.667 GB* >> - Erlang memory: >> [{total,2.4165985733270645}, >> {processes,1.9264011159539223}, >> {processes_used,1.9263720959424973}, >> {system,0.49019745737314224}, >> {atom,4.000673070549965e-4}, >> {atom_used,3.86836938560009e-4}, >> {binary,0.3284950777888298}, >> {code,0.009375222958624363}, >> {ets,0.00480380654335022}] >> - recon_alloc: >> allocated: 4001830256 (3.727 GB) >> used: 2594872464 >> usage: 0.6483950197811689 >> >> It looks like the memory allocated has some mismatch with the memory >> reported by the OS, but maybe this is just a timing issue (since top >> provides an average during a period of time)? >> Anyway, the BEAM process keeps on increasing. Also, it looks like memory >> usage bounces anywhere from 64-72% and I don't know if that's a good figure >> or not. >> This is what I get from a check on memory fragmentation: >> 1> recon_alloc:fragmentation(current). >> [{{eheap_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6252789717100151}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,391670536}, >> {mbcs_carriers_size,626393264}]}, >> {{eheap_alloc,2}, >> [{sbcs_usage,0.6168021896258503}, >> {mbcs_usage,0.6893887270883688}, >> {sbcs_block_size,371384}, >> {sbcs_carriers_size,602112}, >> {mbcs_block_size,370926112}, >> {mbcs_carriers_size,538050736}]}, >> {{eheap_alloc,3}, >> [{sbcs_usage,0.9991333400321544}, >> {mbcs_usage,0.7006580932915004}, >> {sbcs_block_size,5091008}, >> {sbcs_carriers_size,5095424}, >> {mbcs_block_size,324091688}, >> {mbcs_carriers_size,462553264}]}, >> {{eheap_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6924985776876923}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,305976264}, >> {mbcs_carriers_size,441843888}]}, >> {{eheap_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6397496430493375}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,207536944}, >> {mbcs_carriers_size,324403376}]}, >> {{eheap_alloc,8}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6660125315617468}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,166472816}, >> {mbcs_carriers_size,249954480}]}, >> {{eheap_alloc,6}, >> [{sbcs_usage,0.9980070153061225}, >> {mbcs_usage,0.6770963446791575}, >> {sbcs_block_size,600912}, >> {sbcs_carriers_size,602112}, >> {mbcs_block_size,169065768}, >> {mbcs_carriers_size,249692336}]}, >> {{eheap_alloc,7}, >> [{sbcs_usage,0.997382155987395}, >> {mbcs_usage,0.6925225022824623}, >> {sbcs_block_size,972296}, >> {sbcs_carriers_size,974848}, >> {mbcs_block_size,167834424}, >> {mbcs_carriers_size,242352304}]}, >> {{ll_alloc,0}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7387279228243809}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,112706224}, >> {mbcs_carriers_size,152567976}]}, >> {{binary_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7614033191804973}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,58507096}, >> {mbcs_carriers_size,76841136}]}, >> {{binary_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7179048085736076}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,46131288}, >> {mbcs_carriers_size,64258224}]}, >> {{binary_alloc,7}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6128136272843548}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,27651200}, >> {mbcs_carriers_size,45121712}]}, >> {{binary_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6492102167567332}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,32186648}, >> {mbcs_carriers_size,49578160}]}, >> {{binary_alloc,2}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7715637758298944}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,56051664}, >> {mbcs_carriers_size,72646832}]}, >> {{binary_alloc,3}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7514879308127178}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,49077272}, >> {mbcs_carriers_size,65306800}]}, >> {{binary_alloc,6}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.6757064168943689}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,25706456}, >> {mbcs_carriers_size,38043824}]}, >> {{binary_alloc,8}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7016146506167494}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,25956408}, >> {mbcs_carriers_size,36995248}]}, >> {{fix_alloc,4}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7409146075507547}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,22748888}, >> {mbcs_carriers_size,30703792}]}, >> {{fix_alloc,2}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8581522751225302}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,33547232}, >> {mbcs_carriers_size,39092400}]}, >> {{fix_alloc,3}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8630094940716118}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,26497664}, >> {mbcs_carriers_size,...}]}, >> {{driver_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7615924083651237}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,0}, >> {mbcs_block_size,...}, >> {...}]}, >> {{fix_alloc,1}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.8947156992151927}, >> {sbcs_block_size,0}, >> {sbcs_carriers_size,...}, >> {...}|...]}, >> {{fix_alloc,5}, >> [{sbcs_usage,1.0}, >> {mbcs_usage,0.7166474590927997}, >> {sbcs_block_size,...}, >> {...}|...]}, >> {{ll_alloc,7},[{sbcs_usage,1.0},{mbcs_usage,...},{...}|...]}, >> {{driver_alloc,2},[{sbcs_usage,...},{...}|...]}, >> {{driver_alloc,4},[{...}|...]}, >> {{ll_alloc,...},[...]}, >> {{...},...}, >> {...}|...] >> >> Unless I'm mistaken reading this data, everything looks fine there, with >> normal usages. >> What can I do else to address this issue? >> Thank you for your help. >> r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jan 23 20:55:44 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 23 Jan 2015 20:55:44 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: Message-ID: <7DB36157-C26C-4BF1-9A9A-C431A32E9148@feuerlabs.com> > On 22 Jan 2015, at 17:33, Roberto Ostinelli wrote: > > {<0.428.0>,[],142896}, > {<0.429.0>,[],142896}, > {<0.430.0>,[],142896}] > > See the last processes there with all identical memory? These are the processes handling the connections, and they stay stable with the same identical number throughout all test. Are you using min_heap_size on these? One thing we stumbled across fairly early on in the exometer development was that memory increased until the node ran out of memory. What was needed was to lower the fullsweep_after threshold for processes that had an increased min_heap_size. The default fullsweep_after is very high, and what can happen is that the old_heap grows a lot between the fullsweeps. In our case, {fullsweep_after, 10} was sufficient to solve the problem. You did say that you?d try applying fullsweep_after selectively, but wasn?t sure if you tried this. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Fri Jan 23 20:58:02 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Fri, 23 Jan 2015 11:58:02 -0800 Subject: [erlang-questions] unregistering a process during takeover Message-ID: <-4958514093543002976@unknownmsgid> Dear ulf , I used retool.config to create the release. i run the executables created to run the application and added -c option for da various nodes . I did start the executables on node b cuz i had sync_nodes_mandatory option and a restart/run of app on node a requires node b and c to be running . I also run application:info() on b and saw the application running on b It was however not running on node c due to the takeover . Had to create a new shell on node c . Anyway i will try again and note u suggestions about rebar ,relx ,setup . Thanks a lot !!! ------------------------------ From: Ulf Wiger Sent: ?23/?01/?2015 17:49 To: Nuku Ameyibor Cc: erlang questions Subject: Re: [erlang-questions] unregistering a process during takeover Not sure based on what you?ve described. A possibility is that you?ve forgotten to start the application on b. The next step, though, would be to also use a boot script. You could use rebar, relx, OTP systools or my personal pet project, setup [1]. Using a boot script, your applications are started automatically, or in the case of distributed apps, at least prepared to start, so that failover/takeover can work. BR, Ulf W [1] http://github.com/uwiger/setup https://github.com/uwiger/setup/wiki/Example%3A-generating-boot-scripts-from-a-running-system On 23 Jan 2015, at 12:09, Nuku Ameyibor wrote: hi ulf , thanks. the takeover is working okay now !!! . config a is this [{kernel, [{distributed, [{m8ball, 5000, [a@REDACTED, {b@REDACTED, c@REDACTED}]}]}, {sync_nodes_mandatory, [b@REDACTED, c@REDACTED]}, {sync_nodes_timeout, 30000} ] } ]. i start each node with bin\werl.exe -sname c -config C:\wamp\www\erlang\proj\m8ball\apps\m8ball_1_0_0\config\c.config there are 3 nodes .their conf files are similar with sync_nodes_mandatory being slightly different . a small question . the first round of failover and takeover seems to work fine . but after the first round (start node a,b,c . kill a b does failover .kill b c does failover . restart a,b a does takeover from c killing c application on c .i restart shell on c ) * after the first round when i try to do a failover onto b by killing a it doesnt work .* was wondering how come since the first round of failover take over worked fine . On Fri, Jan 23, 2015 at 6:35 AM, Ulf Wiger wrote: > > On 23 Jan 2015, at 03:44, Nuku Ameyibor wrote: > > =INFO REPORT==== 23-Jan-2015::02:20:57 === > global: Name conflict terminating {m8ball_server,<2910.62.0>} > > > This suggests that you?re connecting the nodes *after* starting your > applications, so that the application processes have time to globally > register themselves first. Global detects the name conflict when trying to > merge the registries of the two nodes. > > You can for example use the ?distributed? option, described here: > http://www.erlang.org/doc/apps/kernel/application.html#load-1 > > and here: > > http://www.erlang.org/doc/man/kernel_app.html > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Fri Jan 23 21:40:28 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 21:40:28 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: <7DB36157-C26C-4BF1-9A9A-C431A32E9148@feuerlabs.com> References: <7DB36157-C26C-4BF1-9A9A-C431A32E9148@feuerlabs.com> Message-ID: <88B2D709-359C-40AE-9CC0-A4EA5955B4B0@widetag.com> Hey Ulf! Thank you for your input. What do you mean with "are you using min_heap_size"? > On 23/gen/2015, at 20:55, Ulf Wiger wrote: > > >> On 22 Jan 2015, at 17:33, Roberto Ostinelli wrote: >> >> {<0.428.0>,[],142896}, >> {<0.429.0>,[],142896}, >> {<0.430.0>,[],142896}] >> >> See the last processes there with all identical memory? These are the processes handling the connections, and they stay stable with the same identical number throughout all test. > > Are you using min_heap_size on these? > > One thing we stumbled across fairly early on in the exometer development was that memory increased until the node ran out of memory. What was needed was to lower the fullsweep_after threshold for processes that had an increased min_heap_size. The default fullsweep_after is very high, and what can happen is that the old_heap grows a lot between the fullsweeps. > > In our case, {fullsweep_after, 10} was sufficient to solve the problem. > > You did say that you?d try applying fullsweep_after selectively, but wasn?t sure if you tried this. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Fri Jan 23 21:53:55 2015 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 23 Jan 2015 21:53:55 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: <88B2D709-359C-40AE-9CC0-A4EA5955B4B0@widetag.com> References: <7DB36157-C26C-4BF1-9A9A-C431A32E9148@feuerlabs.com> <88B2D709-359C-40AE-9CC0-A4EA5955B4B0@widetag.com> Message-ID: On Fri, Jan 23, 2015 at 9:40 PM, Roberto Ostinelli wrote: > Hey Ulf! > Thank you for your input. > > What do you mean with "are you using min_heap_size"? See erlang:spawn_opt/3 or erlang:process_flag/2. From roberto.ostinelli@REDACTED Fri Jan 23 22:14:07 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 23 Jan 2015 22:14:07 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <7DB36157-C26C-4BF1-9A9A-C431A32E9148@feuerlabs.com> <88B2D709-359C-40AE-9CC0-A4EA5955B4B0@widetag.com> Message-ID: <402EA6C5-939F-4141-AD5D-516F1F5C82AA@widetag.com> oh ok so the question was if I'm setting it to a custom value? No I'm not. Only fullsweep_after. > On 23/gen/2015, at 21:53, Tuncer Ayaz wrote: > >> On Fri, Jan 23, 2015 at 9:40 PM, Roberto Ostinelli wrote: >> Hey Ulf! >> Thank you for your input. >> >> What do you mean with "are you using min_heap_size"? > > See erlang:spawn_opt/3 or erlang:process_flag/2. From ulf@REDACTED Fri Jan 23 22:30:06 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 23 Jan 2015 22:30:06 +0100 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: <-4958514093543002976@unknownmsgid> References: <-4958514093543002976@unknownmsgid> Message-ID: <37334E5C-AF51-411A-88C9-E0F70B44ABF1@feuerlabs.com> My comment about the boot script was just because I didn?t see a -boot option in your command line. When did you run application:info() on b and saw the application running there? Was it after killing a? To be clear, your config says the application should failover to *either* b or c. BR, Ulf W > On 23 Jan 2015, at 20:58, Nuku Ameyibor wrote: > > > > Dear ulf , > > I used retool.config to create the release. i run the executables created to run the application and added -c option for da various nodes . > I did start the executables on node b cuz i had sync_nodes_mandatory option and a restart/run of app on node a requires node b and c to be running . > I also run application:info() on b and saw the application running on b > It was however not running on node c due to the takeover . > Had to create a new shell on node c . > > Anyway i will try again and note u suggestions about rebar ,relx ,setup . > Thanks a lot !!! > From: Ulf Wiger > Sent: ?23/?01/?2015 17:49 > To: Nuku Ameyibor > Cc: erlang questions > Subject: Re: [erlang-questions] unregistering a process during takeover > > Not sure based on what you?ve described. A possibility is that you?ve forgotten to start the application on b. > > The next step, though, would be to also use a boot script. You could use rebar, relx, OTP systools or my personal pet project, setup [1]. > > Using a boot script, your applications are started automatically, or in the case of distributed apps, at least prepared to start, so that failover/takeover can work. > > BR, > Ulf W > > [1] http://github.com/uwiger/setup > https://github.com/uwiger/setup/wiki/Example%3A-generating-boot-scripts-from-a-running-system > > >> On 23 Jan 2015, at 12:09, Nuku Ameyibor > wrote: >> >> hi ulf , >> thanks. >> the takeover is working okay now !!! . >> config a is this >> >> [{kernel, >> [{distributed, [{m8ball, >> 5000, >> [a@REDACTED, {b@REDACTED, c@REDACTED}]}]}, >> {sync_nodes_mandatory, [b@REDACTED, c@REDACTED]}, >> {sync_nodes_timeout, 30000} >> ] >> } >> ]. >> >> i start each node with >> bin\werl.exe -sname c -config C:\wamp\www\erlang\proj\m8ball\apps\m8ball_1_0_0\config\c.config >> >> there are 3 nodes .their conf files are similar with sync_nodes_mandatory being slightly different . >> >> a small question . >> the first round of failover and takeover seems to work fine . >> but after the first round (start node a,b,c . kill a b does failover .kill b c does failover . restart a,b a does takeover from c killing c application on c .i restart shell on c ) >> >> after the first round when i try to do a failover onto b by killing a it doesnt work . >> was wondering how come since the first round of failover take over worked fine . >> >> >> >> >> On Fri, Jan 23, 2015 at 6:35 AM, Ulf Wiger > wrote: >> >>> On 23 Jan 2015, at 03:44, Nuku Ameyibor > wrote: >>> >>> =INFO REPORT==== 23-Jan-2015::02:20:57 === >>> global: Name conflict terminating {m8ball_server,<2910.62.0>} >> >> This suggests that you?re connecting the nodes *after* starting your applications, so that the application processes have time to globally register themselves first. Global detects the name conflict when trying to merge the registries of the two nodes. >> >> You can for example use the ?distributed? option, described here: >> http://www.erlang.org/doc/apps/kernel/application.html#load-1 >> >> and here: >> >> http://www.erlang.org/doc/man/kernel_app.html >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sat Jan 24 10:37:02 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 24 Jan 2015 10:37:02 +0100 Subject: [erlang-questions] good beginners book/tutorial Message-ID: <54C367BE.4030408@home.nl> Hello, I want to learn erlang and later on maybe phoenix or another web platform to make a e-commerce site But is there a good beginners book so I can learn erlang well with a lot of exercises. This because I learn the best when I have my hands wet. Roelof From czinkos@REDACTED Sat Jan 24 11:03:07 2015 From: czinkos@REDACTED (Zsolt Czinkos) Date: Sat, 24 Jan 2015 11:03:07 +0100 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: <54C367BE.4030408@home.nl> References: <54C367BE.4030408@home.nl> Message-ID: Hi, These books helped me a lot (both contain exercises): http://shop.oreilly.com/product/9780596518189.do https://pragprog.com/book/jaerlang2/programming-erlang zsolt On Sat, Jan 24, 2015 at 10:37 AM, Roelof Wobben wrote: > Hello, > > I want to learn erlang and later on maybe phoenix or another web platform > to make a e-commerce site > > But is there a good beginners book so I can learn erlang well with a lot > of exercises. > This because I learn the best when I have my hands wet. > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sat Jan 24 11:07:09 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 24 Jan 2015 11:07:09 +0100 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: References: <54C367BE.4030408@home.nl> Message-ID: Fred H?bert's book is up to date. It covers more topics, has many examples: http://learnyousomeerlang.com/ From imantc@REDACTED Sat Jan 24 11:19:28 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 24 Jan 2015 11:19:28 +0100 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: References: <54C367BE.4030408@home.nl> Message-ID: "Erlang and OTP in Action" has practical examples: http://www.manning.com/logan/ From imantc@REDACTED Sat Jan 24 11:23:13 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 24 Jan 2015 11:23:13 +0100 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: References: <54C367BE.4030408@home.nl> Message-ID: a good intro to Yaws web server: http://shop.oreilly.com/product/0636920021452.do From nayibor@REDACTED Sat Jan 24 13:56:43 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Sat, 24 Jan 2015 12:56:43 +0000 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: References: <-4958514093543002976@unknownmsgid> <37334E5C-AF51-411A-88C9-E0F70B44ABF1@feuerlabs.com> Message-ID: > > When did you run application:info() on b and saw the application running > there? Was it after killing a? i run that function on node b after restarting node node a and making it takeover from the failover node b. and i make a mistake it wasn't running on node b but on node a . To be clear, your config says the application should failover to *either* b > or c. yes true and that is what happens . but how do errors during restarts affect the distribution mechanism . for instance how do global name clashes affect the distribution mechanism . i got this error during takeover but the takeover still happened thanks !! On Sat, Jan 24, 2015 at 12:53 PM, Nuku Ameyibor wrote: > When did you run application:info() on b and saw the application running >> there? Was it after killing a? > > i run that function on node b after restarting node node a and making > it takeover from the failover node b. > and i make a mistake it wasn't running on node b but on node a . > > > To be clear, your config says the application should failover to *either* >> b or c. > > yes true and that is what happens . > > but how do errors during restarts affect the distribution mechanism . > for instance how do global name clashes affect the distribution > mechanism . > i got this error during takeover but the takeover still happened > thanks !! > > > > On Fri, Jan 23, 2015 at 9:30 PM, Ulf Wiger wrote: > >> My comment about the boot script was just because I didn?t see a -boot >> option in your command line. >> >> When did you run application:info() on b and saw the application running >> there? Was it after killing a? >> >> To be clear, your config says the application should failover to *either* >> b or c. >> >> BR, >> Ulf W >> >> >> On 23 Jan 2015, at 20:58, Nuku Ameyibor wrote: >> >> >> >> Dear ulf , >> >> I used retool.config to create the release. i run the executables >> created to run the application and added -c option for da various nodes . >> I did start the executables on node b cuz i had sync_nodes_mandatory >> option and a restart/run of app on node a requires node b and c to be >> running . >> I also run application:info() on b and saw the application running on b >> It was however not running on node c due to the takeover . >> Had to create a new shell on node c . >> >> Anyway i will try again and note u suggestions about rebar ,relx ,setup >> . >> Thanks a lot !!! >> ------------------------------ >> From: Ulf Wiger >> Sent: ?23/?01/?2015 17:49 >> To: Nuku Ameyibor >> Cc: erlang questions >> Subject: Re: [erlang-questions] unregistering a process during takeover >> >> Not sure based on what you?ve described. A possibility is that you?ve >> forgotten to start the application on b. >> >> The next step, though, would be to also use a boot script. You could use >> rebar, relx, OTP systools or my personal pet project, setup [1]. >> >> Using a boot script, your applications are started automatically, or in >> the case of distributed apps, at least prepared to start, so that >> failover/takeover can work. >> >> BR, >> Ulf W >> >> [1] http://github.com/uwiger/setup >> >> https://github.com/uwiger/setup/wiki/Example%3A-generating-boot-scripts-from-a-running-system >> >> >> >> On 23 Jan 2015, at 12:09, Nuku Ameyibor wrote: >> >> hi ulf , >> thanks. >> the takeover is working okay now !!! . >> config a is this >> >> [{kernel, >> [{distributed, [{m8ball, >> 5000, >> [a@REDACTED, {b@REDACTED, c@REDACTED}]}]}, >> {sync_nodes_mandatory, [b@REDACTED, c@REDACTED]}, >> {sync_nodes_timeout, 30000} >> ] >> } >> ]. >> >> i start each node with >> bin\werl.exe -sname c -config >> C:\wamp\www\erlang\proj\m8ball\apps\m8ball_1_0_0\config\c.config >> >> there are 3 nodes .their conf files are similar with sync_nodes_mandatory >> being slightly different . >> >> a small question . >> the first round of failover and takeover seems to work fine . >> but after the first round (start node a,b,c . kill a b does failover >> .kill b c does failover . restart a,b a does takeover from c killing c >> application on c .i restart shell on c ) >> >> * after the first round when i try to do a failover onto b by killing >> a it doesnt work .* >> was wondering how come since the first round of failover take over worked >> fine . >> >> >> >> >> On Fri, Jan 23, 2015 at 6:35 AM, Ulf Wiger wrote: >> >>> >>> On 23 Jan 2015, at 03:44, Nuku Ameyibor wrote: >>> >>> =INFO REPORT==== 23-Jan-2015::02:20:57 === >>> global: Name conflict terminating {m8ball_server,<2910.62.0>} >>> >>> >>> This suggests that you?re connecting the nodes *after* starting your >>> applications, so that the application processes have time to globally >>> register themselves first. Global detects the name conflict when trying to >>> merge the registries of the two nodes. >>> >>> You can for example use the ?distributed? option, described here: >>> http://www.erlang.org/doc/apps/kernel/application.html#load-1 >>> >>> and here: >>> >>> http://www.erlang.org/doc/man/kernel_app.html >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sat Jan 24 15:00:10 2015 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 24 Jan 2015 15:00:10 +0100 Subject: [erlang-questions] unregistering a process during takeover In-Reply-To: References: <-4958514093543002976@unknownmsgid> <37334E5C-AF51-411A-88C9-E0F70B44ABF1@feuerlabs.com> Message-ID: > On 24 Jan 2015, at 13:56, Nuku Ameyibor wrote: > > but how do errors during restarts affect the distribution mechanism . > for instance how do global name clashes affect the distribution mechanism . When you start applications from the shell using application:start(App), the default restart type is ?temprorary?, which means that if the application start fails, only the application is terminated. If the application is started with application:start(App, permanent), the whole node is terminated if the application crashes. The application start, in its turn, fails if the processes started synchonously from the start function fail during startup. The global name registration usually happens during application startup. Applications started from a boot script are automatically started as ?permanent?. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 24 15:11:45 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 24 Jan 2015 06:11:45 -0800 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: References: <54C367BE.4030408@home.nl> Message-ID: I am new to Erlang and find Joe's book as a wonderful resource to learn Thanks On Sat, Jan 24, 2015 at 2:23 AM, Imants Cekusins wrote: > a good intro to Yaws web server: > > http://shop.oreilly.com/product/0636920021452.do > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 24 15:44:32 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 24 Jan 2015 06:44:32 -0800 Subject: [erlang-questions] undefined function lib_chan:start/0 Message-ID: I am learning Erlang from Joe's book and one of the thing he teaches is about socket programming. His code example runs like 1>* kvs:start().* true 2>* lib_chan:start_server().* Starting a port server on 1234... true I try to do the same thing with code he provided -module(mod_name_server). -author("harith"). %% API -export([start_me_up/3]). start_me_up(MM, _ArgsC, _ArgsS) -> loop(MM). loop(MM) -> receive {chan, MM, {store, K, V}} -> kvs:store(K, V), loop(MM); {chan, MM, {lookup, K}} -> MM ! {send, kvs:lookup(K)}, loop(MM); {chan_closed, MM} -> true end. But when I run, I see following 1> c(kvs). {ok,kvs} 2> c(mod_name_server). {ok,mod_name_server} 3> kvs:start(). true 4> lib_chan:start(). ** exception error: undefined function lib_chan:start/0 5> lib_chan_mm:start(). ** exception error: undefined function lib_chan_mm:start/0 6> I even tried for lib_chan_mm, but no luck What is going wrong here? Thank you + Harit Himanshu -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Sat Jan 24 15:46:19 2015 From: mfidelman@REDACTED (Miles Fidelman) Date: Sat, 24 Jan 2015 09:46:19 -0500 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: References: <54C367BE.4030408@home.nl> Message-ID: <54C3B03B.2080501@meetinghouse.net> Imants Cekusins wrote: > "Erlang and OTP in Action" has practical examples: > > http://www.manning.com/logan/ > _______________________________________________ > I'll echo that one. It's the best one I've found for digging into OTP and real application building. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From jesper.louis.andersen@REDACTED Sat Jan 24 17:08:52 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 24 Jan 2015 17:08:52 +0100 Subject: [erlang-questions] undefined function lib_chan:start/0 In-Reply-To: References: Message-ID: On Sat, Jan 24, 2015 at 3:44 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > What is going wrong here? Good tools for debugging these kinds of things: The 'undefined function' errors occurs even in the case where there is no such module loaded. So first, you should make sure there is such a module, by using, for instance, the m/0 command of the shell. An alternative is to ask the code loader for the location of the module, `code:which(lib_chan)`. If the module is loaded, you can ask for its info, by executing m(lib_chan). which can be used to verify there is a function named `start` of arity 0. In this case, my hunch would be that the module is not loaded, probably due to a loader path which is not set. And since it is nowhere to be found, the system will not allow you to call a function in a module which is not even accessible to the system. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Sat Jan 24 17:32:53 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 24 Jan 2015 08:32:53 -0800 Subject: [erlang-questions] undefined function lib_chan:start/0 In-Reply-To: References: Message-ID: Thanks Jesper, yes you are right, the module is not there On Sat, Jan 24, 2015 at 8:08 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Sat, Jan 24, 2015 at 3:44 PM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> What is going wrong here? > > > Good tools for debugging these kinds of things: > > The 'undefined function' errors occurs even in the case where there is no > such module loaded. So first, you should make sure there is such a module, > by using, for instance, the m/0 command of the shell. An alternative is to > ask the code loader for the location of the module, `code:which(lib_chan)`. > > If the module is loaded, you can ask for its info, by executing > > m(lib_chan). > > which can be used to verify there is a function named `start` of arity 0. > > In this case, my hunch would be that the module is not loaded, probably > due to a loader path which is not set. And since it is nowhere to be found, > the system will not allow you to call a function in a module which is not > even accessible to the system. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Sat Jan 24 17:58:49 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Sat, 24 Jan 2015 08:58:49 -0800 Subject: [erlang-questions] unregistering a process during takeover Message-ID: <5722111692256472138@unknownmsgid> Thanks !! ------------------------------ From: Ulf Wiger Sent: ?24/?01/?2015 14:00 To: Nuku Ameyibor Cc: erlang questions Subject: Re: [erlang-questions] unregistering a process during takeover On 24 Jan 2015, at 13:56, Nuku Ameyibor wrote: but how do errors during restarts affect the distribution mechanism . for instance how do global name clashes affect the distribution mechanism . When you start applications from the shell using application:start(App), the default restart type is ?temprorary?, which means that if the application start fails, only the application is terminated. If the application is started with application:start(App, permanent), the whole node is terminated if the application crashes. The application start, in its turn, fails if the processes started synchonously from the start function fail during startup. The global name registration usually happens during application startup. Applications started from a boot script are automatically started as ?permanent?. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sat Jan 24 18:48:38 2015 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 24 Jan 2015 18:48:38 +0100 Subject: [erlang-questions] undefined function lib_chan:start/0 In-Reply-To: References: Message-ID: lib_chan is the code tarball that accompanies the book Go to https://pragprog.com/titles/jaerlang2/source_code and download the source code from the book lib_chan.erl is in the sub directory of the unpacked code called "socket_dist". The examples in the book should work if you change directory to socket_dist and run the makefle in this directory. Hope this helps /Joe On Sat, Jan 24, 2015 at 3:44 PM, Harit Himanshu < harit.subscriptions@REDACTED> wrote: > I am learning Erlang from Joe's book and one of the thing he teaches is > about socket programming. > His code example runs like > > 1>* kvs:start().* > > true > > 2>* lib_chan:start_server().* > > Starting a port server on 1234... > > true > > I try to do the same thing with code he provided > -module(mod_name_server). > -author("harith"). > > %% API > -export([start_me_up/3]). > > start_me_up(MM, _ArgsC, _ArgsS) -> > loop(MM). > > loop(MM) -> > receive > {chan, MM, {store, K, V}} -> > kvs:store(K, V), > loop(MM); > {chan, MM, {lookup, K}} -> > MM ! {send, kvs:lookup(K)}, > loop(MM); > {chan_closed, MM} -> > true > end. > > > But when I run, I see following > > 1> c(kvs). > > {ok,kvs} > > 2> c(mod_name_server). > > {ok,mod_name_server} > > 3> kvs:start(). > > true > > 4> lib_chan:start(). > > ** exception error: undefined function lib_chan:start/0 > > 5> lib_chan_mm:start(). > > ** exception error: undefined function lib_chan_mm:start/0 > > 6> > > > I even tried for lib_chan_mm, but no luck > > What is going wrong here? > > Thank you > + Harit Himanshu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shawn@REDACTED Sat Jan 24 19:53:22 2015 From: shawn@REDACTED (Shawn Debnath) Date: Sat, 24 Jan 2015 18:53:22 +0000 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: <54C3B03B.2080501@meetinghouse.net> References: <54C367BE.4030408@home.nl> <54C3B03B.2080501@meetinghouse.net> Message-ID: <36112140-8906-4817-A178-386C3310D266@debnath.net> Yep, this is a great book indeed, however, if you are a beginner like me, I found the O?Reilly book to be the best to get your hands dirty with especially if you are new to functional programming. I would recommend reading the first 4-5 chapters dubbed ?Sequential Programming? from either book and trying out every example and exercise, just to get the feel for the language and syntax. After that, you will probably be able to do some research on a web platform and start building a site right away. The basics of writing a web application using any existing framework doesn?t require extensive knowledge of OTP, but that comes right into play as soon as you try to architect a whole system that can scale with components with more than just a web server, i.e., a separate app core to handle common business logic, DAL, workers, etc. On 1/24/15, 2:46 PM, "Miles Fidelman" wrote: >Imants Cekusins wrote: >> "Erlang and OTP in Action" has practical examples: >> >> http://www.manning.com/logan/ >> _______________________________________________ >> > >I'll echo that one. It's the best one I've found for digging into OTP >and real application building. > >Miles Fidelman > > >-- >In theory, there is no difference between theory and practice. >In practice, there is. .... Yogi Berra > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From shawn@REDACTED Sat Jan 24 19:55:36 2015 From: shawn@REDACTED (Shawn Debnath) Date: Sat, 24 Jan 2015 18:55:36 +0000 Subject: [erlang-questions] good beginners book/tutorial In-Reply-To: <36112140-8906-4817-A178-386C3310D266@debnath.net> References: <54C367BE.4030408@home.nl> <54C3B03B.2080501@meetinghouse.net> <36112140-8906-4817-A178-386C3310D266@debnath.net> Message-ID: <1C1E6406-6AED-4CA6-B2DE-C7B6CA7E83B9@debnath.net> By either book, I meant Erlang Programming - http://www.amazon.com/Erlang-Programming-Francesco-Cesarini/dp/0596518188 Programming Erlang - http://www.amazon.com/Programming-Erlang-Concurrent-Pragmatic-Programmers/d p/193778553X On 1/24/15, 6:53 PM, "Shawn Debnath" wrote: >Yep, this is a great book indeed, however, if you are a beginner like me, >I found the O?Reilly book to be the best to get your hands dirty with >especially if you are new to functional programming. I would recommend >reading the first 4-5 chapters dubbed ?Sequential Programming? from either >book and trying out every example and exercise, just to get the feel for >the language and syntax. After that, you will probably be able to do some >research on a web platform and start building a site right away. The >basics of writing a web application using any existing framework doesn?t >require extensive knowledge of OTP, but that comes right into play as >soon as you try to architect a whole system that can scale with components >with more than just a web server, i.e., a separate app core to handle >common business logic, DAL, workers, etc. > > > > >On 1/24/15, 2:46 PM, "Miles Fidelman" wrote: > >>Imants Cekusins wrote: >>> "Erlang and OTP in Action" has practical examples: >>> >>> http://www.manning.com/logan/ >>> _______________________________________________ >>> >> >>I'll echo that one. It's the best one I've found for digging into OTP >>and real application building. >> >>Miles Fidelman >> >> >>-- >>In theory, there is no difference between theory and practice. >>In practice, there is. .... Yogi Berra >> >>_______________________________________________ >>erlang-questions mailing list >>erlang-questions@REDACTED >>http://erlang.org/mailman/listinfo/erlang-questions >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From harit.subscriptions@REDACTED Sat Jan 24 20:45:41 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sat, 24 Jan 2015 11:45:41 -0800 Subject: [erlang-questions] undefined function lib_chan:start/0 In-Reply-To: References: Message-ID: Thanks Joe I will do that and let you know if I face any issues + Harit Himanshu On Sat, Jan 24, 2015 at 9:48 AM, Joe Armstrong wrote: > lib_chan is the code tarball that accompanies the book > > Go to https://pragprog.com/titles/jaerlang2/source_code > > and download the source code from the book > > lib_chan.erl is in the sub directory of the unpacked code called > "socket_dist". > > The examples in the book should work if you change directory to > socket_dist and run the makefle in this directory. > > Hope this helps > > /Joe > > > > > > On Sat, Jan 24, 2015 at 3:44 PM, Harit Himanshu < > harit.subscriptions@REDACTED> wrote: > >> I am learning Erlang from Joe's book and one of the thing he teaches is >> about socket programming. >> His code example runs like >> >> 1>* kvs:start().* >> >> true >> >> 2>* lib_chan:start_server().* >> >> Starting a port server on 1234... >> >> true >> >> I try to do the same thing with code he provided >> -module(mod_name_server). >> -author("harith"). >> >> %% API >> -export([start_me_up/3]). >> >> start_me_up(MM, _ArgsC, _ArgsS) -> >> loop(MM). >> >> loop(MM) -> >> receive >> {chan, MM, {store, K, V}} -> >> kvs:store(K, V), >> loop(MM); >> {chan, MM, {lookup, K}} -> >> MM ! {send, kvs:lookup(K)}, >> loop(MM); >> {chan_closed, MM} -> >> true >> end. >> >> >> But when I run, I see following >> >> 1> c(kvs). >> >> {ok,kvs} >> >> 2> c(mod_name_server). >> >> {ok,mod_name_server} >> >> 3> kvs:start(). >> >> true >> >> 4> lib_chan:start(). >> >> ** exception error: undefined function lib_chan:start/0 >> >> 5> lib_chan_mm:start(). >> >> ** exception error: undefined function lib_chan_mm:start/0 >> >> 6> >> >> >> I even tried for lib_chan_mm, but no luck >> >> What is going wrong here? >> >> Thank you >> + Harit Himanshu >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sun Jan 25 11:28:35 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 11:28:35 +0100 Subject: [erlang-questions] wierd list comprehension outcomes Message-ID: <54C4C553.1070105@home.nl> Hello, I follow this book : Erlang programming and I did read chapter 2. Now I have to do this exercise : C. Recursive list definitions L = [A|[2,3]]. [[3,2]|1]. [H|T] = L. So I did this on the ERl and I saw this outcome * L = L = [A|[2,3]]. [1,2,3] which is expected * [[3,2]|1]. * 2: syntax error before: '[' which I expected to see also [ 1 , 2 , 3] * [H|T] = L. Which I see ** exception error: no match of right hand side value [1,2,3] where I was expected to see [1] [ 2, 3] Can someone help me figure out where my thinking took the wrong turn. Roelof From r.wobben@REDACTED Sun Jan 25 12:12:45 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 12:12:45 +0100 Subject: [erlang-questions] why do I see a message about area of a square Message-ID: <54C4CFAD.2050808@home.nl> An HTML attachment was scrubbed... URL: From taavi@REDACTED Sun Jan 25 12:22:43 2015 From: taavi@REDACTED (Taavi Talvik) Date: Sun, 25 Jan 2015 13:22:43 +0200 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <54C4CFAD.2050808@home.nl> References: <54C4CFAD.2050808@home.nl> Message-ID: <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> Until you export your functions, they remain local to module. And compiler sees here, that they are not used locally here. Try adding -export([area/1]). best regards, taavi On 25 Jan 2015, at 13:12, Roelof Wobben wrote: > Hello, > > I have this programm : > > -module(shapes). > > -import(math, [sqrt/1]). > > area({square, Side}) -> > Side * Side ; > > area({circle, Radius}) -> > math:pi() * Radius * Radius; > > area({triangle, A, B, C}) -> > S = (A + B + C)/2, > math:sqrt(S*(S-A)*(S-B)*(S-C)); > > area(Other) -> > {error, invalid_object}. > > > As soon I as try to compile it , I see these messages : > > > 6> c(shapes). > shapes.erl:5: Warning: function area/1 is unused > shapes.erl:15: Warning: variable 'Other' is unused > > Why does Erlang do this? > On compile time you cannot see which functions are used and which not in my oponion. > > Roelof > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From r.wobben@REDACTED Sun Jan 25 12:29:35 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 12:29:35 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> Message-ID: <54C4D39F.6000604@home.nl> Thanks, That error messages disappear. As I understand it right. The eror message is on all functions so on the square, circle and triangle functions. And if not so, why also not a error functions on the other two. Roelof Taavi Talvik schreef op 25-1-2015 om 12:22: > Until you export your functions, they remain local to > module. And compiler sees here, that they are not used > locally here. > > Try adding > -export([area/1]). > > best regards, > taavi > On 25 Jan 2015, at 13:12, Roelof Wobben wrote: > >> Hello, >> >> I have this programm : >> >> -module(shapes). >> >> -import(math, [sqrt/1]). >> >> area({square, Side}) -> >> Side * Side ; >> >> area({circle, Radius}) -> >> math:pi() * Radius * Radius; >> >> area({triangle, A, B, C}) -> >> S = (A + B + C)/2, >> math:sqrt(S*(S-A)*(S-B)*(S-C)); >> >> area(Other) -> >> {error, invalid_object}. >> >> >> As soon I as try to compile it , I see these messages : >> >> >> 6> c(shapes). >> shapes.erl:5: Warning: function area/1 is unused >> shapes.erl:15: Warning: variable 'Other' is unused >> >> Why does Erlang do this? >> On compile time you cannot see which functions are used and which not in my oponion. >> >> Roelof >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From vychodil.hynek@REDACTED Sun Jan 25 13:07:14 2015 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Sun, 25 Jan 2015 13:07:14 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <54C4D39F.6000604@home.nl> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> Message-ID: Hi, You don't have any functions named square, circle or triangle. You have only one function array/1 with four function clauses, the first for square, the second for circle, the third for triangle and the fourth catch-all one. Hynek On Sun, Jan 25, 2015 at 12:29 PM, Roelof Wobben wrote: > Thanks, > > That error messages disappear. > > As I understand it right. The eror message is on all functions so on the > square, circle and triangle functions. > And if not so, why also not a error functions on the other two. > > Roelof > > > > > Taavi Talvik schreef op 25-1-2015 om 12:22: > > Until you export your functions, they remain local to >> module. And compiler sees here, that they are not used >> locally here. >> >> Try adding >> -export([area/1]). >> >> best regards, >> taavi >> On 25 Jan 2015, at 13:12, Roelof Wobben wrote: >> >> Hello, >>> >>> I have this programm : >>> >>> -module(shapes). >>> >>> -import(math, [sqrt/1]). >>> >>> area({square, Side}) -> >>> Side * Side ; >>> >>> area({circle, Radius}) -> >>> math:pi() * Radius * Radius; >>> >>> area({triangle, A, B, C}) -> >>> S = (A + B + C)/2, >>> math:sqrt(S*(S-A)*(S-B)*(S-C)); >>> >>> area(Other) -> >>> {error, invalid_object}. >>> >>> >>> As soon I as try to compile it , I see these messages : >>> >>> >>> 6> c(shapes). >>> shapes.erl:5: Warning: function area/1 is unused >>> shapes.erl:15: Warning: variable 'Other' is unused >>> >>> Why does Erlang do this? >>> On compile time you cannot see which functions are used and which not in >>> my oponion. >>> >>> Roelof >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sun Jan 25 13:24:11 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 13:24:11 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> Message-ID: <54C4E06B.5030201@home.nl> An HTML attachment was scrubbed... URL: From vychodil.hynek@REDACTED Sun Jan 25 14:23:19 2015 From: vychodil.hynek@REDACTED (Hynek Vychodil) Date: Sun, 25 Jan 2015 14:23:19 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <54C4E06B.5030201@home.nl> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <54C4E06B.5030201@home.nl> Message-ID: Sorry, typo I mean area/1 and arity 1 because you have only one parameter. On Sun, Jan 25, 2015 at 1:24 PM, Roelof Wobben wrote: > Sorry, now im totally confused . > > where does the name array come from and why a arity 1 ? > > Roelof > > > Hynek Vychodil schreef op 25-1-2015 om 13:07: > > Hi, > > You don't have any functions named square, circle or triangle. You have > only one function array/1 with four function clauses, the first for square, > the second for circle, the third for triangle and the fourth catch-all one. > > Hynek > > On Sun, Jan 25, 2015 at 12:29 PM, Roelof Wobben wrote: > >> Thanks, >> >> That error messages disappear. >> >> As I understand it right. The eror message is on all functions so on the >> square, circle and triangle functions. >> And if not so, why also not a error functions on the other two. >> >> Roelof >> >> >> >> >> Taavi Talvik schreef op 25-1-2015 om 12:22: >> >> Until you export your functions, they remain local to >>> module. And compiler sees here, that they are not used >>> locally here. >>> >>> Try adding >>> -export([area/1]). >>> >>> best regards, >>> taavi >>> On 25 Jan 2015, at 13:12, Roelof Wobben wrote: >>> >>> Hello, >>>> >>>> I have this programm : >>>> >>>> -module(shapes). >>>> >>>> -import(math, [sqrt/1]). >>>> >>>> area({square, Side}) -> >>>> Side * Side ; >>>> >>>> area({circle, Radius}) -> >>>> math:pi() * Radius * Radius; >>>> >>>> area({triangle, A, B, C}) -> >>>> S = (A + B + C)/2, >>>> math:sqrt(S*(S-A)*(S-B)*(S-C)); >>>> >>>> area(Other) -> >>>> {error, invalid_object}. >>>> >>>> >>>> As soon I as try to compile it , I see these messages : >>>> >>>> >>>> 6> c(shapes). >>>> shapes.erl:5: Warning: function area/1 is unused >>>> shapes.erl:15: Warning: variable 'Other' is unused >>>> >>>> Why does Erlang do this? >>>> On compile time you cannot see which functions are used and which not >>>> in my oponion. >>>> >>>> Roelof >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sun Jan 25 14:36:25 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 14:36:25 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <54C4E06B.5030201@home.nl> Message-ID: <54C4F159.3060205@home.nl> An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 25 14:47:39 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 25 Jan 2015 14:47:39 +0100 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: <54C4C553.1070105@home.nl> References: <54C4C553.1070105@home.nl> Message-ID: For adding items to lists: this is bad: [ any() | Not_a_list ] ok: [ One_item | T_List ] The "T_List" must be a list - even if an empty one: [ ]. One_item ideally is not a list: atom, number, tuple, etc for "glueing" lists together, use lists:append/1,2 From r.wobben@REDACTED Sun Jan 25 18:16:46 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sun, 25 Jan 2015 18:16:46 +0100 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: References: <54C4C553.1070105@home.nl> Message-ID: <54C524FE.4060404@home.nl> oke, So [[3,2]|1] is bad because there is more then 1 item. If I do [1 | [ 2,3] ] It would be right. Roelof Imants Cekusins schreef op 25-1-2015 om 14:47: > For adding items to lists: > > this is bad: > [ any() | Not_a_list ] > > ok: > [ One_item | T_List ] > > The "T_List" must be a list - even if an empty one: [ ]. > One_item ideally is not a list: atom, number, tuple, etc > > for "glueing" lists together, use > lists:append/1,2 > From bob@REDACTED Sun Jan 25 18:24:09 2015 From: bob@REDACTED (Bob Ippolito) Date: Sun, 25 Jan 2015 09:24:09 -0800 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: <54C524FE.4060404@home.nl> References: <54C4C553.1070105@home.nl> <54C524FE.4060404@home.nl> Message-ID: [X|1] is bad because 1 is not a list. Any X would be valid if the tail was a list. On Sunday, January 25, 2015, Roelof Wobben wrote: > oke, > > So [[3,2]|1] is bad because there is more then 1 item. > If I do [1 | [ 2,3] ] It would be right. > > Roelof > > > Imants Cekusins schreef op 25-1-2015 om 14:47: > >> For adding items to lists: >> >> this is bad: >> [ any() | Not_a_list ] >> >> ok: >> [ One_item | T_List ] >> >> The "T_List" must be a list - even if an empty one: [ ]. >> One_item ideally is not a list: atom, number, tuple, etc >> >> for "glueing" lists together, use >> lists:append/1,2 >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 25 18:30:37 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 25 Jan 2015 18:30:37 +0100 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: <54C524FE.4060404@home.nl> References: <54C4C553.1070105@home.nl> <54C524FE.4060404@home.nl> Message-ID: Well in [[3,2] | 1] it's 1 that is not ok. [3,2] as a head could work. For example, a string (which is a list) can be a Head. The tail however is always a list. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Sun Jan 25 18:53:58 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sun, 25 Jan 2015 18:53:58 +0100 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: References: <54C4C553.1070105@home.nl> <54C524FE.4060404@home.nl> Message-ID: a little more on list as a Head: 2> A = [ [1,2] | [3,4] ]. [[1,2],3,4] 3> [ H | T ] = A. [[1,2],3,4] 4> H. [1,2] in other words, in [ H | T ] H is added as, and stays a single item. From e@REDACTED Sun Jan 25 18:28:01 2015 From: e@REDACTED (e@REDACTED) Date: Sun, 25 Jan 2015 18:28:01 +0100 Subject: [erlang-questions] Ranch + SSL = crash during init:stop/0 Message-ID: <54C527A1.3000505@bestmx.net> Hello. I have a well localized problem with Ranch that raises several basic questions about OTP, which i need to understand in order to solve the problem. i have got the following applications running on a single isolated node: 1> application:which_applications(). [{online37,"My Application","0.1.0"}, {cowboy,"Small, fast, modular HTTP server.","1.0.0"}, {ssl,"Erlang/OTP SSL application","5.3.6"}, {public_key,"Public key infrastructure","0.22.1"}, {cowlib,"Support library for manipulating Web protocols.", "1.0.0"}, {mnesia,"MNESIA CXC 138 12","4.12.3"}, {crypto,"CRYPTO","3.4.1"}, {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, {ranch,"Socket acceptor pool for TCP protocols.","1.0.0"}, {stdlib,"ERTS CXC 138 10","2.2"}, {kernel,"ERTS CXC 138 10","3.0.3"}] Q1: does this list represent the starting order of these applications listed? Q2: may i deduce from this list that 'ssl' will be stopped before 'ranch'? (should be called init:stop/0) the main problem is that during init:stop/0 'ranch' *DELIBERATELY* crashes with the reason: "socket closed" (in src/ranch_acceptor.erl:28) (which in my opinion most probably is caused by preliminary shut down of 'ssl') Q3: am i right in respect of 'ranch' that 'ssl' shall not be stopped before 'ranch'? my .app.src file (which corresponds to the applications listing above) is: {application, online37, [ {description, "My Application"}, {vsn, "0.1.0"}, {modules, []}, {registered, []}, {applications, [ kernel, stdlib, ssl, mnesia, cowboy ]}, {mod, {online37_app, []}}, {env, []} ]}. Q4: how could i affect the starting order of the applications? (provided i am using erlang.mk + relx) From silver.buddy@REDACTED Sun Jan 25 20:24:24 2015 From: silver.buddy@REDACTED (silver.buddy@REDACTED) Date: Sun, 25 Jan 2015 22:24:24 +0300 Subject: [erlang-questions] Ranch + SSL = crash during init:stop/0 In-Reply-To: <54C527A1.3000505@bestmx.net> References: <54C527A1.3000505@bestmx.net> Message-ID: <54C542E8.4050509@yandex.ru> Hello, Look at this https://github.com/ninenines/ranch/issues/90 On 25/01/15 20:28, e@REDACTED wrote: > Hello. > I have a well localized problem with Ranch that raises several basic > questions about OTP, which i need to understand in order to solve the > problem. > > i have got the following applications running on a single isolated node: > > 1> application:which_applications(). > > [{online37,"My Application","0.1.0"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.0"}, > {ssl,"Erlang/OTP SSL application","5.3.6"}, > {public_key,"Public key infrastructure","0.22.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.0"}, > {mnesia,"MNESIA CXC 138 12","4.12.3"}, > {crypto,"CRYPTO","3.4.1"}, > {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.0.0"}, > {stdlib,"ERTS CXC 138 10","2.2"}, > {kernel,"ERTS CXC 138 10","3.0.3"}] > > > Q1: does this list represent the starting order of these applications > listed? > > Q2: may i deduce from this list that 'ssl' will be stopped before > 'ranch'? (should be called init:stop/0) > > > the main problem is that during init:stop/0 > 'ranch' *DELIBERATELY* crashes with the reason: "socket closed" > (in src/ranch_acceptor.erl:28) > (which in my opinion most probably is caused by preliminary shut down of > 'ssl') > > > Q3: am i right in respect of 'ranch' that 'ssl' shall not be stopped > before 'ranch'? > > > my .app.src file (which corresponds to the applications listing above) is: > > {application, online37, [ > {description, "My Application"}, > {vsn, "0.1.0"}, > {modules, []}, > {registered, []}, > {applications, [ > kernel, > stdlib, > ssl, > mnesia, > cowboy > ]}, > {mod, {online37_app, []}}, > {env, []} > ]}. > > > Q4: how could i affect the starting order of the applications? (provided > i am using erlang.mk + relx) From cjsvance@REDACTED Sun Jan 25 23:42:31 2015 From: cjsvance@REDACTED (Christopher Vance) Date: Mon, 26 Jan 2015 09:42:31 +1100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <54C4F159.3060205@home.nl> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <54C4E06B.5030201@home.nl> <54C4F159.3060205@home.nl> Message-ID: The argument is a tuple. Erlang does not have arrays; you have to decide whether to use a list or a tuple. On Mon, Jan 26, 2015 at 12:36 AM, Roelof Wobben wrote: > Nope, Thanks for the patience with me. > > So area is the function with 1 argument a array with the shape and the > parameters to calculate it. > > Roelof > > > > Hynek Vychodil schreef op 25-1-2015 om 14:23: > > Sorry, typo I mean area/1 and arity 1 because you have only one parameter. > > On Sun, Jan 25, 2015 at 1:24 PM, Roelof Wobben wrote: > >> Sorry, now im totally confused . >> >> where does the name array come from and why a arity 1 ? >> >> Roelof >> >> >> Hynek Vychodil schreef op 25-1-2015 om 13:07: >> >> Hi, >> >> You don't have any functions named square, circle or triangle. You have >> only one function array/1 with four function clauses, the first for square, >> the second for circle, the third for triangle and the fourth catch-all one. >> >> Hynek >> >> On Sun, Jan 25, 2015 at 12:29 PM, Roelof Wobben wrote: >> >>> Thanks, >>> >>> That error messages disappear. >>> >>> As I understand it right. The eror message is on all functions so on the >>> square, circle and triangle functions. >>> And if not so, why also not a error functions on the other two. >>> >>> Roelof >>> >>> >>> >>> >>> Taavi Talvik schreef op 25-1-2015 om 12:22: >>> >>> Until you export your functions, they remain local to >>>> module. And compiler sees here, that they are not used >>>> locally here. >>>> >>>> Try adding >>>> -export([area/1]). >>>> >>>> best regards, >>>> taavi >>>> On 25 Jan 2015, at 13:12, Roelof Wobben wrote: >>>> >>>> Hello, >>>>> >>>>> I have this programm : >>>>> >>>>> -module(shapes). >>>>> >>>>> -import(math, [sqrt/1]). >>>>> >>>>> area({square, Side}) -> >>>>> Side * Side ; >>>>> >>>>> area({circle, Radius}) -> >>>>> math:pi() * Radius * Radius; >>>>> >>>>> area({triangle, A, B, C}) -> >>>>> S = (A + B + C)/2, >>>>> math:sqrt(S*(S-A)*(S-B)*(S-C)); >>>>> >>>>> area(Other) -> >>>>> {error, invalid_object}. >>>>> >>>>> >>>>> As soon I as try to compile it , I see these messages : >>>>> >>>>> >>>>> 6> c(shapes). >>>>> shapes.erl:5: Warning: function area/1 is unused >>>>> shapes.erl:15: Warning: variable 'Other' is unused >>>>> >>>>> Why does Erlang do this? >>>>> On compile time you cannot see which functions are used and which not >>>>> in my oponion. >>>>> >>>>> Roelof >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Christopher Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From harit.subscriptions@REDACTED Mon Jan 26 01:40:23 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Sun, 25 Jan 2015 16:40:23 -0800 Subject: [erlang-questions] [Beginner] Project Ideas - Help Needed Message-ID: Hello there, I am new to Erlang and reason I started learning is because I wanted to learn about how concurrency works and how to design such systems. I have been through Programming Erlang book (read and worked up to Files chapter), but I am not able to motivate or find the ideas for projects that I must do to solidify my understanding over concurrency concepts. Good thing is that I really enjoy writing Erlang code and want to continue. I thought of simulating an RTB server, but there are too many components involved and so much of data setup needed(and I do not know completely how RTB works), that I got demotivated. Google searches directed to *ambitious* things like IRC chat, messaging application amongst others Please provide your suggestions/recommendations on the projects ideas that may help beginner to language (and concurrency) to take baby steps and contribute meaningful in terms of projects and eventually feel confident to design, implement and debug bigger systems. Thank you for reading till here, really appreciate your time + Harit Himanshu -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Mon Jan 26 02:54:06 2015 From: rpettit@REDACTED (Rick Pettit) Date: Sun, 25 Jan 2015 19:54:06 -0600 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <54C4E06B.5030201@home.nl> <54C4F159.3060205@home.nl> Message-ID: <40F17287-EC3E-4604-84AC-4CDF83780230@vailsys.com> Technically Erlang does have arrays?see ?erl -man array? for details. -Rick > On Jan 25, 2015, at 4:42 PM, Christopher Vance wrote: > > The argument is a tuple. Erlang does not have arrays; you have to decide whether to use a list or a tuple. > > On Mon, Jan 26, 2015 at 12:36 AM, Roelof Wobben > wrote: > Nope, Thanks for the patience with me. > > So area is the function with 1 argument a array with the shape and the parameters to calculate it. > > Roelof > > > > Hynek Vychodil schreef op 25-1-2015 om 14:23: >> Sorry, typo I mean area/1 and arity 1 because you have only one parameter. >> >> On Sun, Jan 25, 2015 at 1:24 PM, Roelof Wobben > wrote: >> Sorry, now im totally confused . >> >> where does the name array come from and why a arity 1 ? >> >> Roelof >> >> >> Hynek Vychodil schreef op 25-1-2015 om 13:07: >>> Hi, >>> >>> You don't have any functions named square, circle or triangle. You have only one function array/1 with four function clauses, the first for square, the second for circle, the third for triangle and the fourth catch-all one. >>> >>> Hynek >>> >>> On Sun, Jan 25, 2015 at 12:29 PM, Roelof Wobben > wrote: >>> Thanks, >>> >>> That error messages disappear. >>> >>> As I understand it right. The eror message is on all functions so on the square, circle and triangle functions. >>> And if not so, why also not a error functions on the other two. >>> >>> Roelof >>> >>> >>> >>> >>> Taavi Talvik schreef op 25-1-2015 om 12:22: >>> >>> Until you export your functions, they remain local to >>> module. And compiler sees here, that they are not used >>> locally here. >>> >>> Try adding >>> -export([area/1]). >>> >>> best regards, >>> taavi >>> On 25 Jan 2015, at 13:12, Roelof Wobben > wrote: >>> >>> Hello, >>> >>> I have this programm : >>> >>> -module(shapes). >>> >>> -import(math, [sqrt/1]). >>> >>> area({square, Side}) -> >>> Side * Side ; >>> >>> area({circle, Radius}) -> >>> math:pi() * Radius * Radius; >>> >>> area({triangle, A, B, C}) -> >>> S = (A + B + C)/2, >>> math:sqrt(S*(S-A)*(S-B)*(S-C)); >>> >>> area(Other) -> >>> {error, invalid_object}. >>> >>> >>> As soon I as try to compile it , I see these messages : >>> >>> >>> 6> c(shapes). >>> shapes.erl:5: Warning: function area/1 is unused >>> shapes.erl:15: Warning: variable 'Other' is unused >>> >>> Why does Erlang do this? >>> On compile time you cannot see which functions are used and which not in my oponion. >>> >>> Roelof >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Christopher Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Mon Jan 26 00:40:04 2015 From: e@REDACTED (e@REDACTED) Date: Mon, 26 Jan 2015 00:40:04 +0100 Subject: [erlang-questions] Ranch + SSL = crash during init:stop/0 In-Reply-To: <54C542E8.4050509@yandex.ru> References: <54C527A1.3000505@bestmx.net> <54C542E8.4050509@yandex.ru> Message-ID: <54C57ED4.6000808@bestmx.net> On 01/25/2015 08:24 PM, silver.buddy@REDACTED wrote: > Hello, > > Look at this https://github.com/ninenines/ranch/issues/90 does this problem have a general solution? Ranch may or may not depend on ssl (according to the listener i create) Could they fix ranch.app once and for all? > > On 25/01/15 20:28, e@REDACTED wrote: >> Hello. >> I have a well localized problem with Ranch that raises several basic >> questions about OTP, which i need to understand in order to solve the >> problem. >> >> i have got the following applications running on a single isolated node: >> >> 1> application:which_applications(). >> >> [{online37,"My Application","0.1.0"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.0"}, >> {ssl,"Erlang/OTP SSL application","5.3.6"}, >> {public_key,"Public key infrastructure","0.22.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.0"}, >> {mnesia,"MNESIA CXC 138 12","4.12.3"}, >> {crypto,"CRYPTO","3.4.1"}, >> {asn1,"The Erlang ASN1 compiler version 3.0.2","3.0.2"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.0.0"}, >> {stdlib,"ERTS CXC 138 10","2.2"}, >> {kernel,"ERTS CXC 138 10","3.0.3"}] >> >> >> Q1: does this list represent the starting order of these applications >> listed? >> >> Q2: may i deduce from this list that 'ssl' will be stopped before >> 'ranch'? (should be called init:stop/0) >> >> >> the main problem is that during init:stop/0 >> 'ranch' *DELIBERATELY* crashes with the reason: "socket closed" >> (in src/ranch_acceptor.erl:28) >> (which in my opinion most probably is caused by preliminary shut down of >> 'ssl') >> >> >> Q3: am i right in respect of 'ranch' that 'ssl' shall not be stopped >> before 'ranch'? >> >> >> my .app.src file (which corresponds to the applications listing above) >> is: >> >> {application, online37, [ >> {description, "My Application"}, >> {vsn, "0.1.0"}, >> {modules, []}, >> {registered, []}, >> {applications, [ >> kernel, >> stdlib, >> ssl, >> mnesia, >> cowboy >> ]}, >> {mod, {online37_app, []}}, >> {env, []} >> ]}. >> >> >> Q4: how could i affect the starting order of the applications? (provided >> i am using erlang.mk + relx) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From cjsvance@REDACTED Mon Jan 26 08:30:22 2015 From: cjsvance@REDACTED (Christopher Vance) Date: Mon, 26 Jan 2015 18:30:22 +1100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <40F17287-EC3E-4604-84AC-4CDF83780230@vailsys.com> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <54C4E06B.5030201@home.nl> <54C4F159.3060205@home.nl> <40F17287-EC3E-4604-84AC-4CDF83780230@vailsys.com> Message-ID: Even more technically, an Erlang "array" is a tuple containing a tree of tuples. On Mon, Jan 26, 2015 at 12:54 PM, Rick Pettit wrote: > Technically Erlang does have arrays?see ?erl -man array? for details. > > -Rick > > On Jan 25, 2015, at 4:42 PM, Christopher Vance wrote: > > The argument is a tuple. Erlang does not have arrays; you have to decide > whether to use a list or a tuple. > > On Mon, Jan 26, 2015 at 12:36 AM, Roelof Wobben wrote: > >> Nope, Thanks for the patience with me. >> >> So area is the function with 1 argument a array with the shape and the >> parameters to calculate it. >> >> Roelof >> >> >> >> Hynek Vychodil schreef op 25-1-2015 om 14:23: >> >> Sorry, typo I mean area/1 and arity 1 because you have only one parameter. >> >> On Sun, Jan 25, 2015 at 1:24 PM, Roelof Wobben wrote: >> >>> Sorry, now im totally confused . >>> >>> where does the name array come from and why a arity 1 ? >>> >>> Roelof >>> >>> >>> Hynek Vychodil schreef op 25-1-2015 om 13:07: >>> >>> Hi, >>> >>> You don't have any functions named square, circle or triangle. You >>> have only one function array/1 with four function clauses, the first for >>> square, the second for circle, the third for triangle and the fourth >>> catch-all one. >>> >>> Hynek >>> >>> On Sun, Jan 25, 2015 at 12:29 PM, Roelof Wobben >>> wrote: >>> >>>> Thanks, >>>> >>>> That error messages disappear. >>>> >>>> As I understand it right. The eror message is on all functions so on >>>> the square, circle and triangle functions. >>>> And if not so, why also not a error functions on the other two. >>>> >>>> Roelof >>>> >>>> >>>> >>>> >>>> Taavi Talvik schreef op 25-1-2015 om 12:22: >>>> >>>> Until you export your functions, they remain local to >>>>> module. And compiler sees here, that they are not used >>>>> locally here. >>>>> >>>>> Try adding >>>>> -export([area/1]). >>>>> >>>>> best regards, >>>>> taavi >>>>> On 25 Jan 2015, at 13:12, Roelof Wobben wrote: >>>>> >>>>> Hello, >>>>>> >>>>>> I have this programm : >>>>>> >>>>>> -module(shapes). >>>>>> >>>>>> -import(math, [sqrt/1]). >>>>>> >>>>>> area({square, Side}) -> >>>>>> Side * Side ; >>>>>> >>>>>> area({circle, Radius}) -> >>>>>> math:pi() * Radius * Radius; >>>>>> >>>>>> area({triangle, A, B, C}) -> >>>>>> S = (A + B + C)/2, >>>>>> math:sqrt(S*(S-A)*(S-B)*(S-C)); >>>>>> >>>>>> area(Other) -> >>>>>> {error, invalid_object}. >>>>>> >>>>>> >>>>>> As soon I as try to compile it , I see these messages : >>>>>> >>>>>> >>>>>> 6> c(shapes). >>>>>> shapes.erl:5: Warning: function area/1 is unused >>>>>> shapes.erl:15: Warning: variable 'Other' is unused >>>>>> >>>>>> Why does Erlang do this? >>>>>> On compile time you cannot see which functions are used and which not >>>>>> in my oponion. >>>>>> >>>>>> Roelof >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Christopher Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Christopher Vance -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Jan 26 12:44:00 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 26 Jan 2015 12:44:00 +0100 Subject: [erlang-questions] Ranch + SSL = crash during init:stop/0 In-Reply-To: <54C57ED4.6000808@bestmx.net> References: <54C527A1.3000505@bestmx.net> <54C542E8.4050509@yandex.ru> <54C57ED4.6000808@bestmx.net> Message-ID: <54C62880.2080303@ninenines.eu> On 01/26/2015 12:40 AM, e@REDACTED wrote: > On 01/25/2015 08:24 PM, silver.buddy@REDACTED wrote: >> Hello, >> >> Look at this https://github.com/ninenines/ranch/issues/90 > > does this problem have a general solution? > Ranch may or may not depend on ssl (according to the listener i create) > Could they fix ranch.app once and for all? The plan is to fix ranch.app, yes. But the change is kind of a big deal so alternative solutions were sought before committing to this change. -- Lo?c Hoguin http://ninenines.eu From essen@REDACTED Mon Jan 26 12:58:38 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 26 Jan 2015 12:58:38 +0100 Subject: [erlang-questions] [Beginner] Project Ideas - Help Needed In-Reply-To: References: Message-ID: <54C62BEE.2040407@ninenines.eu> Write what you're passionate about. I am passionate about games so my first project was an attempt at reverse engineering a game server and reimplementing it in Erlang. I am passionate about HTTP so my second project was an HTTP server. If you're not passionate about something then there's not much point working on it, you'll lose motivation. The motivation comes from what you're trying to build. Learning Erlang is just a way to achieve that. On 01/26/2015 01:40 AM, Harit Himanshu wrote: > Google searches directed to /ambitious/ things like IRC chat, messaging > application amongst others They're not actually ambitious. Most Erlang books and trainings have some sort of (often distributed) chat, you just need to add the IRC or other protocol on top of it. You also don't need to implement everything. Use hardcoded values until you implement the missing functionality. -- Lo?c Hoguin http://ninenines.eu From r.wobben@REDACTED Mon Jan 26 15:09:19 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 15:09:19 +0100 Subject: [erlang-questions] stuck at a exercise Message-ID: <54C64A8F.2070603@home.nl> Hello, At a book im following with self-study I have to do this exercise : Write a module boolean.erlthat takes logical expressions and Boolean values (represented as the atoms trueand false) and returns their Boolean result. The functions you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You should not use the logical constructs and, or, and not, but instead use pattern matching to achieve your goal. So I tried the first clause and came with this : -module(boolean). -export([boolean/1]). b_not({true}) -> false. But as soon as I compile it I see these error messages : boolean.erl:3: function boolean/1 undefined boolean.erl:5: Warning: function b_not/1 is unused How to solve these ? Roelof From bengt.kleberg@REDACTED Mon Jan 26 15:10:39 2015 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 26 Jan 2015 15:10:39 +0100 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C64A8F.2070603@home.nl> References: <54C64A8F.2070603@home.nl> Message-ID: <54C64ADF.40401@ericsson.com> You should export functions. In your case b_not/1 bengt On 01/26/2015 03:09 PM, Roelof Wobben wrote: > Hello, > > At a book im following with self-study I have to do this exercise : > > Write a module boolean.erlthat takes logical expressions and Boolean > values (represented as the atoms trueand false) and returns their > Boolean result. The functions > you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You > should not use > the logical constructs and, or, and not, but instead use pattern > matching to achieve your > goal. > > So I tried the first clause and came with this : > > -module(boolean). > > -export([boolean/1]). > > b_not({true}) -> > false. > > But as soon as I compile it I see these error messages : > > boolean.erl:3: function boolean/1 undefined > boolean.erl:5: Warning: function b_not/1 is unused > > How to solve these ? > > Roelof > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From leonard.boyce@REDACTED Mon Jan 26 15:24:32 2015 From: leonard.boyce@REDACTED (Leonard Boyce) Date: Mon, 26 Jan 2015 09:24:32 -0500 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C64A8F.2070603@home.nl> References: <54C64A8F.2070603@home.nl> Message-ID: Hi Roelof, Welcome to Erlang. On Mon, Jan 26, 2015 at 9:09 AM, Roelof Wobben wrote: > Hello, > > At a book im following with self-study I have to do this exercise : > > Write a module boolean.erlthat takes logical expressions and Boolean values > (represented as the atoms trueand false) and returns their Boolean result. > The functions > you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You should > not use > the logical constructs and, or, and not, but instead use pattern matching > to achieve your > goal. > > So I tried the first clause and came with this : > > -module(boolean). > > -export([boolean/1]). > > b_not({true}) -> > false. > > But as soon as I compile it I see these error messages : > > boolean.erl:3: function boolean/1 undefined > boolean.erl:5: Warning: function b_not/1 is unused > > How to solve these ? I'm sure to get some hate for this, but I often find it easiest to explain in imperative programming terms to people new to the language; module == class public methods go in -exports([]) private methods are not exported and are only callable from within the module itself In your case you'd want to export the function b_not/1 Which would be callable as boolean:b_not(X) You do not export the module name, only functions you want to be callable. Leonard > Roelof > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From r.wobben@REDACTED Mon Jan 26 15:35:09 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 15:35:09 +0100 Subject: [erlang-questions] stuck at a exercise In-Reply-To: References: <54C64A8F.2070603@home.nl> Message-ID: <54C6509D.5030106@home.nl> Leonard Boyce schreef op 26-1-2015 om 15:24: > Hi Roelof, > > Welcome to Erlang. > > On Mon, Jan 26, 2015 at 9:09 AM, Roelof Wobben wrote: >> Hello, >> >> At a book im following with self-study I have to do this exercise : >> >> Write a module boolean.erlthat takes logical expressions and Boolean values >> (represented as the atoms trueand false) and returns their Boolean result. >> The functions >> you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You should >> not use >> the logical constructs and, or, and not, but instead use pattern matching >> to achieve your >> goal. >> >> So I tried the first clause and came with this : >> >> -module(boolean). >> >> -export([boolean/1]). >> >> b_not({true}) -> >> false. >> >> But as soon as I compile it I see these error messages : >> >> boolean.erl:3: function boolean/1 undefined >> boolean.erl:5: Warning: function b_not/1 is unused >> >> How to solve these ? > I'm sure to get some hate for this, but I often find it easiest to > explain in imperative programming terms to people new to the language; > > module == class > public methods go in -exports([]) > private methods are not exported and are only callable from within the > module itself > > In your case you'd want to export the function b_not/1 > > Which would be callable as boolean:b_not(X) > > You do not export the module name, only functions you want to be callable. > > Leonard > >> Roelof >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions Thanks Second problem . I added a second clause to it like this: -module(boolean). -export([b_not/1]). b_not(true) -> false. b_not(false) -> true. but now I see this message ; boolean.erl:8: function b_not/1 already defined error Which I do not understand because here the same approach is used : -module(shapes). -import(math, [sqrt/1]). -export([area/1]). area({square, Side}) -> Side * Side ; area({circle, Radius}) -> math:pi() * Radius * Radius; From bengt.kleberg@REDACTED Mon Jan 26 15:39:05 2015 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 26 Jan 2015 15:39:05 +0100 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C6509D.5030106@home.nl> References: <54C64A8F.2070603@home.nl> <54C6509D.5030106@home.nl> Message-ID: <54C65189.2070402@ericsson.com> The same function can have different clauses. These have the same name and arity, and are separated with ";" Only after the last clause do you have ".". bengt On 01/26/2015 03:35 PM, Roelof Wobben wrote: > Leonard Boyce schreef op 26-1-2015 om 15:24: >> Hi Roelof, >> >> Welcome to Erlang. >> >> On Mon, Jan 26, 2015 at 9:09 AM, Roelof Wobben wrote: >>> Hello, >>> >>> At a book im following with self-study I have to do this exercise : >>> >>> Write a module boolean.erlthat takes logical expressions and Boolean >>> values >>> (represented as the atoms trueand false) and returns their Boolean >>> result. >>> The functions >>> you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You >>> should >>> not use >>> the logical constructs and, or, and not, but instead use pattern >>> matching >>> to achieve your >>> goal. >>> >>> So I tried the first clause and came with this : >>> >>> -module(boolean). >>> >>> -export([boolean/1]). >>> >>> b_not({true}) -> >>> false. >>> >>> But as soon as I compile it I see these error messages : >>> >>> boolean.erl:3: function boolean/1 undefined >>> boolean.erl:5: Warning: function b_not/1 is unused >>> >>> How to solve these ? >> I'm sure to get some hate for this, but I often find it easiest to >> explain in imperative programming terms to people new to the language; >> >> module == class >> public methods go in -exports([]) >> private methods are not exported and are only callable from within the >> module itself >> >> In your case you'd want to export the function b_not/1 >> >> Which would be callable as boolean:b_not(X) >> >> You do not export the module name, only functions you want to be >> callable. >> >> Leonard >> >>> Roelof >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > Thanks > > Second problem . > > I added a second clause to it like this: > > -module(boolean). > > -export([b_not/1]). > > b_not(true) -> > false. > > b_not(false) -> > true. > > but now I see this message ; > > boolean.erl:8: function b_not/1 already defined > error > > Which I do not understand because here the same approach is used : > > -module(shapes). > > -import(math, [sqrt/1]). > > -export([area/1]). > > area({square, Side}) -> > Side * Side ; > > area({circle, Radius}) -> > math:pi() * Radius * Radius; > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Mon Jan 26 15:40:12 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 26 Jan 2015 14:40:12 +0000 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C6509D.5030106@home.nl> References: <54C64A8F.2070603@home.nl> <54C6509D.5030106@home.nl> Message-ID: <54C651CC.1010809@llaisdy.com> In case you haven't seen it, the Erlang Handbook is an excellent clear short guide to erlang syntax and practice: http://opensource.erlang-solutions.com/erlang-handbook/ Best wishes Ivan On 26/01/2015 14:35, Roelof Wobben wrote: > Leonard Boyce schreef op 26-1-2015 om 15:24: >> Hi Roelof, >> >> Welcome to Erlang. >> >> On Mon, Jan 26, 2015 at 9:09 AM, Roelof Wobben wrote: >>> Hello, >>> >>> At a book im following with self-study I have to do this exercise : >>> >>> Write a module boolean.erlthat takes logical expressions and Boolean >>> values >>> (represented as the atoms trueand false) and returns their Boolean >>> result. >>> The functions >>> you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. You >>> should >>> not use >>> the logical constructs and, or, and not, but instead use pattern >>> matching >>> to achieve your >>> goal. >>> >>> So I tried the first clause and came with this : >>> >>> -module(boolean). >>> >>> -export([boolean/1]). >>> >>> b_not({true}) -> >>> false. >>> >>> But as soon as I compile it I see these error messages : >>> >>> boolean.erl:3: function boolean/1 undefined >>> boolean.erl:5: Warning: function b_not/1 is unused >>> >>> How to solve these ? >> I'm sure to get some hate for this, but I often find it easiest to >> explain in imperative programming terms to people new to the language; >> >> module == class >> public methods go in -exports([]) >> private methods are not exported and are only callable from within the >> module itself >> >> In your case you'd want to export the function b_not/1 >> >> Which would be callable as boolean:b_not(X) >> >> You do not export the module name, only functions you want to be >> callable. >> >> Leonard >> >>> Roelof >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > Thanks > > Second problem . > > I added a second clause to it like this: > > -module(boolean). > > -export([b_not/1]). > > b_not(true) -> > false. > > b_not(false) -> > true. > > but now I see this message ; > > boolean.erl:8: function b_not/1 already defined > error > > Which I do not understand because here the same approach is used : > > -module(shapes). > > -import(math, [sqrt/1]). > > -export([area/1]). > > area({square, Side}) -> > Side * Side ; > > area({circle, Radius}) -> > math:pi() * Radius * Radius; > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From r.wobben@REDACTED Mon Jan 26 15:43:55 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 15:43:55 +0100 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C651CC.1010809@llaisdy.com> References: <54C64A8F.2070603@home.nl> <54C6509D.5030106@home.nl> <54C651CC.1010809@llaisdy.com> Message-ID: <54C652AB.5010406@home.nl> Ivan Uemlianin schreef op 26-1-2015 om 15:40: > In case you haven't seen it, the Erlang Handbook is an excellent clear > short guide to erlang syntax and practice: > > http://opensource.erlang-solutions.com/erlang-handbook/ > > Best wishes > > Ivan > > > On 26/01/2015 14:35, Roelof Wobben wrote: >> Leonard Boyce schreef op 26-1-2015 om 15:24: >>> Hi Roelof, >>> >>> Welcome to Erlang. >>> >>> On Mon, Jan 26, 2015 at 9:09 AM, Roelof Wobben >>> wrote: >>>> Hello, >>>> >>>> At a book im following with self-study I have to do this exercise : >>>> >>>> Write a module boolean.erlthat takes logical expressions and >>>> Boolean values >>>> (represented as the atoms trueand false) and returns their >>>> Boolean result. >>>> The functions >>>> you write should include b_not/1, b_and/2, b_or/2, and b_nand/2. >>>> You should >>>> not use >>>> the logical constructs and, or, and not, but instead use pattern >>>> matching >>>> to achieve your >>>> goal. >>>> >>>> So I tried the first clause and came with this : >>>> >>>> -module(boolean). >>>> >>>> -export([boolean/1]). >>>> >>>> b_not({true}) -> >>>> false. >>>> >>>> But as soon as I compile it I see these error messages : >>>> >>>> boolean.erl:3: function boolean/1 undefined >>>> boolean.erl:5: Warning: function b_not/1 is unused >>>> >>>> How to solve these ? >>> I'm sure to get some hate for this, but I often find it easiest to >>> explain in imperative programming terms to people new to the language; >>> >>> module == class >>> public methods go in -exports([]) >>> private methods are not exported and are only callable from within the >>> module itself >>> >>> In your case you'd want to export the function b_not/1 >>> >>> Which would be callable as boolean:b_not(X) >>> >>> You do not export the module name, only functions you want to be >>> callable. >>> >>> Leonard >>> >>>> Roelof >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> Thanks >> >> Second problem . >> >> I added a second clause to it like this: >> >> -module(boolean). >> >> -export([b_not/1]). >> >> b_not(true) -> >> false. >> >> b_not(false) -> >> true. >> >> but now I see this message ; >> >> boolean.erl:8: function b_not/1 already defined >> error >> >> Which I do not understand because here the same approach is used : >> >> -module(shapes). >> >> -import(math, [sqrt/1]). >> >> -export([area/1]). >> >> area({square, Side}) -> >> Side * Side ; >> >> area({circle, Radius}) -> >> math:pi() * Radius * Radius; >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Thanks all. Im reading the erlang programming book. Roelof From jesper.louis.andersen@REDACTED Mon Jan 26 15:47:32 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 26 Jan 2015 15:47:32 +0100 Subject: [erlang-questions] stuck at a exercise In-Reply-To: References: <54C64A8F.2070603@home.nl> Message-ID: On Mon, Jan 26, 2015 at 3:24 PM, Leonard Boyce wrote: > I'm sure to get some hate for this, but I often find it easiest to > explain in imperative programming terms to people new to the language; > > module == class > public methods go in -exports([]) > private methods are not exported and are only callable from within the > module itself > The only problem with such an explanation is that it isn't precise, which may lead people to make the wrong train-of-thought later on in the process. The imprecision is due to how many OOP-languages conflate different concepts[0] such as class, module, encapsulation, namespace, scope, and package into the very same thing. In Erlang, you would only ascribe some of those properties to a module, but not all of them. Also, the term "class" means different things depending on your programming language background. And some languages with OOP concepts do not have a notion of class, while they do have the module concept. On the other hand, it may propel some learners to a better position quickly, so it falls both ways. [0] Even the concept of OO is not that well defined. I love to point out Jonathan Rees on OO: http://www.paulgraham.com/reesoo.html in which he describes how different OO concepts are cherry picked to form the viewpoint that *your* language is object oriented. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Mon Jan 26 17:21:42 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 17:21:42 +0100 Subject: [erlang-questions] nand problem Message-ID: <54C66996.1030109@home.nl> Hello, I now have to make a nand boolean with pattern matching. I have found out that nand is true except when its true and true. as hint I get to implement nand using and and or. Can someone give me a hint how to do this ? My code looks like this at the moment : -module(boolean). -export([b_not/1, b_and/2, b_or/2]). b_not(true) -> false; b_not(false) -> true. b_and(true, true) -> true; b_and(true, false) -> false; b_and(false, true) -> false; b_and(false, false) -> false. b_or(true, true) -> true; b_or(true, false) -> true; b_or(false, true) -> true; b_or(false, false) -> false. Roelof From bob@REDACTED Mon Jan 26 17:27:30 2015 From: bob@REDACTED (Bob Ippolito) Date: Mon, 26 Jan 2015 08:27:30 -0800 Subject: [erlang-questions] nand problem In-Reply-To: <54C66996.1030109@home.nl> References: <54C66996.1030109@home.nl> Message-ID: Here's a hint: http://en.wikipedia.org/wiki/NAND_gate You already have an inverter by another name. Although in this case I think implementing NAND as a pattern match would be quite simple. Have you learned about wildcard patterns yet (the _ symbol)? On Mon, Jan 26, 2015 at 8:21 AM, Roelof Wobben wrote: > Hello, > > I now have to make a nand boolean with pattern matching. > I have found out that nand is true except when its true and true. > > as hint I get to implement nand using and and or. > > Can someone give me a hint how to do this ? > > > My code looks like this at the moment : > > -module(boolean). > > -export([b_not/1, b_and/2, b_or/2]). > > b_not(true) -> > false; > > b_not(false) -> > true. > > b_and(true, true) -> > true; > > b_and(true, false) -> > false; > > b_and(false, true) -> > false; > > b_and(false, false) -> > false. > > > b_or(true, true) -> > true; > > b_or(true, false) -> > true; > > b_or(false, true) -> > true; > > b_or(false, false) -> > false. > > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Mon Jan 26 18:01:42 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 18:01:42 +0100 Subject: [erlang-questions] nand problem In-Reply-To: References: <54C66996.1030109@home.nl> Message-ID: <54C672F6.6020304@home.nl> An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Jan 26 18:11:44 2015 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 26 Jan 2015 17:11:44 +0000 Subject: [erlang-questions] nand problem In-Reply-To: <54C672F6.6020304@home.nl> References: <54C66996.1030109@home.nl> <54C672F6.6020304@home.nl> Message-ID: <54C67550.8000708@llaisdy.com> Yes, you can do: b_nand(X,Y) -> b_not(b_and(X,Y)). Ivan On 26/01/2015 17:01, Roelof Wobben wrote: > Yes. I have learned the wildcardpattern but also there was a text > beneath it saying not to use it for defensive programming. > > oke I have misread it and have to use the not and and. > > Or can i do things like > > b_nand { true, true) -> > b-not(b_and(true, true) > > Roelof > > > > Bob Ippolito schreef op 26-1-2015 om 17:27: >> Here's a hint: >> >> http://en.wikipedia.org/wiki/NAND_gate >> >> You already have an inverter by another name. >> >> Although in this case I think implementing NAND as a pattern match >> would be quite simple. Have you learned about wildcard patterns yet >> (the _ symbol)? >> >> >> On Mon, Jan 26, 2015 at 8:21 AM, Roelof Wobben > > wrote: >> >> Hello, >> >> I now have to make a nand boolean with pattern matching. >> I have found out that nand is true except when its true and true. >> >> as hint I get to implement nand using and and or. >> >> Can someone give me a hint how to do this ? >> >> >> My code looks like this at the moment : >> >> -module(boolean). >> >> -export([b_not/1, b_and/2, b_or/2]). >> >> b_not(true) -> >> false; >> >> b_not(false) -> >> true. >> >> b_and(true, true) -> >> true; >> >> b_and(true, false) -> >> false; >> >> b_and(false, true) -> >> false; >> >> b_and(false, false) -> >> false. >> >> >> b_or(true, true) -> >> true; >> >> b_or(true, false) -> >> true; >> >> b_or(false, true) -> >> true; >> >> b_or(false, false) -> >> false. >> >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Mon Jan 26 18:26:32 2015 From: hugo@REDACTED (Hugo Mills) Date: Mon, 26 Jan 2015 17:26:32 +0000 Subject: [erlang-questions] nand problem In-Reply-To: <54C66996.1030109@home.nl> References: <54C66996.1030109@home.nl> Message-ID: <20150126172632.GH1654@carfax.org.uk> On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: [snip] > b_and(true, true) -> > true; > > b_and(true, false) -> > false; > > b_and(false, true) -> > false; > > b_and(false, false) -> > false. You could make this shorter and possibly easier to read with b_and(true, true) -> true; b_and(_, _) -> false. i.e. define the (one) special case, and then just say that everything else evaluates to false. You can do something similar with b_or. Hugo. -- Hugo Mills | Two things came out of Berkeley in the 1960s: LSD hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. http://carfax.org.uk/ | PGP: 65E74AC0 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From r.wobben@REDACTED Mon Jan 26 18:30:26 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 18:30:26 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <54C67550.8000708@llaisdy.com> References: <54C66996.1030109@home.nl> <54C672F6.6020304@home.nl> <54C67550.8000708@llaisdy.com> Message-ID: <54C679B2.2070809@home.nl> An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Mon Jan 26 18:36:57 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 18:36:57 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <20150126172632.GH1654@carfax.org.uk> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> Message-ID: <54C67B39.30702@home.nl> Hugo Mills schreef op 26-1-2015 om 18:26: > On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: > [snip] >> b_and(true, true) -> >> true; >> >> b_and(true, false) -> >> false; >> >> b_and(false, true) -> >> false; >> >> b_and(false, false) -> >> false. > You could make this shorter and possibly easier to read with > > b_and(true, true) -> > true; > b_and(_, _) -> > false. > > i.e. define the (one) special case, and then just say that > everything else evaluates to false. You can do something similar with > b_or. > > Hugo. > Yes, I could do that . I thought I have read somewhere that using _ for defensive programming was nog good practice but that was on using other on case on the next chapter. I will change my code. Roelof From bryan@REDACTED Mon Jan 26 18:39:30 2015 From: bryan@REDACTED (Bryan) Date: Mon, 26 Jan 2015 09:39:30 -0800 Subject: [erlang-questions] Cast versus Call and timeouts Message-ID: Hi Everyone, I was hoping to better understand an interesting condition I recently encountered, and was able to alleviate though I am not 100% clear why. In our system, we have two types of main processes: for simplicity sake, lets just call them groups (A) and endpoints (B). Each of the processes are implemented as gen_servers. Process A implements functionality and represents a group of endpoints. These endpoints are then each instantiated as a process B. Each endpoint can then be in multiple groups. If I have two groups, then I will have two A processes. If I have five endpoints, I will then have five B processes. In our example, endpoint process #3 is a member in groups one and two. The system is very simple. If a change occurs in A, a message is then sent to each endpoint process B that is a member. In our example, group #1 process would send a message to five endpoint processes. If a change occurs in the endpoint process B, a message is sent to each group process A it is a member of. In our example, if this is endpoint #3, it sends a message to both group one and two. Seems simple enough. My interesting condition that I ran into was where one of the messages from the group process A to the endpoint process B was a cast. All others for both gen_servers are calls. When A sent the cast message to B, B simply updates its state. For reasons that are not clear to me, this ultimately reaches a timeout state, where all the processes start timing out, even though there are no calling/casting cycles. I know that calling cycles introduce a deadlock condition, but I trying to understand why a cast, which is suppose to return immediately and be handled asynchronously would produce a timeout? When I move this message from a cast to a call, the system works perfectly. Thanks in advance for any help. Bryan ---- Bryan Hughes Go Factory http://www.go-factory.net Connecting the Internet of Things -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Mon Jan 26 18:40:56 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 18:40:56 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <54C67B39.30702@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C67B39.30702@home.nl> Message-ID: <54C67C28.5040206@home.nl> Roelof Wobben schreef op 26-1-2015 om 18:36: > Hugo Mills schreef op 26-1-2015 om 18:26: >> On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: >> [snip] >>> b_and(true, true) -> >>> true; >>> >>> b_and(true, false) -> >>> false; >>> >>> b_and(false, true) -> >>> false; >>> >>> b_and(false, false) -> >>> false. >> You could make this shorter and possibly easier to read with >> >> b_and(true, true) -> >> true; >> b_and(_, _) -> >> false. >> >> i.e. define the (one) special case, and then just say that >> everything else evaluates to false. You can do something similar with >> b_or. >> >> Hugo. >> > > Yes, I could do that . > I thought I have read somewhere that using _ for defensive programming > was nog good practice > but that was on using other on case on the next chapter. > > I will change my code. > > Roelof > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > This one better : -module(boolean). -export([b_not/1, b_and/2, b_or/2, b_nand/2]). b_not(true) -> false; b_not(_) -> true. b_and(true, true) -> true; b_and(_, _) -> false. b_or(true, _) -> true; b_or(_, true) -> true; b_or(false, false) -> false. b_nand(true,true) -> b_not(b_and(true, true)); b_nand(_, _) -> b_not(b_and(true, false)); Roelof From hugo@REDACTED Mon Jan 26 18:46:57 2015 From: hugo@REDACTED (Hugo Mills) Date: Mon, 26 Jan 2015 17:46:57 +0000 Subject: [erlang-questions] nand problem In-Reply-To: <54C67B39.30702@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C67B39.30702@home.nl> Message-ID: <20150126174657.GI1654@carfax.org.uk> On Mon, Jan 26, 2015 at 06:36:57PM +0100, Roelof Wobben wrote: > Hugo Mills schreef op 26-1-2015 om 18:26: > >On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: > >[snip] > >>b_and(true, true) -> > >> true; > >> > >>b_and(true, false) -> > >> false; > >> > >>b_and(false, true) -> > >> false; > >> > >>b_and(false, false) -> > >> false. > > You could make this shorter and possibly easier to read with > > > >b_and(true, true) -> > > true; > >b_and(_, _) -> > > false. > > > > i.e. define the (one) special case, and then just say that > >everything else evaluates to false. You can do something similar with > >b_or. > > > > Hugo. > > > > Yes, I could do that . > I thought I have read somewhere that using _ for defensive > programming was nog good practice > but that was on using other on case on the next chapter. I'd argue that this isn't defensive programming at all. It's just stating the behaviour of your function in the most concise form. "If both parameters are true, return true, otherwise return false." Hugo. > I will change my code. > > Roelof > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | Two things came out of Berkeley in the 1960s: LSD hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. http://carfax.org.uk/ | PGP: 65E74AC0 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From hugo@REDACTED Mon Jan 26 18:50:05 2015 From: hugo@REDACTED (Hugo Mills) Date: Mon, 26 Jan 2015 17:50:05 +0000 Subject: [erlang-questions] nand problem In-Reply-To: <54C67C28.5040206@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C67B39.30702@home.nl> <54C67C28.5040206@home.nl> Message-ID: <20150126175005.GJ1654@carfax.org.uk> On Mon, Jan 26, 2015 at 06:40:56PM +0100, Roelof Wobben wrote: > Roelof Wobben schreef op 26-1-2015 om 18:36: > >Hugo Mills schreef op 26-1-2015 om 18:26: > >>On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: > >>[snip] > >>>b_and(true, true) -> > >>> true; > >>> > >>>b_and(true, false) -> > >>> false; > >>> > >>>b_and(false, true) -> > >>> false; > >>> > >>>b_and(false, false) -> > >>> false. > >> You could make this shorter and possibly easier to read with > >> > >>b_and(true, true) -> > >> true; > >>b_and(_, _) -> > >> false. > >> > >> i.e. define the (one) special case, and then just say that > >>everything else evaluates to false. You can do something similar with > >>b_or. > >> > >> Hugo. > >> > > > >Yes, I could do that . > >I thought I have read somewhere that using _ for defensive > >programming was nog good practice > >but that was on using other on case on the next chapter. > > > >I will change my code. > > > >Roelof > > > > > > > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > > > This one better : > > -module(boolean). > > -export([b_not/1, b_and/2, b_or/2, b_nand/2]). > > b_not(true) -> > false; > > b_not(_) -> > true. > > b_and(true, true) -> > true; > > b_and(_, _) -> > false. > > > b_or(true, _) -> > true; > > b_or(_, true) -> > true; > > b_or(false, false) -> > false. Or just: b_or(false, false) -> false; b_or(_, _) -> true. > b_nand(true,true) -> > b_not(b_and(true, true)); > > b_nand(_, _) -> > b_not(b_and(true, false)); You don't need different clauses here. NAND is defined as the NOT of the AND of its two arguments. Or, in other words: b_nand(X, Y) -> b_not(b_and(X, Y)). just as Ivan suggested. Hugo. -- Hugo Mills | Two things came out of Berkeley in the 1960s: LSD hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. http://carfax.org.uk/ | PGP: 65E74AC0 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From r.wobben@REDACTED Mon Jan 26 18:59:51 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 18:59:51 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <20150126175005.GJ1654@carfax.org.uk> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C67B39.30702@home.nl> <54C67C28.5040206@home.nl> <20150126175005.GJ1654@carfax.org.uk> Message-ID: <54C68097.9020807@home.nl> Hugo Mills schreef op 26-1-2015 om 18:50: > On Mon, Jan 26, 2015 at 06:40:56PM +0100, Roelof Wobben wrote: >> Roelof Wobben schreef op 26-1-2015 om 18:36: >>> Hugo Mills schreef op 26-1-2015 om 18:26: >>>> On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: >>>> [snip] >>>>> b_and(true, true) -> >>>>> true; >>>>> >>>>> b_and(true, false) -> >>>>> false; >>>>> >>>>> b_and(false, true) -> >>>>> false; >>>>> >>>>> b_and(false, false) -> >>>>> false. >>>> You could make this shorter and possibly easier to read with >>>> >>>> b_and(true, true) -> >>>> true; >>>> b_and(_, _) -> >>>> false. >>>> >>>> i.e. define the (one) special case, and then just say that >>>> everything else evaluates to false. You can do something similar with >>>> b_or. >>>> >>>> Hugo. >>>> >>> Yes, I could do that . >>> I thought I have read somewhere that using _ for defensive >>> programming was nog good practice >>> but that was on using other on case on the next chapter. >>> >>> I will change my code. >>> >>> Roelof >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> This one better : >> >> -module(boolean). >> >> -export([b_not/1, b_and/2, b_or/2, b_nand/2]). >> >> b_not(true) -> >> false; >> >> b_not(_) -> >> true. >> >> b_and(true, true) -> >> true; >> >> b_and(_, _) -> >> false. >> >> >> b_or(true, _) -> >> true; >> >> b_or(_, true) -> >> true; >> >> b_or(false, false) -> >> false. > Or just: > > b_or(false, false) -> > false; > b_or(_, _) -> > true. > >> b_nand(true,true) -> >> b_not(b_and(true, true)); >> >> b_nand(_, _) -> >> b_not(b_and(true, false)); > You don't need different clauses here. NAND is defined as the NOT > of the AND of its two arguments. Or, in other words: > > b_nand(X, Y) -> > b_not(b_and(X, Y)). > > just as Ivan suggested. > > Hugo. Oke, I have read that nand gives true only when it true true otherwise it's always false. But it seems that I have misread it. Again thanks for the help. Roelof From imantc@REDACTED Mon Jan 26 19:46:01 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 26 Jan 2015 19:46:01 +0100 Subject: [erlang-questions] Cast versus Call and timeouts In-Reply-To: References: Message-ID: it's a mystery alright. wouldn't gen_event be better suited for group notifications? From jobs@REDACTED Mon Jan 26 17:36:45 2015 From: jobs@REDACTED (JV) Date: Mon, 26 Jan 2015 16:36:45 +0000 Subject: [erlang-questions] Erlang Developer Opportunity (competitive salary with stock options) Message-ID: We are looking for talented Erlang developers to join our team in our vibrant riverside workspace. Competitive salary with stock options offered. Who we are and what are we doing? This is a great opportunity to work within a London based FinTech company looking to innovate within the financial services sector and offer a smart company expense solution. We have an ethical approach, we don?t make money off of people needing money or have anything to do with loans (like the companies you read about in the paper). Put simply we save companies time and money while helping them stay on budget and keep control on outgoings. This is a huge sector primarily run by dinosaurs using outdated approaches ? it?s in massive need of change. We?re tackling the problem head on by updating and improving the process using the latest technology and sprinkling of creativity. We?re creating a service to revolutionise our target market ? providing a simple, intuitive service with unique functionality. We have numerous partnerships with financial product and service companies. Many of which you will know. Typical Tasks: * You will work alongside the existing team to help build and improve our core transaction and backend systems. * You will have the opportunity to work with a talented team including an Erlang veteran with 18 years experience. The Ideal Candidate: * Is proficient in Erlang and knows how to write reliable code with automated tests using eunit and quickcheck. * Helpful but not essential: Databases (PostgreSQL), IOS programming using Swift and Objective C, HTML and JavaScript. * Is security conscious. * Appreciates modern agile development. * Someone who is willing to take direction but not be afraid to make suggestions. * Organised, can meet targets and communicate well. * Creative and willing to learn. * Driven, reliable, works well with others and alone. * Can travel to Central London area. Opportunity: To be a part of an core team with generous stock options offering both a fulfilling role and potential for large financial reward as the company grows. We have a big vision and ambitions for our company and team. Finally, we?re a friendly, easy going company who look to create a challenging but enjoyable and rewarding work environment. If you think you?ll be a good addition to our team and want to explore this opportunity then please send CV?s and covering letter to jobs@REDACTED more info at www.expend.io/jobs -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Mon Jan 26 20:09:36 2015 From: bryan@REDACTED (Bryan) Date: Mon, 26 Jan 2015 11:09:36 -0800 Subject: [erlang-questions] Cast versus Call and timeouts In-Reply-To: References: Message-ID: <9DE6D153-3A10-4522-859B-B61FE1A3535C@go-factory.net> > > On Jan 26, 2015, at 10:46 AM, Imants Cekusins wrote: > > it's a mystery alright. > > wouldn't gen_event be better suited for group notifications? I have greatly simplified our system to describe the problem - our ?group? process does much more than just notifications. We require gen_server processes to handle complex networking messaging and routing, something the gen_event manager would not be a good choice for TCP requests. Also, our system is not a pub/sub solution. There are usually only tens to a few hundred processes that are grouped and communicating with each other. Each type of process is very long lived, and the messages can also be heavy. An event manager, even for notification forwarding between processes, would introduce a level of complexity that should not be needed. Thanks! ---- Bryan Hughes Go Factory http://www.go-factory.net Connecting the Internet of Things From bob@REDACTED Mon Jan 26 20:18:11 2015 From: bob@REDACTED (Bob Ippolito) Date: Mon, 26 Jan 2015 11:18:11 -0800 Subject: [erlang-questions] Cast versus Call and timeouts In-Reply-To: References: Message-ID: On Mon, Jan 26, 2015 at 9:39 AM, Bryan wrote: > Hi Everyone, > > I was hoping to better understand an interesting condition I recently > encountered, and was able to alleviate though I am not 100% clear why. > > In our system, we have two types of main processes: for simplicity sake, > lets just call them groups (A) and endpoints (B). Each of the processes are > implemented as gen_servers. > > Process A implements functionality and represents a group of endpoints. > These endpoints are then each instantiated as a process B. Each endpoint > can then be in multiple groups. If I have two groups, then I will have two > A processes. If I have five endpoints, I will then have five B processes. > In our example, endpoint process #3 is a member in groups one and two. > > The system is very simple. If a change occurs in A, a message is then sent > to each endpoint process B that is a member. In our example, group #1 > process would send a message to five endpoint processes. If a change occurs > in the endpoint process B, a message is sent to each group process A it is > a member of. In our example, if this is endpoint #3, it sends a message to > both group one and two. > > Seems simple enough. My interesting condition that I ran into was where > one of the messages from the group process A to the endpoint process B was > a cast. All others for both gen_servers are calls. When A sent the cast > message to B, B simply updates its state. For reasons that are not clear to > me, this ultimately reaches a timeout state, where all the processes start > timing out, even though there are no calling/casting cycles. > > I know that calling cycles introduce a deadlock condition, but I trying to > understand why a cast, which is suppose to return immediately and be > handled asynchronously would produce a timeout? > > When I move this message from a cast to a call, the system works perfectly. > Just a guess, but I would check to make sure that the code for handle_cast in the recipient "B" process wasn't doing something to make it unresponsive, such that the next call to that process would timeout. Are you sure that there was no call as a result of that handle_cast? -------------- next part -------------- An HTML attachment was scrubbed... URL: From imantc@REDACTED Mon Jan 26 20:22:31 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 26 Jan 2015 20:22:31 +0100 Subject: [erlang-questions] Cast versus Call and timeouts In-Reply-To: <9DE6D153-3A10-4522-859B-B61FE1A3535C@go-factory.net> References: <9DE6D153-3A10-4522-859B-B61FE1A3535C@go-factory.net> Message-ID: do the processes P* timing out call the C1 gen_server that was just sent a cast? is it possible that C1 takes a while to update its state? did you try to fwd the "update state" process (via cast) to another gen_server on receipt of the cast? - as a test, to verify? From bryan@REDACTED Mon Jan 26 20:38:24 2015 From: bryan@REDACTED (Bryan) Date: Mon, 26 Jan 2015 11:38:24 -0800 Subject: [erlang-questions] Cast versus Call and timeouts In-Reply-To: References: Message-ID: > On Jan 26, 2015, at 11:18 AM, Bob Ippolito wrote: > > On Mon, Jan 26, 2015 at 9:39 AM, Bryan > wrote: > Hi Everyone, > > I was hoping to better understand an interesting condition I recently encountered, and was able to alleviate though I am not 100% clear why. > > In our system, we have two types of main processes: for simplicity sake, lets just call them groups (A) and endpoints (B). Each of the processes are implemented as gen_servers. > > Process A implements functionality and represents a group of endpoints. These endpoints are then each instantiated as a process B. Each endpoint can then be in multiple groups. If I have two groups, then I will have two A processes. If I have five endpoints, I will then have five B processes. In our example, endpoint process #3 is a member in groups one and two. > > The system is very simple. If a change occurs in A, a message is then sent to each endpoint process B that is a member. In our example, group #1 process would send a message to five endpoint processes. If a change occurs in the endpoint process B, a message is sent to each group process A it is a member of. In our example, if this is endpoint #3, it sends a message to both group one and two. > > Seems simple enough. My interesting condition that I ran into was where one of the messages from the group process A to the endpoint process B was a cast. All others for both gen_servers are calls. When A sent the cast message to B, B simply updates its state. For reasons that are not clear to me, this ultimately reaches a timeout state, where all the processes start timing out, even though there are no calling/casting cycles. > > I know that calling cycles introduce a deadlock condition, but I trying to understand why a cast, which is suppose to return immediately and be handled asynchronously would produce a timeout? > > When I move this message from a cast to a call, the system works perfectly. > > Just a guess, but I would check to make sure that the code for handle_cast in the recipient "B" process wasn't doing something to make it unresponsive, such that the next call to that process would timeout. Are you sure that there was no call as a result of that handle_cast? > The handle_cast simply updates the state of the recipient process. It adds an element to a very small list of fewer than 10 entries kept in the state record of the recipient process, then returns immediately. There are no other function calls. That is what is curious about this. I did more investigation, tracing the function calls looking for any cycles anywhere in the stack. I found something interesting, which I still find a bit confusing. The call that is timing out is a simple handle_call that I describe above. There is another message that was a handle_cast that when I switched it to a handle_call, the system works fine - no timeouts. What I discovered in the function body of the handle_cast (which I switched to handle_call) is that there is a cycle - the recipient process then does a handle_call back to the calling process to notify it of the change that introduced. This is definitely broken. What I am now more confused by is why it is working at all. In more detail: process A function_a() does a handle_call to process B to simply update its state. Very simple and light call. process A function_b() does a handle_cast to process B to handle a more complex message, updates it state, but then accidentally does a handle_call back to Process A (the cycle). This should not work under any circumstance, but when I switch handle_cast to handle_call in function_b(), there are no timeouts. Thanks for input! Much appreciated. Cheers, Bryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Jan 26 22:35:43 2015 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 26 Jan 2015 22:35:43 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <20150126172632.GH1654@carfax.org.uk> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> Message-ID: On Mon, Jan 26, 2015 at 6:26 PM, Hugo Mills wrote: > On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: > [snip] >> b_and(true, true) -> >> true; >> >> b_and(true, false) -> >> false; >> >> b_and(false, true) -> >> false; >> >> b_and(false, false) -> >> false. > > You could make this shorter and possibly easier to read with > > b_and(true, true) -> > true; > b_and(_, _) -> > false. At which point I feel compelled to ask do you really really really want b_and(1, any_old_stupid_argument) to evaluate to false? by explicitly requiring the arguments of b_and to be true or false *and nothing else* you ensure that a program calling b_and with stupid arguments will crash *as soon as possible* which will make debugging easier. It also enables a whole bundle of compiler optimisations - making arguments more general than they need to be makes analysis more difficult and compiled code more general. The more you tell the compiler, then better code it can produce. Yes the program is shorter (good) - but it's not totally correct (bad). This is not important for small programs, but for huge programs making every function totally correct is essential - correct well typed code can be written stuck in a library and forgotten forever - code that does silly things with silly arguments is not good programming practise. b_not should be defined over boolean arguments - any non boolean argument should raise an exception *immediately* anything else violates the principle of least astonishment. Cheers /Joe > i.e. define the (one) special case, and then just say that > everything else evaluates to false. You can do something similar with > b_or. > > Hugo. > > -- > Hugo Mills | Two things came out of Berkeley in the 1960s: LSD > hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. > http://carfax.org.uk/ | > PGP: 65E74AC0 | > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From r.wobben@REDACTED Mon Jan 26 22:53:09 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Mon, 26 Jan 2015 22:53:09 +0100 Subject: [erlang-questions] nand problem In-Reply-To: References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> Message-ID: <54C6B745.3000704@home.nl> Joe Armstrong schreef op 26-1-2015 om 22:35: > On Mon, Jan 26, 2015 at 6:26 PM, Hugo Mills wrote: >> On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: >> [snip] >>> b_and(true, true) -> >>> true; >>> >>> b_and(true, false) -> >>> false; >>> >>> b_and(false, true) -> >>> false; >>> >>> b_and(false, false) -> >>> false. >> You could make this shorter and possibly easier to read with >> >> b_and(true, true) -> >> true; >> b_and(_, _) -> >> false. > At which point I feel compelled to ask > do you really really really want > > b_and(1, any_old_stupid_argument) to evaluate to false? > > by explicitly requiring the arguments of b_and to be true or false > *and nothing else* you ensure that a program calling b_and with stupid > arguments will crash *as soon as possible* which will make debugging > easier. It also enables a whole bundle of compiler optimisations - > making arguments more general than they need to be makes analysis more > difficult > and compiled code more general. The more you tell the compiler, then > better code it can produce. > > Yes the program is shorter (good) - but it's not totally correct (bad). > > This is not important for small programs, but for huge programs making every > function totally correct is essential - correct well typed code can be written > stuck in a library and forgotten forever - code that does silly things > with silly > arguments is not good programming practise. > > b_not should be defined over boolean arguments - any non boolean > argument should raise an exception *immediately* anything else > violates > the principle of least astonishment. > > Cheers > > /Joe > > > > > > >> i.e. define the (one) special case, and then just say that >> everything else evaluates to false. You can do something similar with >> b_or. >> >> Hugo. >> >> -- >> Hugo Mills | Two things came out of Berkeley in the 1960s: LSD >> hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. >> http://carfax.org.uk/ | >> PGP: 65E74AC0 | >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> Oke, So for you the long version is well. Write every case so when someone uses something else then true or false will raise a error. Or do I misunderstood you ? Roelof From imantc@REDACTED Mon Jan 26 23:09:51 2015 From: imantc@REDACTED (Imants Cekusins) Date: Mon, 26 Jan 2015 23:09:51 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <54C6B745.3000704@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C6B745.3000704@home.nl> Message-ID: or use guards: b_and(A,B) when is_boolean(A), is_boolean(B)-> A and B. guards would cause a crash if non-booleans are passed. From e@REDACTED Tue Jan 27 02:15:53 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 02:15:53 +0100 Subject: [erlang-questions] relx extended deps Message-ID: <54C6E6C9.2050506@bestmx.net> Hi, all. in order to solve the ranch's problem with ssl during shutdown (https://github.com/ninenines/ranch/issues/90) I have created the following fix to relx (do not laugh, it is strongly related to the issue) http://file.bestmx.net/ee/software/relx-extended-deps/ it is a proof-of-concept module, which solves the problem in isolation of relx. INPUT IS: a concatenation of multiple .app.src files (that represent a hypothetical release with its deps) OUTPUT IS: a linear sequence of applications that satisfies each and every individual dep described in the given input. (OR crash if the given deps contain a loop.) this tiny module assumes one elegant extension to the .app.src syntax: the option 'sequence' adding this option to _your_ .app.src you can instruct relx to arrange some of your deps to an arbitrary position in the startup sequence (anywhere between deps of deps of deps). for example: {application, my_app, [ {applications, [ kernel, stdlib, cowboy, ssl ]}, {sequence, [[ssl,ranch]]} ]}. instructs my dependency resolver to start ssl BEFORE ranch, regardless any other circumstances. {'sequence', Sequences} Sequences = [ Seq ] Seq = [ atom() ] A Seq is a requirement to run some (two or more) application in the given order. You may specify multiple independent Seqs. From the relx's perspective sequence is not a dep, it actually reads: IF the app A will be started as a dep of any other app in our deps-tree THEN it should be started before the app B. Thus, if relx developers accept this fix, then you shall add: "{sequence, [[ssl,ranch]]}" to the ranch's .app file and the proper sequence will be ensured, BUT ranch will not have a redundant dependency. From desired.mta@REDACTED Tue Jan 27 09:56:47 2015 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 27 Jan 2015 09:56:47 +0100 Subject: [erlang-questions] handling bugs in the program (was: nand problem) Message-ID: On Mon, Jan 26, 2015 at 10:35 PM, Joe Armstrong wrote: > by explicitly requiring the arguments of b_and to be true or false > *and nothing else* you ensure that a program calling b_and with stupid > arguments will crash *as soon as possible* which will make debugging > easier. It also enables a whole bundle of compiler optimisations - > making arguments more general than they need to be makes analysis more > difficult > and compiled code more general. The more you tell the compiler, then > better code it can produce. > > Yes the program is shorter (good) - but it's not totally correct (bad). > > This is not important for small programs, but for huge programs making every > function totally correct is essential - correct well typed code can be written > stuck in a library and forgotten forever - code that does silly things > with silly > arguments is not good programming practise. > > b_not should be defined over boolean arguments - any non boolean > argument should raise an exception *immediately* anything else > violates > the principle of least astonishment. Hi Joe, does this also apply to messages which receiver is not expecting to receive? If an unexpected message is received by a server, should it crash? In other words, should this[1] be exit(badarg) instead? Motiejus [1]: https://github.com/joearms/elib1/blob/a4e794d506ba39be14fef7ada10826c2c1b0f947/src/gen_component.erl#L88 From erlang@REDACTED Tue Jan 27 10:09:28 2015 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 27 Jan 2015 10:09:28 +0100 Subject: [erlang-questions] nand problem In-Reply-To: <54C6B745.3000704@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C6B745.3000704@home.nl> Message-ID: On Mon, Jan 26, 2015 at 10:53 PM, Roelof Wobben wrote: > Joe Armstrong schreef op 26-1-2015 om 22:35: > >> On Mon, Jan 26, 2015 at 6:26 PM, Hugo Mills wrote: >>> >>> On Mon, Jan 26, 2015 at 05:21:42PM +0100, Roelof Wobben wrote: >>> [snip] >>>> >>>> b_and(true, true) -> >>>> true; >>>> >>>> b_and(true, false) -> >>>> false; >>>> >>>> b_and(false, true) -> >>>> false; >>>> >>>> b_and(false, false) -> >>>> false. >>> >>> You could make this shorter and possibly easier to read with >>> >>> b_and(true, true) -> >>> true; >>> b_and(_, _) -> >>> false. >> >> At which point I feel compelled to ask >> do you really really really want >> >> b_and(1, any_old_stupid_argument) to evaluate to false? >> >> by explicitly requiring the arguments of b_and to be true or false >> *and nothing else* you ensure that a program calling b_and with stupid >> arguments will crash *as soon as possible* which will make debugging >> easier. It also enables a whole bundle of compiler optimisations - >> making arguments more general than they need to be makes analysis more >> difficult >> and compiled code more general. The more you tell the compiler, then >> better code it can produce. >> >> Yes the program is shorter (good) - but it's not totally correct (bad). >> >> This is not important for small programs, but for huge programs making >> every >> function totally correct is essential - correct well typed code can be >> written >> stuck in a library and forgotten forever - code that does silly things >> with silly >> arguments is not good programming practise. >> >> b_not should be defined over boolean arguments - any non boolean >> argument should raise an exception *immediately* anything else >> violates >> the principle of least astonishment. >> >> Cheers >> >> /Joe >> >> >> >> >> >> >>> i.e. define the (one) special case, and then just say that >>> everything else evaluates to false. You can do something similar with >>> b_or. >>> >>> Hugo. >>> >>> -- >>> Hugo Mills | Two things came out of Berkeley in the 1960s: >>> LSD >>> hugo@REDACTED carfax.org.uk | and Unix. This is not a coincidence. >>> http://carfax.org.uk/ | >>> PGP: 65E74AC0 | >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > > Oke, So for you the long version is well. Write every case so when someone > uses something else then true or false will raise a error. > Or do I misunderstood you ? That's write - correctness is preferable to brevity /Joe > > Roelof > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Tue Jan 27 10:14:06 2015 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 27 Jan 2015 10:14:06 +0100 Subject: [erlang-questions] handling bugs in the program (was: nand problem) In-Reply-To: References: Message-ID: On Tue, Jan 27, 2015 at 9:56 AM, Motiejus Jak?tys wrote: > On Mon, Jan 26, 2015 at 10:35 PM, Joe Armstrong wrote: >> by explicitly requiring the arguments of b_and to be true or false >> *and nothing else* you ensure that a program calling b_and with stupid >> arguments will crash *as soon as possible* which will make debugging >> easier. It also enables a whole bundle of compiler optimisations - >> making arguments more general than they need to be makes analysis more >> difficult >> and compiled code more general. The more you tell the compiler, then >> better code it can produce. >> >> Yes the program is shorter (good) - but it's not totally correct (bad). >> >> This is not important for small programs, but for huge programs making every >> function totally correct is essential - correct well typed code can be written >> stuck in a library and forgotten forever - code that does silly things >> with silly >> arguments is not good programming practise. >> >> b_not should be defined over boolean arguments - any non boolean >> argument should raise an exception *immediately* anything else >> violates >> the principle of least astonishment. > > Hi Joe, > > does this also apply to messages which receiver is not expecting to > receive? If an unexpected message is received by a server, should it > crash? > > In other words, should this[1] be exit(badarg) instead? No necessarily - it depends upon the protocol. You'd have to distinguish out-of-order messages from incorrect messages. I guess incorrect messages should crash both the sender and the receiver. Out of order messages may or may not be a problem depending on the logic of the program so there is no automatic "right thing to do" /Joe > > Motiejus > > > [1]: https://github.com/joearms/elib1/blob/a4e794d506ba39be14fef7ada10826c2c1b0f947/src/gen_component.erl#L88 From schneider@REDACTED Tue Jan 27 12:50:18 2015 From: schneider@REDACTED (Frans Schneider) Date: Tue, 27 Jan 2015 12:50:18 +0100 Subject: [erlang-questions] Implement parallel sessions Message-ID: <54C77B7A.4040401@xs4all.nl> Hi list, I am implementing a kind of IM proxy which currently handles one single session for one user. The session is implemented as a stack of processes with from top to bottom the session control process, a stanza encoder / decoder process, an en- / decrypt process and finally a process which handles the tcp connection. Each process is a FSM and holds state such as RC4 encryption state, tcp data etc. The session is controlled by a supervisor which is part of a overall supervisor structure. A session will be open for a long period. To implement for multiple users, I want to run the processes in the session stack for each user separately to keep them as simple as possible and isolate crashes, Each user session should be controlled by a supervisor. What is a common approach to implementing a thing like this, i.e. a system with some interdependent processes in parallel? I was thinking about using the via option for registering processes and passing the session id along when calling processes. Where does one keep the registry in such a scenario? Would some of the supervisors be a proper place? Since I am still learning to do it the Erlang way, any hint is appreciated, Frans From ok@REDACTED Tue Jan 27 13:03:31 2015 From: ok@REDACTED (ok@REDACTED) Date: Wed, 28 Jan 2015 01:03:31 +1300 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <54C4D39F.6000604@home.nl> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> Message-ID: <38d9ce6f94061363a614331e0eb8766a.squirrel@chasm.otago.ac.nz> > As I understand it right. The eror message is on all functions so on the > square, circle and triangle functions. > And if not so, why also not a error functions on the other two. You have ONE area/1 function; the error message applies to it. By the way, you are -import+ing math:sqrt/1. I like that; I appreciate seeing in one place which external functions are used. But the actual point of -import in Erlang is to let you call the imported function *without* a module prefix. That's considered a bad idea, and indeed, you do use a module prefix. But that means the -import directive was pointless. You are calling math:pi() without any -import directive, after all. I know this is a toy exercise, but I do think I should point out for the unwary that >>> area({triangle, A, B, C}) -> >>> S = (A + B + C)/2, >>> math:sqrt(S*(S-A)*(S-B)*(S-C)); is a poor way to calculate the area of a triangle. It's great mathematics, but Kahan ("Dr IEEE floats") has explained that it is very bad *computationally*. Let's see if I can approximate his explanation. Suppose we have a tall skinny triangle, so that A and B are nearly equal, and C is quite small in comparison. Then S is very nearly A (and so very nearly B). This means that S-A involves *catastrophic cancellation*, where the result has a lot fewer significant digits than you expect. For example, suppose we have 7 decimal digits of precision, and A = B = 1000.0, C = 1.0. Then S = 1000.5. A = 1.000000e3 S = 1.000500e3 S-A = #.###500e3 (using # for 0 to highlight lost significance) and the result has 3 digits of precision, not 7. Ah. Here's Kahan's paper: www.cs.berkeley.edu/~wkahan/Triangle.pdf area({triangle, A, B, C}) -> 0.25*math:sqrt((A+(B+C))*(C-(A-B))*(C+(A-B))*(A+(B-C))); is an Erlang version of Kahan's improved formula -- assuming, which I would not advise, that I have not made any mistakes in copying it. From imantc@REDACTED Tue Jan 27 13:14:05 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 13:14:05 +0100 Subject: [erlang-questions] Implement parallel sessions In-Reply-To: <54C77B7A.4040401@xs4all.nl> References: <54C77B7A.4040401@xs4all.nl> Message-ID: supervisors are meant to supervise only afaik, not to store any state or do anything else. From luca@REDACTED Tue Jan 27 13:08:28 2015 From: luca@REDACTED (Luca Tavazzani) Date: Tue, 27 Jan 2015 12:08:28 +0000 Subject: [erlang-questions] New Erlang-related job openings at exciting startup Message-ID: Hi, We at Spatch are creating a new form of communication built on Erlang. A modern, federated messaging platform with mutable, collaborative objects being at the heart of it. Imagine what would happen if you took the best components of email (IMAP/SMTP), chat (IRC/XMPP) and collaboration (Wave)... and created a federated platform and new set of open protocols around it? That?s what we?re building. Currently looking for people who can join the platform team to help us build, scale and maintain the software services which define the core messaging components of Spatch. Senior back-end engineer: http://spatch.co/careers/1 Senior DevOps engineer: http://spatch.co/careers/2 Cheers, Luca Tavazzani Spatch.co -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Tue Jan 27 13:40:11 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 13:40:11 +0100 Subject: [erlang-questions] New Erlang-related job openings at exciting startup In-Reply-To: References: Message-ID: <54C7872B.3070005@bestmx.net> why don't you begin those announcements with the most important filter-property: "to whom it is addressed". you require from your employees to be residents of UK, how many readers of the international mailing-list possess this property now? > We at Spatch are creating a new form of communication > built on Erlang. A modern, federated messaging platform with mutable, > collaborative objects being at the heart of it. Imagine what would happen > if you took the best components of email (IMAP/SMTP), chat (IRC/XMPP) and > collaboration (Wave)... and created a federated platform and new set of > open protocols around it? That?s what we?re building. > > Currently looking for people who can join the platform team to help us > build, scale and maintain the software services which define the core > messaging components of Spatch. > > Senior back-end engineer: > http://spatch.co/careers/1 > > > Senior DevOps engineer: > http://spatch.co/careers/2 > > Cheers, > Luca Tavazzani > Spatch.co > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From davidnwelton@REDACTED Tue Jan 27 14:13:52 2015 From: davidnwelton@REDACTED (David Welton) Date: Tue, 27 Jan 2015 14:13:52 +0100 Subject: [erlang-questions] Application supervisor Message-ID: I hacked this together a while ago: https://github.com/davidw/hardcore The idea is to have something like Fuse ( https://github.com/jlouis/fuse ) but that works at the Erlang application level. It lets you start Erlang applications under a system that gets notified when they crash and can then act on that. It's not in use, so the kinks have not been worked out, but it might be useful for someone. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From r.wobben@REDACTED Tue Jan 27 14:14:20 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 14:14:20 +0100 Subject: [erlang-questions] why do I see a message about area of a square In-Reply-To: <38d9ce6f94061363a614331e0eb8766a.squirrel@chasm.otago.ac.nz> References: <54C4CFAD.2050808@home.nl> <9DD67ADA-356A-473C-AD5F-9F033D5F3A38@uninet.ee> <54C4D39F.6000604@home.nl> <38d9ce6f94061363a614331e0eb8766a.squirrel@chasm.otago.ac.nz> Message-ID: <54C78F2C.4050107@home.nl> ok@REDACTED schreef op 27-1-2015 om 13:03: >> As I understand it right. The eror message is on all functions so on the >> square, circle and triangle functions. >> And if not so, why also not a error functions on the other two. > You have ONE area/1 function; the error message applies to it. > > By the way, you are -import+ing math:sqrt/1. > I like that; I appreciate seeing in one place which external > functions are used. But the actual point of -import in Erlang > is to let you call the imported function *without* a module > prefix. That's considered a bad idea, and indeed, you do use > a module prefix. But that means the -import directive was > pointless. You are calling math:pi() without any -import > directive, after all. > > I know this is a toy exercise, but I do think I should point out > for the unwary that > >>>> area({triangle, A, B, C}) -> >>>> S = (A + B + C)/2, >>>> math:sqrt(S*(S-A)*(S-B)*(S-C)); > is a poor way to calculate the area of a triangle. > It's great mathematics, but Kahan ("Dr IEEE floats") has > explained that it is very bad *computationally*. > Let's see if I can approximate his explanation. > > Suppose we have a tall skinny triangle, so that A and B are nearly > equal, and C is quite small in comparison. Then S is very nearly > A (and so very nearly B). This means that S-A involves > *catastrophic cancellation*, where the result has a lot fewer > significant digits than you expect. > > For example, suppose we have 7 decimal digits of precision, > and A = B = 1000.0, C = 1.0. Then S = 1000.5. > A = 1.000000e3 > S = 1.000500e3 > S-A = #.###500e3 (using # for 0 to highlight lost significance) > and the result has 3 digits of precision, not 7. > > Ah. Here's Kahan's paper: www.cs.berkeley.edu/~wkahan/Triangle.pdf > > area({triangle, A, B, C}) -> > 0.25*math:sqrt((A+(B+C))*(C-(A-B))*(C+(A-B))*(A+(B-C))); > > is an Erlang version of Kahan's improved formula -- assuming, > which I would not advise, that I have not made any mistakes in copying it. > > Thanks for the explanation. Roelof From sean@REDACTED Tue Jan 27 15:10:32 2015 From: sean@REDACTED (Sean Cribbs) Date: Tue, 27 Jan 2015 08:10:32 -0600 Subject: [erlang-questions] Implement parallel sessions In-Reply-To: References: <54C77B7A.4040401@xs4all.nl> Message-ID: For those processes that are per-user, or otherwise spin up and do some work and then go away, we use the simple_one_for_one strategy in the supervisor. In your case, the simple_one_for_one supervisor would start the master process of the session (whether it be a supervisor or something else). If the master process of the user session is a supervisor, you can create whatever state is needed to be shared (an ETS table perhaps, or maybe its own pid) and then pass it to the children when starting. This supervisor would be a one_for_one, one_for_all, or rest_for_one strategy, depending on the dependencies between its children. I wonder whether a single process per session will do (or two, one for user connection and one for the proxy connection), with different functionality being kept simply in multiple modules rather than FSMs. For example, if your connection process handling the TCP socket simply shuttles data to and from, you might consider collapsing it into the process that's next up the stack. On Tue, Jan 27, 2015 at 6:14 AM, Imants Cekusins wrote: > supervisors are meant to supervise only afaik, not to store any state > or do anything else. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sean Cribbs Sr. Software Engineer Basho Technologies, Inc. http://basho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Tue Jan 27 15:15:59 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 15:15:59 +0100 Subject: [erlang-questions] create list quesion Message-ID: <54C79D9F.8080607@home.nl> Hello, I try now to solve a challenge where I must make a list from a range, So for example if I do create(2) the outcome have to be [ 1, 2] So i can do create(0) -> print array but as far as I know the list is then not in the scope so I cannot use it when I do create(2) -> add the 2 to the array. How do I take care that the list is in the scope and still using only create(number) Can anyone give me a tip about it ? Roelof From essen@REDACTED Tue Jan 27 15:17:14 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 27 Jan 2015 15:17:14 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C79D9F.8080607@home.nl> References: <54C79D9F.8080607@home.nl> Message-ID: <54C79DEA.6020407@ninenines.eu> You can make a local function that takes more arguments and call it. On 01/27/2015 03:15 PM, Roelof Wobben wrote: > Hello, > > I try now to solve a challenge where I must make a list from a range, > > So for example if I do create(2) the outcome have to be [ 1, 2] > > So i can do > > create(0) -> > print array > > but as far as I know the list is then not in the scope so I cannot use > it when I do > > create(2) -> > add the 2 to the array. > > How do I take care that the list is in the scope and still using only > create(number) > > Can anyone give me a tip about it ? > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin http://ninenines.eu From t@REDACTED Tue Jan 27 15:33:42 2015 From: t@REDACTED (Tristan Sloughter) Date: Tue, 27 Jan 2015 08:33:42 -0600 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C6E6C9.2050506@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> Message-ID: <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> Sorry, I don't think we'll accept this :). Ranch needs to depend on ssl for this to work. I'll also note that it isn't only relx that reserves the right to reorder your apps, systools does as well. So even if this currently works for you it may not for others and may break in the future. -- Tristan Sloughter t@REDACTED On Mon, Jan 26, 2015, at 07:15 PM, e@REDACTED wrote: > Hi, all. > in order to solve the ranch's problem with ssl during shutdown > (https://github.com/ninenines/ranch/issues/90) > > I have created the following fix to relx > (do not laugh, it is strongly related to the issue) > http://file.bestmx.net/ee/software/relx-extended-deps/ > > it is a proof-of-concept module, which solves the problem in isolation > of relx. > INPUT IS: > a concatenation of multiple .app.src files (that represent a > hypothetical release with its deps) > OUTPUT IS: > a linear sequence of applications that satisfies each and every > individual dep described in the given input. > (OR crash if the given deps contain a loop.) > > this tiny module assumes one elegant extension to the .app.src syntax: > the option 'sequence' > > adding this option to _your_ .app.src you can instruct relx to arrange > some of your deps to an arbitrary position in the startup sequence > (anywhere between deps of deps of deps). > > for example: > {application, my_app, [ > {applications, [ > kernel, > stdlib, > cowboy, > ssl > ]}, > {sequence, [[ssl,ranch]]} > ]}. > > instructs my dependency resolver to start ssl BEFORE ranch, regardless > any other circumstances. > > {'sequence', Sequences} > > Sequences = [ Seq ] > Seq = [ atom() ] > > A Seq is a requirement to run some (two or more) application in the > given order. You may specify multiple independent Seqs. > > > From the relx's perspective sequence is not a dep, it actually reads: > IF the app A will be started as a dep of any other app in our deps-tree > THEN it should be started before the app B. > > Thus, if relx developers accept this fix, then you shall add: > "{sequence, [[ssl,ranch]]}" > to the ranch's .app file and the proper sequence will be ensured, > BUT ranch will not have a redundant dependency. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From e@REDACTED Tue Jan 27 15:35:27 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 15:35:27 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> Message-ID: <54C7A22F.70507@bestmx.net> On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > Sorry, I don't think we'll accept this :). > > Ranch needs to depend on ssl for this to work. > > I'll also note that it isn't only relx that reserves the right to > reorder your apps, systools does as well. So even if this currently > works for you it may not for others and may break in the future. > of course you don't, you didn't even read the proposal From essen@REDACTED Tue Jan 27 15:36:36 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 27 Jan 2015 15:36:36 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> Message-ID: <54C7A274.3010905@ninenines.eu> On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > Sorry, I don't think we'll accept this :). > > Ranch needs to depend on ssl for this to work. This is the plan, but I am wondering if relx has a way to remove apps the same way reltool has? That would be a good enough alternative if I add instructions in the docs on how to remove ssl if you don't need it. -- Lo?c Hoguin http://ninenines.eu From t@REDACTED Tue Jan 27 15:42:15 2015 From: t@REDACTED (Tristan Sloughter) Date: Tue, 27 Jan 2015 08:42:15 -0600 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7A274.3010905@ninenines.eu> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A274.3010905@ninenines.eu> Message-ID: <1422369735.3121218.219509945.4AFCD0F0@webmail.messagingengine.com> It doesn't currently, but I can add a feature like that pretty quickly. Open an issue and I'll get to it this week. -- Tristan Sloughter t@REDACTED On Tue, Jan 27, 2015, at 08:36 AM, Lo?c Hoguin wrote: > On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > > Sorry, I don't think we'll accept this :). > > > > Ranch needs to depend on ssl for this to work. > > This is the plan, but I am wondering if relx has a way to remove apps > the same way reltool has? That would be a good enough alternative if I > add instructions in the docs on how to remove ssl if you don't need it. > > -- > Lo?c Hoguin > http://ninenines.eu From t@REDACTED Tue Jan 27 15:42:34 2015 From: t@REDACTED (Tristan Sloughter) Date: Tue, 27 Jan 2015 08:42:34 -0600 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7A22F.70507@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A22F.70507@bestmx.net> Message-ID: <1422369754.3121297.219511305.67AE0A3A@webmail.messagingengine.com> ? -- Tristan Sloughter t@REDACTED On Tue, Jan 27, 2015, at 08:35 AM, e@REDACTED wrote: > On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > > Sorry, I don't think we'll accept this :). > > > > Ranch needs to depend on ssl for this to work. > > > > I'll also note that it isn't only relx that reserves the right to > > reorder your apps, systools does as well. So even if this currently > > works for you it may not for others and may break in the future. > > > > of course you don't, you didn't even read the proposal > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From harit.subscriptions@REDACTED Tue Jan 27 15:43:44 2015 From: harit.subscriptions@REDACTED (Harit Himanshu) Date: Tue, 27 Jan 2015 06:43:44 -0800 Subject: [erlang-questions] Question on building a Parallel Server Message-ID: Hello As I learn from Programming Erlang, the following is a way to create a parallel server start_parallel_server() -> {ok, Listen} = gen_tcp:listen(...), spawn(fun() -> par_connect(Listen) end). par_connect(Listen) -> {ok, Socket} = gen_tcp:accept(Listen), spawn(fun() -> par_connect(Listen) end), loop(Socket). I wanted to dissect this code to understand what's going on. *Questions* 1. When we call *start_parallel_server() * on a single machine, a separate process is started (which is controlling process?) executing *par_connect*, correct? 2. When *gen_tcp:accept* accepts a new connection, immediately a new process is created executing *par_connect(Listen)*. This makes sure that we can accept more requests from the client. Now who is the controlling process for this new process? is the same as Listen? Thanks + Harit Himanshu -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Tue Jan 27 15:48:45 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 15:48:45 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7A274.3010905@ninenines.eu> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A274.3010905@ninenines.eu> Message-ID: <54C7A54D.2080201@bestmx.net> On 01/27/2015 03:36 PM, Lo?c Hoguin wrote: > On 01/27/2015 03:33 PM, Tristan Sloughter wrote: >> Sorry, I don't think we'll accept this :). >> >> Ranch needs to depend on ssl for this to work. > > This is the plan, but I am wondering if relx has a way to remove apps > the same way reltool has? That would be a good enough alternative if I > add instructions in the docs on how to remove ssl if you don't need it. you may add also a notion of "conditional dep" say: "ranch depends on ssl if ssl is a dep of an app that depends on ranch" it is much narrower that my proposal, but it should work perfectly for all similar cases, and it is much easier to implement outside relx. in the context of the initial issue it would look like the following: you specify 'ssl' separately from usual deps, and (during make, or systool) in case 'ssl' is found somewhere in 'applications' you then rewrite .app file adding 'ssl' the 'applications' of 'ranch'. effectively it is like pulling deps deeper into the deps-tree. very smooth move, and very versatile From r.wobben@REDACTED Tue Jan 27 15:53:07 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 15:53:07 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C79DEA.6020407@ninenines.eu> References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> Message-ID: <54C7A653.4080802@home.nl> Hello, Oke, Something like this in pseudo code : create(0) -> print empty array create(x) -> create_array(x, []) ) create_array(0, array) -> print array create_array(x, array) -> add the first item to the array and run create_array again Roelof Lo?c Hoguin schreef op 27-1-2015 om 15:17: > You can make a local function that takes more arguments and call it. > > On 01/27/2015 03:15 PM, Roelof Wobben wrote: >> Hello, >> >> I try now to solve a challenge where I must make a list from a range, >> >> So for example if I do create(2) the outcome have to be [ 1, 2] >> >> So i can do >> >> create(0) -> >> print array >> >> but as far as I know the list is then not in the scope so I cannot use >> it when I do >> >> create(2) -> >> add the 2 to the array. >> >> How do I take care that the list is in the scope and still using only >> create(number) >> >> Can anyone give me a tip about it ? >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From e@REDACTED Tue Jan 27 15:52:25 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 15:52:25 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> Message-ID: <54C7A629.80508@bestmx.net> On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > Sorry, I don't think we'll accept this :). > > Ranch needs to depend on ssl for this to work. > > I'll also note that it isn't only relx that reserves the right to > reorder your apps, systools does as well. So even if this currently > works for you it may not for others and may break in the future. > what makes you assume so rudely that the issue is unique for 'ranch' From n.oxyde@REDACTED Tue Jan 27 16:17:16 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:17:16 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7A629.80508@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> Message-ID: <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> Le 27 janv. 2015 ? 15:52, e@REDACTED a ?crit : > what makes you assume so rudely that the issue is unique for 'ranch' What makes you feel that his response is rude? That many applications do the dubious thing of not properly specifying their dependencies is a problem in these applications, not in relx. From t@REDACTED Tue Jan 27 16:20:12 2015 From: t@REDACTED (Tristan Sloughter) Date: Tue, 27 Jan 2015 09:20:12 -0600 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7A629.80508@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> Message-ID: <1422372012.3139042.219527373.6B6FD75E@webmail.messagingengine.com> Sorry if I came off rude. I can understand you feeling that way without context. I have discussed this issue with others for a while, not related to ranch, and we don't want to have a fix that involves the user setting any sort of order, leaving that always up to a topological sort of dependencies. -- Tristan Sloughter t@REDACTED On Tue, Jan 27, 2015, at 08:52 AM, e@REDACTED wrote: > On 01/27/2015 03:33 PM, Tristan Sloughter wrote: > > Sorry, I don't think we'll accept this :). > > > > Ranch needs to depend on ssl for this to work. > > > > I'll also note that it isn't only relx that reserves the right to > > reorder your apps, systools does as well. So even if this currently > > works for you it may not for others and may break in the future. > > > > > what makes you assume so rudely that the issue is unique for 'ranch' > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From e@REDACTED Tue Jan 27 16:23:10 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:23:10 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> Message-ID: <54C7AD5E.7090208@bestmx.net> On 01/27/2015 04:17 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 15:52, e@REDACTED a ?crit : > >> what makes you assume so rudely that the issue is unique for 'ranch' > > What makes you feel that his response is rude? That many applications do the dubious thing of not properly specifying their dependencies is a problem in these applications, not in relx. the problem is that we are facing another type of dependency here, something like a "runtime dependency". by the moment of the development the developer DOES NOT KNOW will his application depend on the 'ssl' or not. therefore he can not specify a dep. however, we know for sure how to determine the very same dep LATER ON. it is clear that 'ranch's dependency on 'ssl' is implied by the deps of a top-application. so that i propose a clear unambiguous algorithmic way how to alter some deps within the deps-tree according to the deps of the top-app. From e@REDACTED Tue Jan 27 16:26:01 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:26:01 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <1422372012.3139042.219527373.6B6FD75E@webmail.messagingengine.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <1422372012.3139042.219527373.6B6FD75E@webmail.messagingengine.com> Message-ID: <54C7AE09.5030003@bestmx.net> > I have discussed this issue with others for a while, not related to > ranch, and we don't want to have a fix that involves the user setting > any sort of order, leaving that always up to a topological sort of > dependencies. this is why i have provided the working code that solves the problem in total. however, i agree to narrow down the problem, and i described that in other messages of the %subj% From n.oxyde@REDACTED Tue Jan 27 16:39:27 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:39:27 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7AD5E.7090208@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> Message-ID: <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> Le 27 janv. 2015 ? 16:23, e@REDACTED a ?crit : > by the moment of the development the developer DOES NOT KNOW will his application depend on the 'ssl' or not. > therefore he can not specify a dep. I disagree, Ranch's code calls into ssl, therefore it depends on ssl. From n.oxyde@REDACTED Tue Jan 27 16:41:03 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:41:03 +0100 Subject: [erlang-questions] nand problem In-Reply-To: References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C6B745.3000704@home.nl> Message-ID: <2AB5C4BF-60EE-428A-9D89-CBFFE1C1310F@gmail.com> Le 26 janv. 2015 ? 23:09, Imants Cekusins a ?crit : > or use guards: > > b_and(A,B) when is_boolean(A), is_boolean(B)-> > A and B. > > guards would cause a crash if non-booleans are passed. No need for guards if you are using the 'and' operator. It will crash too. From e@REDACTED Tue Jan 27 16:41:15 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:41:15 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> Message-ID: <54C7B19B.8000109@bestmx.net> On 01/27/2015 04:39 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:23, e@REDACTED a ?crit : > >> by the moment of the development the developer DOES NOT KNOW will his application depend on the 'ssl' or not. >> therefore he can not specify a dep. > > I disagree, Ranch's code calls into ssl, therefore it depends on ssl. could you please quote the line? > From n.oxyde@REDACTED Tue Jan 27 16:44:21 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:44:21 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7B19B.8000109@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> Message-ID: Le 27 janv. 2015 ? 16:41, e@REDACTED a ?crit : > On 01/27/2015 04:39 PM, Anthony Ramine wrote: >> Le 27 janv. 2015 ? 16:23, e@REDACTED a ?crit : >> >>> by the moment of the development the developer DOES NOT KNOW will his application depend on the 'ssl' or not. >>> therefore he can not specify a dep. >> >> I disagree, Ranch's code calls into ssl, therefore it depends on ssl. > > could you please quote the line? The whole module named ranch_ssl. https://github.com/ninenines/ranch/blob/master/src/ranch_ssl.erl From e@REDACTED Tue Jan 27 16:44:49 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:44:49 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> Message-ID: <54C7B271.6070004@bestmx.net> On 01/27/2015 04:44 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:41, e@REDACTED a ?crit : > >> On 01/27/2015 04:39 PM, Anthony Ramine wrote: >>> Le 27 janv. 2015 ? 16:23, e@REDACTED a ?crit : >>> >>>> by the moment of the development the developer DOES NOT KNOW will his application depend on the 'ssl' or not. >>>> therefore he can not specify a dep. >>> >>> I disagree, Ranch's code calls into ssl, therefore it depends on ssl. >> >> could you please quote the line? > > The whole module named ranch_ssl. > > https://github.com/ninenines/ranch/blob/master/src/ranch_ssl.erl ? how am i running ranch without ssl now? From n.oxyde@REDACTED Tue Jan 27 16:47:32 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:47:32 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7B271.6070004@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> Message-ID: <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : > how am i running ranch without ssl now? By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. From e@REDACTED Tue Jan 27 16:49:23 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:49:23 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> Message-ID: <54C7B383.2030603@bestmx.net> On 01/27/2015 04:47 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : > >> how am i running ranch without ssl now? > > By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. hold on, i am not into political question, i want to keep it technical. i am running ranch without ssl. doesn't it contradict the statement of dependency? From n.oxyde@REDACTED Tue Jan 27 16:53:40 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 16:53:40 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7B383.2030603@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> Message-ID: <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> Le 27 janv. 2015 ? 16:49, e@REDACTED a ?crit : > On 01/27/2015 04:47 PM, Anthony Ramine wrote: >> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : >> >>> how am i running ranch without ssl now? >> >> By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. > > hold on, i am not into political question, i want to keep it technical. > i am running ranch without ssl. > doesn't it contradict the statement of dependency? And anyone actually using ranch with ssl now needs an additional non-standard 'sequence' key in their app file. See the problem? The proper solution is to remove ranch_ssl from ranch and let you use that slimmed down version of it, not to accommodate one developer's laziness in a general purpose release tool. From kyle@REDACTED Tue Jan 27 16:07:42 2015 From: kyle@REDACTED (Kyle Neal) Date: Tue, 27 Jan 2015 10:07:42 -0500 Subject: [erlang-questions] create list quesion In-Reply-To: <54C79D9F.8080607@home.nl> References: <54C79D9F.8080607@home.nl> Message-ID: I think we need a little more clarification... On Tue, Jan 27, 2015 at 9:15 AM, Roelof Wobben wrote: > Hello, > > I try now to solve a challenge where I must make a list from a range, > > So for example if I do create(2) the outcome have to be [ 1, 2] > Here you are saying create(N) should create a list from 1-N. > > So i can do > > create(0) -> > print array > > but as far as I know the list is then not in the scope so I cannot use it > when I do > > create(2) -> > add the 2 to the array. > Here you are saying create(2) should ONLY add 2 to the list, when previously you said create(2) should create a list from 1-2. Now, if you are saying that create(N) should create a list from 1-N OR list:last()-N (meaning it should be in range), then you would first have to check if N is in the current list. If not, create the list, if so cancel EX) (Using my assumption) create(0) -> print array. create(N) -> case lists:member(N, list) of true -> exit; false -> get last element of list (L), append L-N to current list (L can also be nothing, if this is the first call to this function in which case it should append 1-N) end. This should always work because the function is generating a list from A-B. And you can using something like ets with ordered_set or something similar to keep it in order. -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Tue Jan 27 16:58:43 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 16:58:43 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> Message-ID: <54C7B5B3.1000108@bestmx.net> On 01/27/2015 04:53 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:49, e@REDACTED a ?crit : > >> On 01/27/2015 04:47 PM, Anthony Ramine wrote: >>> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : >>> >>>> how am i running ranch without ssl now? >>> >>> By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. >> >> hold on, i am not into political question, i want to keep it technical. >> i am running ranch without ssl. >> doesn't it contradict the statement of dependency? > > And anyone actually using ranch with ssl now needs an additional non-standard 'sequence' key in their app file. on the contrary! it is ranch who needed this additional 'sequence' key. because 'ranch' knows the requirement, while a top-app does not have to know. you may call it a 'conditional-dep' where the condition is triggered by the top-app (without awareness) From n.oxyde@REDACTED Tue Jan 27 17:04:34 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 17:04:34 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7B5B3.1000108@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7B5B3.1000108@bestmx.net> Message-ID: Le 27 janv. 2015 ? 16:58, e@REDACTED a ?crit : > On 01/27/2015 04:53 PM, Anthony Ramine wrote: >> Le 27 janv. 2015 ? 16:49, e@REDACTED a ?crit : >> >>> On 01/27/2015 04:47 PM, Anthony Ramine wrote: >>>> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : >>>> >>>>> how am i running ranch without ssl now? >>>> >>>> By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. >>> >>> hold on, i am not into political question, i want to keep it technical. >>> i am running ranch without ssl. >>> doesn't it contradict the statement of dependency? >> >> And anyone actually using ranch with ssl now needs an additional non-standard 'sequence' key in their app file. > > on the contrary! > it is ranch who needed this additional 'sequence' key. > because 'ranch' knows the requirement, > while a top-app does not have to know. > > you may call it a 'conditional-dep' > where the condition is triggered by the top-app > (without awareness) They need the machinery behind it, and which is unknown to OTP itself, even if they didn't write the actual 'sequence' property themselves. I am against adding conditional dependencies, given they are mostly a way for developers to be lazy when it comes down to splitting their own projects decently. From e@REDACTED Tue Jan 27 17:07:41 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 17:07:41 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7B5B3.1000108@bestmx.net> Message-ID: <54C7B7CD.5090101@bestmx.net> On 01/27/2015 05:04 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:58, e@REDACTED a ?crit : > >> On 01/27/2015 04:53 PM, Anthony Ramine wrote: >>> Le 27 janv. 2015 ? 16:49, e@REDACTED a ?crit : >>> >>>> On 01/27/2015 04:47 PM, Anthony Ramine wrote: >>>>> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : >>>>> >>>>>> how am i running ranch without ssl now? >>>>> >>>>> By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. >>>> >>>> hold on, i am not into political question, i want to keep it technical. >>>> i am running ranch without ssl. >>>> doesn't it contradict the statement of dependency? >>> >>> And anyone actually using ranch with ssl now needs an additional non-standard 'sequence' key in their app file. >> >> on the contrary! >> it is ranch who needed this additional 'sequence' key. >> because 'ranch' knows the requirement, >> while a top-app does not have to know. >> >> you may call it a 'conditional-dep' >> where the condition is triggered by the top-app >> (without awareness) > > They need the machinery behind it, and which is unknown to OTP itself, even if they didn't write the actual 'sequence' property themselves. I am against adding conditional dependencies, given they are mostly a way for developers to be lazy when it comes down to splitting their own projects decently. sorry, i failed to understand the roles behind 'they' in your statement. From n.oxyde@REDACTED Tue Jan 27 17:23:34 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 17:23:34 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7B7CD.5090101@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7B5B3.1000108@bestmx.net> <54C7B7CD.5090101@bestmx.net> Message-ID: <4BA7C77F-BC2E-4131-B7AA-1C764CCBB19C@gmail.com> Le 27 janv. 2015 ? 17:07, e@REDACTED a ?crit : > sorry, i failed to understand the roles behind 'they' in your statement. Developers using ranch. From e@REDACTED Tue Jan 27 17:36:22 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 17:36:22 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7B5B3.1000108@bestmx.net> Message-ID: <54C7BE86.2040301@bestmx.net> i understand that my example was confusing, so i have rewritten it in the way it reflects my idea more exactly. http://file.bestmx.net/ee/software/relx-extended-deps/ notice the ssl dep in the top-app, it causes ssl to be started before ranch because ranch declares the 'sequence' if you remove ssl from the top-app, ssl will not be started. so that top-app have to care only about its own deps and does not have to know what ranch thinks about ssl. at the same time ranch does not make ssl to be started From essen@REDACTED Tue Jan 27 18:31:10 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 27 Jan 2015 18:31:10 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> Message-ID: <54C7CB5E.7080109@ninenines.eu> On 01/27/2015 04:53 PM, Anthony Ramine wrote: > Le 27 janv. 2015 ? 16:49, e@REDACTED a ?crit : > >> On 01/27/2015 04:47 PM, Anthony Ramine wrote: >>> Le 27 janv. 2015 ? 16:44, e@REDACTED a ?crit : >>> >>>> how am i running ranch without ssl now? >>> >>> By complaining to Ranch's author that he should either remove the code depending on ssl and put it in another separate project, or properly make his project depend on ssl and thus just not let you use ranch without ssl. >> >> hold on, i am not into political question, i want to keep it technical. >> i am running ranch without ssl. >> doesn't it contradict the statement of dependency? > > And anyone actually using ranch with ssl now needs an additional non-standard 'sequence' key in their app file. See the problem? The proper solution is to remove ranch_ssl from ranch and let you use that slimmed down version of it, not to accommodate one developer's laziness in a general purpose release tool. Slow down on the attacks there. The issue stems initially from a misunderstanding of the applications key in the .app file, which unlike what I believed until fairly recently, is not an ordered list of applications, but an unordered one. Despite this it worked without any issue until Erlang 17.3. I am guessing something changed in 17.3 that made it break, but it's fair game because it was properly documented as unordered. Years ago I pseudo-validated that this method was OK because another application in OTP was doing it: observer. Observer only lists kernel and stdlib as dependencies; on the other hand the newly added key runtime_dependencies lists wx, runtime_tools, inets and et. These are optionally required if you want to run the GUI (at least wx is). SSL is not listed in Ranch because some users do not use it and do not want 3 extra applications they do not use in their release. A number of solutions have been investigated to keep this feature for all current users. One of the more interesting solutions that was discarded was to make ranch_ssl its own application. But this only fixes Ranch and moves the problem one level up to Cowboy, which requires SSL for SPDY and soon for HTTP/2. These are the future protocols of the Web and Cowboy is aiming to support them at an equal level to HTTP/1.1 and Websocket, so it makes little sense to split HTTP/2 out. The solution retained is to fix the .app file, and document a way to remove SSL if you don't want it. For some people it will involve editing the .app file directly (possibly as part of the build process); for others it will involve telling reltool to exclude certain applications they don't need. Relx has currently no equivalent but it looks like it will be quickly addressed (thanks!). Now the only thing I have to figure out is if this warrants a Ranch 2.0 release or not. It's not obvious it needs one, since release tools will fetch SSL automatically just fine. Perhaps you can help answering that one? -- Lo?c Hoguin http://ninenines.eu From r.wobben@REDACTED Tue Jan 27 18:34:12 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 18:34:12 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C7A653.4080802@home.nl> References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> Message-ID: <54C7CC14.6070604@home.nl> Roelof Wobben schreef op 27-1-2015 om 15:53: > Hello, > > Oke, > > Something like this in pseudo code : > > create(0) -> > print empty array > > create(x) -> > create_array(x, []) ) > > create_array(0, array) -> > print array > > create_array(x, array) -> > add the first item to the array and run create_array again > > Roelof > > > > > > Lo?c Hoguin schreef op 27-1-2015 om 15:17: >> You can make a local function that takes more arguments and call it. >> >> On 01/27/2015 03:15 PM, Roelof Wobben wrote: >>> Hello, >>> >>> I try now to solve a challenge where I must make a list from a range, >>> >>> So for example if I do create(2) the outcome have to be [ 1, 2] >>> >>> So i can do >>> >>> create(0) -> >>> print array >>> >>> but as far as I know the list is then not in the scope so I cannot use >>> it when I do >>> >>> create(2) -> >>> add the 2 to the array. >>> >>> How do I take care that the list is in the scope and still using only >>> create(number) >>> >>> Can anyone give me a tip about it ? >>> >>> Roelof >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > I have now this: -module(create). -export([create_list/2, create/1]). create(0) -> []; create(number) -> create_list(number, [] ). create_list(0, [list]) -> [list]; create_list(number, [list]) -> create_list( number -1, [number | list] ). but when i do create:create(1) I see this error : ** exception error: no function clause matching create:create(1) (create.erl, line 5) which is correct and wierd. Correct because create(0) is not the right one. Wierd because it has to look at the second clause. Roelof From e@REDACTED Tue Jan 27 18:48:16 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 18:48:16 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7CB5E.7080109@ninenines.eu> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7CB5E.7080109@ninenines.eu> Message-ID: <54C7CF60.2070105@bestmx.net> > One of the more interesting solutions that was discarded was to make > ranch_ssl its own application. it is actually the worst solution, for the reason of "unnecessary duplication" (the worst evil ever in IT) and for the reason you have just described below. > But this only fixes Ranch and moves the > problem one level up to Cowboy, which requires SSL for SPDY and soon for > HTTP/2. These are the future protocols of the Web and Cowboy is aiming > to support them at an equal level to HTTP/1.1 and Websocket, so it makes > little sense to split HTTP/2 out. > The solution retained is to fix the .app file, and document a way to > remove SSL if you don't want it. Seems to me rather a workaround than a solution. It is acceptable, as well as it is acceptable to "sed" the .app file from within the Makefile, but a proper solution must not involve manual labour, especially when it is tinkering with a 3rd-party software (what is provided as a dep ideally must work without tinkering) Also, i want to emphasize, that this problem has an *ALGORITHMIC* solution, which automatically moves the problem from the realm of humans to the real of computers -- no human should be forced to do algorithmic job. From n.oxyde@REDACTED Tue Jan 27 19:01:14 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 27 Jan 2015 19:01:14 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7CF60.2070105@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7CB5E.7080109@ninenines.eu> <54C7CF60.2070105@bestmx.net> Message-ID: <2FE92FF2-BB6A-4F40-BACE-CFFD7728E8B1@gmail.com> Le 27 janv. 2015 ? 18:48, e@REDACTED a ?crit : > it is actually the worst solution, for the reason of "unnecessary duplication" (the worst evil ever in IT) and for the reason you have just described below. Le what? Making two unrelated things separate is unnecessary duplication? Duplication of what? From essen@REDACTED Tue Jan 27 19:02:54 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 27 Jan 2015 19:02:54 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7CF60.2070105@bestmx.net> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7CB5E.7080109@ninenines.eu> <54C7CF60.2070105@bestmx.net> Message-ID: <54C7D2CE.60702@ninenines.eu> On 01/27/2015 06:48 PM, e@REDACTED wrote: >> The solution retained is to fix the .app file, and document a way to >> remove SSL if you don't want it. > > Seems to me rather a workaround than a solution. > It is acceptable, as well as it is acceptable to "sed" the .app file > from within the Makefile, but a proper solution must not involve manual > labour, especially when it is tinkering with a 3rd-party software (what > is provided as a dep ideally must work without tinkering) > > Also, i want to emphasize, that this problem has an *ALGORITHMIC* > solution, which automatically moves the problem from the realm of humans > to the real of computers -- no human should be forced to do algorithmic > job. Not my call on any of these things though. OTP team acknowledged the need for runtime dependencies but as far as I know there are no solution for this today. Similarly, Erlware is aware of it but dismissed making applications start ordered. Traditionally it seems like the solution was to exclude unneeded applications from the release when generating it (using the reltool.config file), so the retained solution brings nothing new, and that's a good thing. If you are really serious on your solution, then the first step would probably to write an EEP, and then submit a patch to OTP with an implementation of said EEP for all involved code (reltool, kernel, possibly systool also...) -- Lo?c Hoguin http://ninenines.eu From imantc@REDACTED Tue Jan 27 19:03:58 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:03:58 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C7CC14.6070604@home.nl> References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> Message-ID: try changing number to Number? From r.wobben@REDACTED Tue Jan 27 19:12:45 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 19:12:45 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> Message-ID: <54C7D51D.8090202@home.nl> Imants Cekusins schreef op 27-1-2015 om 19:03: > try changing number to Number? > I tried and change everything to this : -module(create). -export([create_list/2, create/1]). create(0) -> []; create(Number) -> create_list(Number, [] ). create_list(0, [list]) -> [list]; create_list(Number, [list]) -> create_list( Number -1, [Number | list] ). and still this error message : ** exception error: no function clause matching create:create_list(1,[]) (create.erl, line 11) From imantc@REDACTED Tue Jan 27 19:17:46 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:17:46 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C7D51D.8090202@home.nl> References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> Message-ID: .. sorry: just noticed [list] should be List From imantc@REDACTED Tue Jan 27 19:20:02 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:20:02 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> Message-ID: lowcase values are atoms, not variables [list] means a list with one item: atom 'list' variables begin with upper case functions and modules begin with low case From r.wobben@REDACTED Tue Jan 27 19:22:47 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Tue, 27 Jan 2015 19:22:47 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> Message-ID: <54C7D777.7040609@home.nl> Imants Cekusins schreef op 27-1-2015 om 19:20: > lowcase values are atoms, not variables > > [list] means a list with one item: atom 'list' > > variables begin with upper case > > functions and modules begin with low case > oke, what I tried to say is a list with one or more items. then I still have to be List ? Roelof From imantc@REDACTED Tue Jan 27 19:23:58 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:23:58 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: <54C7D777.7040609@home.nl> References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> <54C7D777.7040609@home.nl> Message-ID: this works: -module(create). -export([create_list/2, create/1]). create(0) -> []; create(Number) -> create_list(Number, [] ). create_list(0, List) -> List; create_list(Number, List) -> create_list( Number -1, [Number | List] ). From e@REDACTED Tue Jan 27 19:25:18 2015 From: e@REDACTED (e@REDACTED) Date: Tue, 27 Jan 2015 19:25:18 +0100 Subject: [erlang-questions] relx extended deps In-Reply-To: <54C7D2CE.60702@ninenines.eu> References: <54C6E6C9.2050506@bestmx.net> <1422369222.3117304.219505485.4EE3D9A8@webmail.messagingengine.com> <54C7A629.80508@bestmx.net> <18299ABD-07D3-44CF-95D0-566E57E29B7C@gmail.com> <54C7AD5E.7090208@bestmx.net> <0C88BCCB-2C3A-400F-9025-9036C15167C9@gmail.com> <54C7B19B.8000109@bestmx.net> <54C7B271.6070004@bestmx.net> <85A7C8D6-CE52-4400-A6AC-9DBFBA63B559@gmail.com> <54C7B383.2030603@bestmx.net> <27F335D2-D045-461D-B11B-0C1D860EE3D7@gmail.com> <54C7CB5E.7080109@ninenines.eu> <54C7CF60.2070105@bestmx.net> <54C7D2CE.60702@ninenines.eu> Message-ID: <54C7D80E.1050600@bestmx.net> > OTP team acknowledged the > need for runtime dependencies thanx for noticing that, i could prevent lots of anger in the thread, should it be mentioned earlier. > Similarly, Erlware is aware of it but dismissed making > applications start ordered. Yes. Because no one can specify the complete order, But separate elements of the order, such as "conditional dependency" -- these could and should be specified. From imantc@REDACTED Tue Jan 27 19:28:47 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:28:47 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> <54C7D777.7040609@home.nl> Message-ID: a variable which is a list appears as a regular variable: List = [1,2]. [H | T] = List. H = 1. T = [2]. % tail is always a list [List] with List being a list, is a deep / nested list: [[1,2]] = [List]. [List,3] = [[1,2],3]. From imantc@REDACTED Tue Jan 27 19:34:32 2015 From: imantc@REDACTED (Imants Cekusins) Date: Tue, 27 Jan 2015 19:34:32 +0100 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> <54C7D777.7040609@home.nl> Message-ID: One_item_list = [atom1]. [A] = One_item_list. A = atom1. One item list may be pattern matched to H|T: [H|T] = One_item_list. H = atom1. T = []. [A,B] = One_item_list. ** exception error: no match of right hand side value [atom1] [A,B] = [1,2,3]. ** exception error: no match of right hand side value [1,2,3] From erlang@REDACTED Tue Jan 27 19:48:10 2015 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 27 Jan 2015 18:48:10 +0000 Subject: [erlang-questions] Question on building a Parallel Server In-Reply-To: References: Message-ID: On Tue, Jan 27, 2015 at 2:43 PM, Harit Himanshu wrote: > Hello > > As I learn from Programming Erlang, the following is a way to create a > parallel server > > start_parallel_server() -> > {ok, Listen} = gen_tcp:listen(...), spawn(fun() -> par_connect(Listen) end). > > par_connect(Listen) -> > {ok, Socket} = gen_tcp:accept(Listen), spawn(fun() -> par_connect(Listen) > end), loop(Socket). > > > I wanted to dissect this code to understand what's going on. > > Questions > > When we call start_parallel_server() on a single machine, a separate > process is started (which is controlling process?) executing par_connect, > correct? > When gen_tcp:accept accepts a new connection, immediately a new process is > created executing par_connect(Listen). This makes sure that we can accept > more requests from the client. Now who is the controlling process for this > new process? is the same as Listen? no Listen is owned by the process that called gen_tcp:listen Socket is owned by the process that called gen_tcp:accept So you end up with one Listen process and N different values of Socket (one per spawned process). Each parallel process has it's own socket /Joe > > > Thanks > + Harit Himanshu > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Tue Jan 27 19:48:59 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 27 Jan 2015 19:48:59 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Fri, Jan 23, 2015 at 6:03 PM, Lukas Larsson wrote: > > With a recon_alloc:memory(usage, current) value of 65 - 70%, this > difference is expected. What is happening is that the memory allocators of > the VM is not finding any good slots to put new heaps in, so it keeps > requesting more memory from the OS. erlang:memory reports used memory, > which is not the same as the memory requested from the OS. > recon_alloc:memory(allocated, current) reports something very close to what > has actually been requested from the OS, which is why this value is much > closer to what you see with top. > > It would be interesting to see if you get the same results if you start > the VM with "+Muacul 0", or if you get better utlization if you use that. > If you want to know what it is that you are disabling you can read this: > https://github.com/erlang/otp/blob/master/erts/emulator/internal_doc/CarrierMigration.md > . > > Lukas > Hi again, I had the time to try the +Muacul 0 option you suggested. It actually works pretty well since the RAM usage ratio (erlang / VM) improves considerably. Unfortunately both with or without the option I am getting the same problem: the machine seems stable, but at a certain moment *within 30 seconds* the VM memory usage suddenly increases from 11GB to 15GB until it crashes. I don't get it because the erlang usage is *very* stable now, I always get this kind of data: 1> [{K,V / math:pow(1024,3)} || {K,V} <- erlang:memory()]. [{total,8.803221724927425}, {processes,7.404303453862667}, {processes_used,7.404133327305317}, {system,1.3989182710647583}, {atom,0.0015203068032860756}, {atom_used,0.0015129167586565018}, {binary,0.9653719812631607}, {code,0.049161157570779324}, {ets,0.007966354489326477}] I see consistent total, process and binary usage. Unfortunately the ratio falls: 2> recon_alloc:memory(usage, current). 0.7353255106318904 ..after a while: 3> recon_alloc:memory(usage, current). 0.5630988225908702 Why is the VM so eager on memory if the underlying erlang usage is stable? Is there anything I can do? I honestly don't know where else to look. - Binaries are optimized (checked with +bin_opt_info). - Erlang reported memory for total, process and binary is linear. - I'm using some gimmicks like fullsweep_after 10 as a system flag. - I hibernate the long living TCP connections (which is where the problem comes from, since I ran tests on short lived connections and had no issues). Any help would be greatly appreciated. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Jan 27 20:09:44 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 27 Jan 2015 14:09:44 -0500 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: <20150127190943.GA90397@ferdair.local> On 01/27, Roberto Ostinelli wrote: > > I see consistent total, process and binary usage. Unfortunately the ratio > falls: > > 2> recon_alloc:memory(usage, current). > 0.7353255106318904 > > ..after a while: > > 3> recon_alloc:memory(usage, current). > 0.5630988225908702 > Yeah that does look like some good indication there's an allocator 'leak' (or bad usage). You can possibly look at other recon functions to try and figure things are wrong in specific ways (a given allocator is worse than others -- if it's allocator 0, that's for NIFs and drivers -- or other ones) > Why is the VM so eager on memory if the underlying erlang usage is stable? > > Is there anything I can do? I honestly don't know where else to look. > > - Binaries are optimized (checked with +bin_opt_info). > - Erlang reported memory for total, process and binary is linear. > - I'm using some gimmicks like fullsweep_after 10 as a system flag. > - I hibernate the long living TCP connections (which is where the > problem comes from, since I ran tests on short lived connections and had no > issues). > > Any help would be greatly appreciated. > What this looks like from the usage metrics is possibly the need for different memory allocation strategy. There'S unfortunately no super easy way to do it, but if the problem shows up quickly, that at least makes it a lot easier to experiment. I have covered the topic in Section 7.3 of Erlang in Anger (http://erlang-in-anger.com), Memory Fragmentation. The steps are usually: 1. Find that you have fragmentation issues (done) 2. Find which allocator is to blame 3. Take note of what your usage pattern is for that allocator. Is data held for a long time? Only some of it? Does it get released in large blocks? What's the variation in datasize type? Max, min, p95 or p99 size? 4. Check the different strategies available (p.71-73) and see if one could make sense for your usage. 5. Check your average block size (in recon_alloc) and see if you need to tweak yours up or down so more or less data gets to be used in the same operation (and may hold them back if they need to be released) 6. Experiment a lot and report back. If you tend to see lots of holes, doing things like reducing block sizes (while making sure your sbcs/mbcs ratio remains low enough) and looking for some address-order strategy (rather than best fit) might end up helping by reusing already-allocated blocks more, and also reducing how much spread there is. Anyway, that's more or less the extent of the experimenting I've done that can be applied in a generic manner. From mmartin4242@REDACTED Tue Jan 27 21:30:50 2015 From: mmartin4242@REDACTED (Michael L Martin) Date: Tue, 27 Jan 2015 14:30:50 -0600 Subject: [erlang-questions] mod_auth usage? Message-ID: <54C7F57A.5010707@gmail.com> Hello all, I'm trying to get my head around mod_auth in httpd. The doc for add_user/2 says that Password, UserdData, Port, and Dir are required options, but there is no description of what these are or should be (well, Password is pretty obvious). Can someone out there shine some light on this subject? Thanks, Michael From nathanfiedler@REDACTED Wed Jan 28 07:20:28 2015 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Tue, 27 Jan 2015 22:20:28 -0800 Subject: [erlang-questions] escript and application environment Message-ID: Web search yields many irrelevant results and the man pages for erl, escript, app, and application make no mention of this, that I can see. Let's say I have a simple escript as shown below. #!/usr/bin/env escript %%! -emu_args -myapp val var main([]) -> application:load(myapp), Results = application:get_all_env(myapp), io:format("results: ~p~n", [Results]), ok. Running this yields: $ ./myapp.escript Executing: /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp -B -- -root /usr/local/Cellar/erlang/17.4/lib/erlang -progname erl -- -home /Users/nfiedler -- -boot start_clean -noshell -myapp val var -run escript start -extra ./myapp.escript results: [] Does anyone know why the val setting does not show up? In a full blown application, everything is as expected, but why not in escript? Is there another call I need to make to facilitate this functionality? Thanks any help you can offer. n From mabrek@REDACTED Wed Jan 28 08:16:43 2015 From: mabrek@REDACTED (Anton Lebedevich) Date: Wed, 28 Jan 2015 11:16:43 +0400 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Tue, Jan 27, 2015 at 9:48 PM, Roberto Ostinelli wrote: ... > Unfortunately both with or without the option I am getting the same problem: > the machine seems stable, but at a certain moment *within 30 seconds* the VM > memory usage suddenly increases from 11GB to 15GB until it crashes. One way to catch it is to run a shell script that checks beam memory usage every second and forces crash dump via kill -USR1 when it passes some threshold. Then you can analyse obtained dump file for big processes or allocator usage. Regards, Anton Lebedevich. From n.oxyde@REDACTED Wed Jan 28 10:42:53 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 28 Jan 2015 10:42:53 +0100 Subject: [erlang-questions] escript and application environment In-Reply-To: References: Message-ID: <359E01EE-5812-4217-857F-CE406E5BB6EC@gmail.com> Le 28 janv. 2015 ? 07:20, Nathan Fiedler a ?crit : > Web search yields many irrelevant results and the man pages for erl, > escript, app, and application make no mention of this, that I can see. > Let's say I have a simple escript as shown below. > > #!/usr/bin/env escript > %%! -emu_args -myapp val var > > main([]) -> > application:load(myapp), > Results = application:get_all_env(myapp), > io:format("results: ~p~n", [Results]), > ok. > > Running this yields: > > $ ./myapp.escript > Executing: /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp > /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp -B -- > -root /usr/local/Cellar/erlang/17.4/lib/erlang -progname erl -- -home > /Users/nfiedler -- -boot start_clean -noshell -myapp val var -run > escript start -extra ./myapp.escript > > results: [] > > Does anyone know why the val setting does not show up? In a full blown > application, everything is as expected, but why not in escript? Is > there another call I need to make to facilitate this functionality? > > Thanks any help you can offer. > > n Replace "application:load(myapp)" by "ok = application:load(myapp)" and run the script again. I am pretty sure the application didn't even get loaded. Regards. From roberto@REDACTED Wed Jan 28 12:47:56 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 12:47:56 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: <20150127190943.GA90397@ferdair.local> References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: Hi Fred, Thank you for this. I was collecting allocator and other data to go down this route. Then I plotted all of this in a graph, and I'm actually seeing that the problem does not seem to be fragmentation at all. This is what I see: erlang memory and recon allocated: https://cldup.com/pbYr_YNfXw-3000x3000.png allocator memory: https://cldup.com/S9OaQZ84Zw-3000x3000.png recon usage: https://cldup.com/J9WpssSmt6-3000x3000.png I've got my system under controlled load, so the incoming requests are all the same at any given time. I'm using the +Muacul 0 option Lukas suggested, so that's why the recon usage never goes down to the 50% that I experienced before. As you can see, it looks that my problem comes from eheap memory, allocated for processes. Binary usage is now definitely stable. The last allocator info that I've got before the VM blew up where: *recon_alloc:fragmentation(current)* [{{eheap_alloc,1}, [{sbcs_usage,0.9998089938211238}, {mbcs_usage,0.8436328764998643}, {sbcs_block_size,11515752}, {sbcs_carriers_size,11517952}, {mbcs_block_size,1398905280}, {mbcs_carriers_size,1658191992}]}, {{eheap_alloc,6}, [{sbcs_usage,0.6174983671623794}, {mbcs_usage,0.8458609176542206}, {sbcs_block_size,3146416}, {sbcs_carriers_size,5095424}, {mbcs_block_size,1402599800}, {mbcs_carriers_size,1658191992}]}, {{eheap_alloc,7}, [{sbcs_usage,0.9996959805427548}, {mbcs_usage,0.8455184225713416}, {sbcs_block_size,7997056}, {sbcs_carriers_size,7999488}, {mbcs_block_size,1409124600}, {mbcs_carriers_size,1666580600}]}, {{eheap_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.8449706854053697}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1401788136}, {mbcs_carriers_size,1658978424}]}, {{eheap_alloc,3}, [{sbcs_usage,0.997382155987395}, {mbcs_usage,0.8450586945061064}, {sbcs_block_size,972296}, {sbcs_carriers_size,974848}, {mbcs_block_size,1401269560}, {mbcs_carriers_size,1658191992}]}, {{eheap_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.8460061373569331}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1409937416}, {mbcs_carriers_size,1666580600}]}, {{eheap_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.8445225737274885}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1393296200}, {mbcs_carriers_size,1649803384}]}, {{eheap_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.8437420231923147}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1384930624}, {mbcs_carriers_size,1641414776}]}, {{binary_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.6716575320561889}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,99502200}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.6781073538541975}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,100457704}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.6796450983368588}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,100685512}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.6840474562333329}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,101337696}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.7055153695407212}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,98599752}, {mbcs_carriers_size,139755640}]}, {{binary_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7102128257578728}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,99256248}, {mbcs_carriers_size,139755640}]}, {{binary_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.7147085012096829}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,99884544}, {mbcs_carriers_size,139755640}]}, {{binary_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.7225102042050122}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,102300688}, {mbcs_carriers_size,141590648}]}, {{ll_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.8925413646389566}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,211512896}, {mbcs_carriers_size,236978256}]}, {{driver_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.8187761906238381}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25139488}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.8188191821347083}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25140808}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.8218319751055703}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25233312}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.8219393236054401}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25236608}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.8316551445074958}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25534920}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.8330978353904555}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25579216}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.8339579261624709}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25605624}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.844592202069481}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25932136}, {mbcs_carriers_size,30703736}]}, {{fix_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.9433490069813972}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,68531264}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.9573151050777532}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,69545856}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.9603094292856162}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,69763384}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.9669017658815307}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70242296}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.9680059580345314}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70322512}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.9682964595703463}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70343616}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.9716501114929037}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70587248}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.9716850201308314}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70589784}, {mbcs_carriers_size,72646776}]}, {{std_alloc,7}, [{sbcs_usage,0.8571690150669643}, {mbcs_usage,0.7343726386030993}, {sbcs_block_size,1572912}, {sbcs_carriers_size,1835008}, {mbcs_block_size,2526800}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.669268417442658}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2302792}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.6742580127646217}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2319960}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.6778409421174392}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2332288}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.6785849637870703}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2334848}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.6792173822062567}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2337024}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.679821899812832}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2339104}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.6809565328590195}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2343008}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.6915890675315919}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2379592}, {mbcs_carriers_size,3440760}]}, {{std_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.7155860914449134}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2462160}, {mbcs_carriers_size,3440760}]}, {{std_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7317499622176495}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2517776}, {mbcs_carriers_size,3440760}]}, {{std_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.7326009370022902}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2520704}, {mbcs_carriers_size,3440760}]}, {{std_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.7368139597065765}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2535200}, {mbcs_carriers_size,3440760}]}, {{std_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.7383717550773666}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2540560}, {mbcs_carriers_size,3440760}]}, {{std_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.7468582522465966}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2569760}, {mbcs_carriers_size,3440760}]}, {{std_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.7473279159255514}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2571376}, {mbcs_carriers_size,3440760}]}, {{ets_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.643050651678168}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,864008}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.6700704372108531}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,900312}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.6724282677685753}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,903480}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.6745300712707873}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,906304}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.6789420723901615}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,912232}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.6872540205178892}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,923400}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.7930825062071676}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1065592}, {mbcs_carriers_size,1343608}]}, {{eheap_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.33664856509447394}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,132416}, {mbcs_carriers_size,393336}]}, {{ets_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.8979523789676751}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1206496}, {mbcs_carriers_size,1343608}]}, {{temp_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{temp_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,131192}]}, {{std_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.8726646601046666}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,257464}, {mbcs_carriers_size,295032}]}, {{sl_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.0}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,0}, {mbcs_carriers_size,32888}]}, {{fix_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.02456823157382632}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,808}, {mbcs_carriers_size,32888}]}, {{driver_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.039649720262709805}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1304}, {mbcs_carriers_size,32888}]}, {{binary_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.41303819022135735}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,13584}, {mbcs_carriers_size,32888}]}, {{ets_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.4582826562880078}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15072}, {mbcs_carriers_size,32888}]}] and *recon_alloc:fragmentation(max)* [{{eheap_alloc,1}, [{sbcs_usage,0.9995836942360554}, {mbcs_usage,0.8438346046481209}, {sbcs_block_size,35747288}, {sbcs_carriers_size,35762176}, {mbcs_block_size,1399239784}, {mbcs_carriers_size,1658191992}]}, {{eheap_alloc,5}, [{sbcs_usage,0.947959190616155}, {mbcs_usage,0.8462006049992422}, {sbcs_block_size,17833888}, {sbcs_carriers_size,18812928}, {mbcs_block_size,1410261512}, {mbcs_carriers_size,1666580600}]}, {{eheap_alloc,2}, [{sbcs_usage,0.9996057750948681}, {mbcs_usage,0.8451859214776624}, {sbcs_block_size,22658328}, {sbcs_carriers_size,22667264}, {mbcs_block_size,1402145208}, {mbcs_carriers_size,1658978424}]}, {{eheap_alloc,7}, [{sbcs_usage,0.9994766588528468}, {mbcs_usage,0.8459256516006487}, {sbcs_block_size,24231536}, {sbcs_carriers_size,24244224}, {mbcs_block_size,1409803280}, {mbcs_carriers_size,1666580600}]}, {{eheap_alloc,3}, [{sbcs_usage,0.9996129234481453}, {mbcs_usage,0.8454477688733163}, {sbcs_block_size,27043608}, {sbcs_carriers_size,27054080}, {mbcs_block_size,1401914720}, {mbcs_carriers_size,1658191992}]}, {{eheap_alloc,8}, [{sbcs_usage,0.9996057750948681}, {mbcs_usage,0.8446861617056788}, {sbcs_block_size,22658328}, {sbcs_carriers_size,22667264}, {mbcs_block_size,1393566088}, {mbcs_carriers_size,1649803384}]}, {{eheap_alloc,4}, [{sbcs_usage,0.9996057750948681}, {mbcs_usage,0.8443270209723029}, {sbcs_block_size,22658328}, {sbcs_carriers_size,22667264}, {mbcs_block_size,1385890848}, {mbcs_carriers_size,1641414776}]}, {{eheap_alloc,6}, [{sbcs_usage,0.999511325925181}, {mbcs_usage,0.8462386302490357}, {sbcs_block_size,25444200}, {sbcs_carriers_size,25456640}, {mbcs_block_size,1403226120}, {mbcs_carriers_size,1658191992}]}, {{ll_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.9004818737462563}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,213394624}, {mbcs_carriers_size,236978256}]}, {{binary_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.8944150703711425}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,132502448}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.8954607538998072}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,132657360}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.8966842101085153}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,132838608}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.8991528310974315}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,133204320}, {mbcs_carriers_size,148144248}]}, {{binary_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.9407874487212108}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,131480352}, {mbcs_carriers_size,139755640}]}, {{binary_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.9438688842897504}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,131911000}, {mbcs_carriers_size,139755640}]}, {{binary_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.9512322593509142}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,134685592}, {mbcs_carriers_size,141590648}]}, {{binary_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.9556126965609403}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,133552264}, {mbcs_carriers_size,139755640}]}, {{driver_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.8197313838289907}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25168816}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.8225703868740925}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25255984}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.825497196823214}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25345848}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.8260857896902188}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25363920}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.8351752373066261}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25643000}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.8361403315870095}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25672632}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.8387523915656387}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25752832}, {mbcs_carriers_size,30703736}]}, {{driver_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.8461701207957234}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,25980584}, {mbcs_carriers_size,30703736}]}, {{fix_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.9439967439160686}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,68578320}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.9576758643769684}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,69572064}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.9603094292856162}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,69763384}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.9676070965626885}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70293536}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.9680071693752796}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70322600}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.9685851991559818}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70364592}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.9718468993035562}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70601544}, {mbcs_carriers_size,72646776}]}, {{fix_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.9719647297217979}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,70610104}, {mbcs_carriers_size,72646776}]}, {{std_alloc,4}, [{sbcs_usage,0.7500457763671875}, {mbcs_usage,0.7213592345877073}, {sbcs_block_size,786480}, {sbcs_carriers_size,1048576}, {mbcs_block_size,2482024}, {mbcs_carriers_size,3440760}]}, {{std_alloc,7}, [{sbcs_usage,0.8571690150669643}, {mbcs_usage,0.7376068077982771}, {sbcs_block_size,1572912}, {sbcs_carriers_size,1835008}, {mbcs_block_size,2537928}, {mbcs_carriers_size,3440760}]}, {{std_alloc,3}, [{sbcs_usage,0.80003662109375}, {mbcs_usage,0.7450307490205652}, {sbcs_block_size,1048624}, {sbcs_carriers_size,1310720}, {mbcs_block_size,2563472}, {mbcs_carriers_size,3440760}]}, {{std_alloc,5}, [{sbcs_usage,0.8333638509114584}, {mbcs_usage,0.7537404526906846}, {sbcs_block_size,1310768}, {sbcs_carriers_size,1572864}, {mbcs_block_size,2593440}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.6812471663237192}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2344008}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.6843627570652995}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2354728}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.6919796789081482}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2380936}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.7003545728269336}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2409752}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.7007591346097956}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2411144}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.7018333158953255}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2414840}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.7091782048152152}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2440112}, {mbcs_carriers_size,3440760}]}, {{std_alloc,6}, [{sbcs_usage,0.90003662109375}, {mbcs_usage,0.7518385472976901}, {sbcs_block_size,1179696}, {sbcs_carriers_size,1310720}, {mbcs_block_size,2586896}, {mbcs_carriers_size,3440760}]}, {{std_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.7386042618491263}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2541360}, {mbcs_carriers_size,3440760}]}, {{sl_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7387228403027238}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2541768}, {mbcs_carriers_size,3440760}]}, {{std_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.7394575617014846}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,2544296}, {mbcs_carriers_size,3440760}]}, {{std_alloc,2}, [{sbcs_usage,0.9938373447204969}, {mbcs_usage,0.7439193666515538}, {sbcs_block_size,655392}, {sbcs_carriers_size,659456}, {mbcs_block_size,2559648}, {mbcs_carriers_size,3440760}]}, {{temp_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.2534939072766849}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,299064}, {mbcs_carriers_size,1179768}]}, {{binary_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.39198933022131455}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,526680}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.6432173669701282}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,864232}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.6702371525028133}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,900536}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.6746967865627475}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,906528}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.6770010300623396}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,909624}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.6791087876821216}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,912456}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.6874207358098493}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,923624}, {mbcs_carriers_size,1343608}]}, {{ets_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.7939398991372484}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1066744}, {mbcs_carriers_size,1343608}]}, {{eheap_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.3466756157585372}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,136360}, {mbcs_carriers_size,393336}]}, {{sl_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.1492448276797093}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,44032}, {mbcs_carriers_size,295032}]}, {{driver_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.4211610943897611}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,124256}, {mbcs_carriers_size,295032}]}, {{ets_alloc,3}, [{sbcs_usage,1.0}, {mbcs_usage,0.8981190942596352}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,1206720}, {mbcs_carriers_size,1343608}]}, {{temp_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.07579730471370205}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,9944}, {mbcs_carriers_size,131192}]}, {{temp_alloc,8}, [{sbcs_usage,1.0}, {mbcs_usage,0.11823891700713458}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15512}, {mbcs_carriers_size,131192}]}, {{temp_alloc,7}, [{sbcs_usage,1.0}, {mbcs_usage,0.11823891700713458}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15512}, {mbcs_carriers_size,131192}]}, {{temp_alloc,4}, [{sbcs_usage,1.0}, {mbcs_usage,0.11823891700713458}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15512}, {mbcs_carriers_size,131192}]}, {{temp_alloc,2}, [{sbcs_usage,1.0}, {mbcs_usage,0.11823891700713458}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15512}, {mbcs_carriers_size,131192}]}, {{temp_alloc,1}, [{sbcs_usage,1.0}, {mbcs_usage,0.11823891700713458}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15512}, {mbcs_carriers_size,131192}]}, {{temp_alloc,5}, [{sbcs_usage,1.0}, {mbcs_usage,0.12494664308799316}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,16392}, {mbcs_carriers_size,131192}]}, {{temp_alloc,6}, [{sbcs_usage,1.0}, {mbcs_usage,0.2140984206354046}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,28088}, {mbcs_carriers_size,131192}]}, {{fix_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.02627098029676478}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,864}, {mbcs_carriers_size,32888}]}, {{std_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.90007863553784}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,265552}, {mbcs_carriers_size,295032}]}, {{ets_alloc,0}, [{sbcs_usage,1.0}, {mbcs_usage,0.4582826562880078}, {sbcs_block_size,0}, {sbcs_carriers_size,0}, {mbcs_block_size,15072}, {mbcs_carriers_size,32888}]}] Both of these look fine to me, unless I've missed a spot. What I do not understand is why, in a system under stable load, suddenly the eheap blows up like that, eventually crashing the system (this box only has 15GB of RAM). Any suggestions on the steps I could make to debug this? Best, r. On Tue, Jan 27, 2015 at 8:09 PM, Fred Hebert wrote: > On 01/27, Roberto Ostinelli wrote: > > > > I see consistent total, process and binary usage. Unfortunately the ratio > > falls: > > > > 2> recon_alloc:memory(usage, current). > > 0.7353255106318904 > > > > ..after a while: > > > > 3> recon_alloc:memory(usage, current). > > 0.5630988225908702 > > > > Yeah that does look like some good indication there's an allocator > 'leak' (or bad usage). You can possibly look at other recon functions to > try and figure things are wrong in specific ways (a given allocator is > worse than others -- if it's allocator 0, that's for NIFs and drivers -- > or other ones) > > > Why is the VM so eager on memory if the underlying erlang usage is > stable? > > > > Is there anything I can do? I honestly don't know where else to look. > > > > - Binaries are optimized (checked with +bin_opt_info). > > - Erlang reported memory for total, process and binary is linear. > > - I'm using some gimmicks like fullsweep_after 10 as a system flag. > > - I hibernate the long living TCP connections (which is where the > > problem comes from, since I ran tests on short lived connections and > had no > > issues). > > > > Any help would be greatly appreciated. > > > > What this looks like from the usage metrics is possibly the need for > different memory allocation strategy. There'S unfortunately no super > easy way to do it, but if the problem shows up quickly, that at least > makes it a lot easier to experiment. > > I have covered the topic in Section 7.3 of Erlang in Anger > (http://erlang-in-anger.com), Memory Fragmentation. > > The steps are usually: > > 1. Find that you have fragmentation issues (done) > 2. Find which allocator is to blame > 3. Take note of what your usage pattern is for that allocator. Is data > held for a long time? Only some of it? Does it get released in large > blocks? What's the variation in datasize type? Max, min, p95 or p99 > size? > 4. Check the different strategies available (p.71-73) and see if one > could make sense for your usage. > 5. Check your average block size (in recon_alloc) and see if you need to > tweak yours up or down so more or less data gets to be used in the > same operation (and may hold them back if they need to be released) > 6. Experiment a lot and report back. > > If you tend to see lots of holes, doing things like reducing block sizes > (while making sure your sbcs/mbcs ratio remains low enough) and looking > for some address-order strategy (rather than best fit) might end up > helping by reusing already-allocated blocks more, and also reducing how > much spread there is. > > Anyway, that's more or less the extent of the experimenting I've done > that can be applied in a generic manner. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jan 28 12:50:00 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 12:50:00 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Wed, Jan 28, 2015 at 8:16 AM, Anton Lebedevich wrote: > One way to catch it is to run a shell script that checks beam memory > usage every second and forces crash dump via kill -USR1 when it passes > some threshold. Then you can analyse obtained dump file for big > processes or allocator usage. > > Regards, > Anton Lebedevich. > Hi Anton, Yes I'm also puzzled why no crash dump is generated for me. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabrek@REDACTED Wed Jan 28 13:21:03 2015 From: mabrek@REDACTED (Anton Lebedevich) Date: Wed, 28 Jan 2015 16:21:03 +0400 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> Message-ID: On Wed, Jan 28, 2015 at 2:50 PM, Roberto Ostinelli wrote: > On Wed, Jan 28, 2015 at 8:16 AM, Anton Lebedevich wrote: >> >> One way to catch it is to run a shell script that checks beam memory >> usage every second and forces crash dump via kill -USR1 when it passes >> some threshold. Then you can analyse obtained dump file for big >> processes or allocator usage. > > Hi Anton, > Yes I'm also puzzled why no crash dump is generated for me. Crash with no dump might be caused by system OOM killer. Regards, Anton From lukas@REDACTED Wed Jan 28 13:35:53 2015 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 28 Jan 2015 13:35:53 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: Hello, On Wed, Jan 28, 2015 at 12:47 PM, Roberto Ostinelli wrote: What I do not understand is why, in a system under stable load, suddenly > the eheap blows up like that, eventually crashing the system (this box only > has 15GB of RAM). > Could it be that you at this point trigger a lot of garbage collections due to calling erlang:garbage_collect/1? If you do that on a lot of processes at the same time you might get an explosion in eheap memory usage. > > Any suggestions on the steps I could make to debug this? > > If you cannot get a crash dump, I would look for changes in erlang:statistics/1 counters, to see if that can indicate if something in the system is changing drastically at that point, i.e. the number of reductions executed, the number of gc's done etc etc. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jan 28 14:42:57 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 14:42:57 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: On Wed, Jan 28, 2015 at 1:35 PM, Lukas Larsson wrote: > > Could it be that you at this point trigger a lot of garbage collections > due to calling erlang:garbage_collect/1? If you do that on a lot of > processes at the same time you might get an explosion in eheap memory usage. > I constantly hibernate long term processes, all along the run. Nothing specific happens after a while. On Wed, Jan 28, 2015 at 1:35 PM, Lukas Larsson wrote: > > If you cannot get a crash dump, I would look for changes in > erlang:statistics/1 counters, to see if that can indicate if something in > the system is changing drastically at that point, i.e. the number of > reductions executed, the number of gc's done etc etc. > The number of reductions stay constant: https://cldup.com/SiwAVf1VT5-3000x3000.png So does the number of GC: https://cldup.com/9Q5ac7Q5eK-3000x3000.png The process count is always the same: https://cldup.com/tsuorbj06u-3000x3000.png And the maximum message box length of any process in the system is always between 1-3 (despite the peak at the start): https://cldup.com/x4KYsykOrH-3000x3000.png ...Any other ideas? I'm getting appalled. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanfiedler@REDACTED Wed Jan 28 15:02:45 2015 From: nathanfiedler@REDACTED (Nathan Fiedler) Date: Wed, 28 Jan 2015 06:02:45 -0800 Subject: [erlang-questions] escript and application environment In-Reply-To: <359E01EE-5812-4217-857F-CE406E5BB6EC@gmail.com> References: <359E01EE-5812-4217-857F-CE406E5BB6EC@gmail.com> Message-ID: Oh yeah, I need a myapp.app file, duh. Thanks for addressing my question, dumb as it was. n On Wed, Jan 28, 2015 at 1:42 AM, Anthony Ramine wrote: > Le 28 janv. 2015 ? 07:20, Nathan Fiedler a ?crit : > >> Web search yields many irrelevant results and the man pages for erl, >> escript, app, and application make no mention of this, that I can see. >> Let's say I have a simple escript as shown below. >> >> #!/usr/bin/env escript >> %%! -emu_args -myapp val var >> >> main([]) -> >> application:load(myapp), >> Results = application:get_all_env(myapp), >> io:format("results: ~p~n", [Results]), >> ok. >> >> Running this yields: >> >> $ ./myapp.escript >> Executing: /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp >> /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp -B -- >> -root /usr/local/Cellar/erlang/17.4/lib/erlang -progname erl -- -home >> /Users/nfiedler -- -boot start_clean -noshell -myapp val var -run >> escript start -extra ./myapp.escript >> >> results: [] >> >> Does anyone know why the val setting does not show up? In a full blown >> application, everything is as expected, but why not in escript? Is >> there another call I need to make to facilitate this functionality? >> >> Thanks any help you can offer. >> >> n > > Replace "application:load(myapp)" by "ok = application:load(myapp)" and run the script again. I am pretty sure the application didn't even get loaded. > > Regards. From mabrek@REDACTED Wed Jan 28 15:08:07 2015 From: mabrek@REDACTED (Anton Lebedevich) Date: Wed, 28 Jan 2015 18:08:07 +0400 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: > The number of reductions stay constant: > https://cldup.com/SiwAVf1VT5-3000x3000.png > > So does the number of GC: > https://cldup.com/9Q5ac7Q5eK-3000x3000.png > > The process count is always the same: > https://cldup.com/tsuorbj06u-3000x3000.png > > And the maximum message box length of any process in the system is always > between 1-3 (despite the peak at the start): > https://cldup.com/x4KYsykOrH-3000x3000.png > > ...Any other ideas? I'm getting appalled. Time scale is different so it's not possible to correlate process memory with number of reductions or number of GCs. It seems that some process (or processes) starts allocating memory much faster than before and linux OOM kills the beam when it runs out of memory on the box. You can try setting a watchdog process (something like "while true; check memory usage and kill -USR1 beam.smp when it's close to the limit; sleep 1") to get crash dump before OOM kills beam.smp Is there anything unusual in logs at the momeng when memory usage is jumping? Maybe something gets printed to stdout. Regards, Anton. From roberto@REDACTED Wed Jan 28 15:15:59 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 15:15:59 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: On Wed, Jan 28, 2015 at 3:08 PM, Anton Lebedevich wrote: > Time scale is different so it's not possible to correlate process > memory with number of reductions or number of GCs. > Time scale is different from previous test, but the graph is the same. Things get ugly at 140. > It seems that some process (or processes) starts allocating memory > much faster than before and linux OOM kills the beam when it runs out > of memory on the box. How can you see that? > You can try setting a watchdog process > (something like "while true; check memory usage and kill -USR1 > beam.smp when it's close to the limit; sleep 1") to get crash dump > before OOM kills beam.smp > I did that, I'm currently waiting for the crash dump to finish (it is 2.9GB right now, and still piling up). > Is there anything unusual in logs at the momeng when memory usage is > jumping? Maybe something gets printed to stdout. Nothing unfortunately. Thank you for your help, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Wed Jan 28 15:21:21 2015 From: community-manager@REDACTED (Bruce Yinhe) Date: Wed, 28 Jan 2015 15:21:21 +0100 Subject: [erlang-questions] =?utf-8?q?=5BANN=5D_OSCON_2015_Call_for_Speake?= =?utf-8?q?rs_July_20=E2=80=9324=2C_2015_-_Portland=2C_OR?= Message-ID: Hi everyone, This is a reminder that OSCON 2015 (O'Reilly Open Source Convention) submission deadline is on Monday Feb 2nd. It is a great place for some Erlang talks. For more info and to submit talks, visit: http://www.oscon.com/open-source-2015/public/cfp/360 Cheers, Bruce OSCON 2015 CALL FOR SPEAKERS Call closes *11:59pm 02/02/2015 PST*. Submit a proposal OSCON 2015 will celebrate, explain, and demonstrate the power of open source technologies from the inception of languages and frameworks up through their use in the enterprise. We invite you join us as we bring together a large community of contributors, learners, and leveragers. Please submit original session and tutorial ideas that share your technology passions. Proposals should include as much detail about the topic and format for the presentation as possible. Detail matters; vague proposals face an uphill climb. If you are one or more of the following, we invite you to submit a proposal to lead sessions and/or tutorials at OSCON 2015: - Developer or programmer - Systems administrator - Hacker or geek - Enterprise developer or manager - IT manager, CxO, or entrepreneur - Trainer or educator - User experience designer - Open source enthusiast or activist In the past, OSCON?s amazing talks have been grouped by programming verticals. However, as open source now resides from the bottom to the very top of the programming stack, we?re evolving the way we organize the stories. All the technology you love will still be represented at the event, but we?re grouping sessions in a more holistic way to help you discover, consider, and solve groups of critical problems. More info at http://www.oscon.com/open-source-2015/public/cfp/360 -- Bruce Yinhe Erlang Community Manager Industrial Erlang User Group community-manager@REDACTED +46 72 311 43 89 -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Wed Jan 28 15:45:46 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 15:45:46 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <54C14448.3070804@gmail.com> <20150127190943.GA90397@ferdair.local> Message-ID: Here's the erl_crash.dump analysis, done with the nice Fred's script: analyzing erl_crash.dump, generated on: Wed Jan 28 13:59:36 2015 Slogan: Received SIGUSR1 Memory: === processes: 8870 Mb processes_used: 8869 Mb system: 1138 Mb atom: 0 Mb atom_used: 0 Mb binary: 750 Mb code: 9 Mb ets: 7 Mb --- total: 10008 Mb Different message queue lengths (5 largest different): === 540314 0 Error logger queue length: === File descriptors open: === UDP: 0 TCP: 180071 Files: 6 --- Total: 180077 Number of processes: === 540314 Processes Heap+Stack memory sizes (words) used in the VM (5 largest different): === 2 196650 1 28690 1 17731 1 10958 4677 6772 Processes OldHeap memory sizes (words) used in the VM (5 largest different): === 1 1439468 1 999631 1 75113 1 28690 1 17731 Process States when crashing (sum): === 540314 Waiting Do you see anything wrong there? I honestly don't. Best, r. On Wed, Jan 28, 2015 at 3:15 PM, Roberto Ostinelli wrote: > > On Wed, Jan 28, 2015 at 3:08 PM, Anton Lebedevich > wrote: > >> Time scale is different so it's not possible to correlate process >> memory with number of reductions or number of GCs. >> > > Time scale is different from previous test, but the graph is the same. > Things get ugly at 140. > > > >> It seems that some process (or processes) starts allocating memory >> much faster than before and linux OOM kills the beam when it runs out >> of memory on the box. > > > How can you see that? > > > >> You can try setting a watchdog process >> (something like "while true; check memory usage and kill -USR1 >> beam.smp when it's close to the limit; sleep 1") to get crash dump >> before OOM kills beam.smp >> > > I did that, I'm currently waiting for the crash dump to finish (it is > 2.9GB right now, and still piling up). > > > >> Is there anything unusual in logs at the momeng when memory usage is >> jumping? Maybe something gets printed to stdout. > > > Nothing unfortunately. > > Thank you for your help, > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Chris.Nicel@REDACTED Wed Jan 28 15:48:32 2015 From: Chris.Nicel@REDACTED (Chris Nicel) Date: Wed, 28 Jan 2015 08:48:32 -0600 Subject: [erlang-questions] FW: [rabbitmq-users] glibc GHOST vulnerability and rabbitmq In-Reply-To: References: Message-ID: Hi All, I have a question about the GHOST vulnerability and erlangs use of the gethostbyname() function. We use RabbitMQ here and I am attempting to understand how vulnerable to attack we are on our Linux servers so I can weigh up the odds and give my superiors a good reason to upgrade and reboot the servers. RabbitMQ invokes the gethostbyname() function through it?s erlang library. How does the erlang library handle calls to gethostbyname? Does it sanitise the inputs or limit the length of the hostname prior to calling out? Cheers Chris From: Michael Klishin [mailto:mklishin@REDACTED] Sent: 28 January 2015 13:28 To: Chris Nicel Cc: rabbitmq-users@REDACTED Subject: Re: [rabbitmq-users] glibc GHOST vulnerability and rabbitmq On 28/1/2015, at 16:17, Chris Nicel > wrote: Can you confirm if either of the following conditions are true related to erlang and rabbitmq: 1. The service's protocol involves it being given a hostname which needs resolving to an IP RabbitMQ server in a cluster performs hostname resolution. So does rabbitmqctl (in most cases). 2. The service doesn't sanitise or limit the length of the given hostname before calling getHostByName RabbitMQ does not do that, however, it also does not invoke gethostbyname(2) directly. Please ask on erlang-questions, since this is handled by the runtime. MK 15below Limited: Company registered in England and Wales No 3945289 Registered Office: Lyndean House, 43-46 Queens Road, Brighton BN1 3XB, United Kingdom 15below Australia Pty Limited: ABN 25 132 716 379 Level 21, Tower 2 Darling Park, 201 Sussex Street, Sydney, NSW 2000, Australia Please think about the environment before printing this email. ************************************************************************ This email and any attachments may be confidential and/or legally privileged and are solely for the use of the intended recipient. If you have received this email in error please contact the sender. Any views or opinions expressed within this e-mail are solely those of the sender, and do not necessarily represent those of 15below unless otherwise specifically stated. Although 15below has taken every reasonable precaution to ensure that any attachment to this e-mail has been checked for viruses, it is strongly recommended that you carry out your own virus check before opening any attachment, as we cannot accept liability for any damage sustained as a result of software virus infection. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Jan 28 16:38:53 2015 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 28 Jan 2015 10:38:53 -0500 Subject: [erlang-questions] FW: [rabbitmq-users] glibc GHOST vulnerability and rabbitmq In-Reply-To: References: Message-ID: <20150128153853.GA7061@brk> On Wed, Jan 28, 2015 at 08:48:32AM -0600, Chris Nicel wrote: > Hi All, > > I have a question about the GHOST vulnerability and erlangs use of the gethostbyname() function. We use RabbitMQ here and I am attempting to understand how vulnerable to attack we are on our Linux servers so I can weigh up the odds and give my superiors a good reason to upgrade and reboot the servers. > > RabbitMQ invokes the gethostbyname() function through it?s erlang library. How does the erlang library handle calls to gethostbyname? Does it sanitise the inputs or limit the length of the hostname prior to calling out? > > Cheers > > Chris inet:gethostbyname/1 starts up a native port process that calls gethostbyname(3) directly: ~~~ erts/etc/common/inet_gethost.c case PROTO_IPV4: { /* switch (proto) { */ DEBUGF(5,("Starting gethostbyname(%s)",data)); he = gethostbyname((char*)data); ~~~ $ erl 1> X = string:copies("0", 16#10000-16*1-2*4-1-4). 2> inet:gethostbyname(X). {error,nxdomain} 3> inet:gethostbyname(X). *** glibc detected *** inet_gethost: free(): invalid next size (normal): 0x01f9bdb0 *** inet_gethost[3018]: WARNING:Malformed reply (header) from worker process 3019. > From: Michael Klishin [mailto:mklishin@REDACTED] > Sent: 28 January 2015 13:28 > To: Chris Nicel > Cc: rabbitmq-users@REDACTED > Subject: Re: [rabbitmq-users] glibc GHOST vulnerability and rabbitmq > > On 28/1/2015, at 16:17, Chris Nicel > wrote: > Can you confirm if either of the following conditions are true related to erlang and rabbitmq: > > > 1. The service's protocol involves it being given a hostname which needs resolving to an IP > RabbitMQ server in a cluster performs hostname resolution. So does rabbitmqctl (in most cases). > > > 2. The service doesn't sanitise or limit the length of the given hostname before calling getHostByName > RabbitMQ does not do that, however, it also does not invoke gethostbyname(2) directly. Please ask on erlang-questions, since this is handled by the runtime. > > MK > > 15below Limited: Company registered in England and Wales No 3945289 > Registered Office: Lyndean House, 43-46 Queens Road, Brighton BN1 3XB, United Kingdom > > 15below Australia Pty Limited: ABN 25 132 716 379 > Level 21, Tower 2 Darling Park, 201 Sussex Street, Sydney, NSW 2000, Australia > > Please think about the environment before printing this email. > > ************************************************************************ > This email and any attachments may be confidential and/or legally privileged and are solely for the use of the intended recipient. If you have received this email in error please contact the sender. Any views or opinions expressed within this e-mail are solely those of the sender, and do not necessarily represent those of 15below unless otherwise specifically stated. Although 15below has taken every reasonable precaution to ensure that any attachment to this e-mail has been checked for viruses, it is strongly recommended that you carry out your own virus check before opening any attachment, as we cannot accept liability for any damage sustained as a result of software virus infection. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From drormein@REDACTED Wed Jan 28 17:00:41 2015 From: drormein@REDACTED (Dror Mein) Date: Wed, 28 Jan 2015 16:00:41 +0000 (UTC) Subject: [erlang-questions] how to dynamically create function to run in a different node In-Reply-To: References: Message-ID: <1382600135.1832124.1422460842002.JavaMail.yahoo@mail.yahoo.com> thanks Ulf for the help, just to rap things up, what I ended up doing was this:http://stackoverflow.com/questions/2008777/convert-a-string-into-a-fun which If I understand is based on the same data types you told me about.Seeing the way a token and exprs of fun looks is very enlightening On Thursday, January 1, 2015 11:15 PM, Ulf Wiger wrote: On 31 Dec 2014, at 18:49, Dror Mein wrote: Thank you! so parse_transform is responsible of returning {M,F,A} where A is something I don't yet understand :).is there a way to turn {M,F,A} to fun? Well, if you want to dispatch it to another node, using a (compiled) fun is problematic. This is the main reason why I used {erl_eval, exprs, [Exprs, Bindings]}. Is there a simpler way to create a fun that can be run on a different node? A fun can be run on a different node if the module it was compiled in has a compatible version (it doesn?t have to be the exact same version). BR,Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Jan 28 17:19:37 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 28 Jan 2015 11:19:37 -0500 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <20150127190943.GA90397@ferdair.local> Message-ID: <20150128161934.GB90397@ferdair.local> On 01/28, Roberto Ostinelli wrote: > Here's the erl_crash.dump analysis, done with the nice Fred's script: > > analyzing erl_crash.dump, generated on: Wed Jan 28 13:59:36 2015 > > Slogan: Received SIGUSR1 > > File descriptors open: > === > UDP: 0 > TCP: 180071 > Files: 6 > --- > Total: 180077 > So uh, is 180,000 TCP ports the number you have or expect? Could you be running out of these and then massive erroring takes place or anything like that? What are the limits in place for your system? > Do you see anything wrong there? I honestly don't. > Nothing looks obviously wrong. At this point my number one suspsicion would be some process(es) that suddenly get lots of messages (links, monitors, node monitors, general events) and fill their mailboxes. This then prompts for GCs, which suspends the process, copies the mailbox to the heap, and then runs again. This kind of stuff can sometimes spiral out of control. From roberto@REDACTED Wed Jan 28 17:34:15 2015 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 17:34:15 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: <20150128161934.GB90397@ferdair.local> References: <20150127190943.GA90397@ferdair.local> <20150128161934.GB90397@ferdair.local> Message-ID: On Wed, Jan 28, 2015 at 5:19 PM, Fred Hebert wrote: > On 01/28, Roberto Ostinelli wrote: > > Here's the erl_crash.dump analysis, done with the nice Fred's script: > > > > analyzing erl_crash.dump, generated on: Wed Jan 28 13:59:36 2015 > > > > Slogan: Received SIGUSR1 > > > > File descriptors open: > > === > > UDP: 0 > > TCP: 180071 > > Files: 6 > > --- > > Total: 180077 > > > > So uh, is 180,000 TCP ports the number you have or expect? Could you be > running out of these and then massive erroring takes place or anything > like that? What are the limits in place for your system? > That's exactly the number of long lived connections to the system, so yes it's expected. ulimit (hard and soft) are set to 1M. > Do you see anything wrong there? I honestly don't. > > > > Nothing looks obviously wrong. At this point my number one suspsicion > would be some process(es) that suddenly get lots of messages (links, > monitors, node monitors, general events) and fill their mailboxes. This > then prompts for GCs, which suspends the process, copies the mailbox to > the heap, and then runs again. > > This kind of stuff can sometimes spiral out of control. > So, here's the thing. I got a bigger box, a 30GB. I'm launching the same test, and What I have is the following: https://cldup.com/1M4qzvbLp_-3000x3000.png Basically what happens is that the memory "ramp-up" happens in two phases. The second phase was the one previously making my VM blowup. With this bigger box, this phase continues but then stabilizes. Not sure why this happens in this way, but with a bigger box, as you can see, when 22.3 GB of memory are reached, everything is stable. This was confusing, but it simply looks like there's some time needed before the memory gets allocated. I'm assuming this has to do with fullsweep_after. With it set to 10, we get to the 22.3GB illustrated; with it set to 5, we get to 19.3GB (3GB less). I guess this is it for now. I can only thank you for your time. Any additional remarks? Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Wed Jan 28 17:47:13 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Wed, 28 Jan 2015 17:47:13 +0100 Subject: [erlang-questions] framework for a ecommerce shop Message-ID: <54C91291.2010002@home.nl> Hello, Can any recommend a good framework for a ecommerce shop I want to make later on when I more familiair with Erlang. I already tried ChicagoBoss but for some reason I cannot start a project. I have heard of Phoenix but that seems to be for elixer Regards Roelof From e@REDACTED Wed Jan 28 17:49:55 2015 From: e@REDACTED (e@REDACTED) Date: Wed, 28 Jan 2015 17:49:55 +0100 Subject: [erlang-questions] framework for a ecommerce shop In-Reply-To: <54C91291.2010002@home.nl> References: <54C91291.2010002@home.nl> Message-ID: <54C91333.9000404@bestmx.net> On 01/28/2015 05:47 PM, Roelof Wobben wrote: > Hello, > > Can any recommend a good framework for a ecommerce shop I want to make > later on when I more familiair with Erlang. as a person who dealt with all sorts of frameworks for more than 10 years, i strongly recommend you the VOID framework. > I already tried ChicagoBoss but for some reason I cannot start a project. > > I have heard of Phoenix but that seems to be for elixer > > Regards > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From daniel.widgren@REDACTED Wed Jan 28 18:11:17 2015 From: daniel.widgren@REDACTED (Daniel Hallin Widgren) Date: Wed, 28 Jan 2015 18:11:17 +0100 Subject: [erlang-questions] framework for a ecommerce shop In-Reply-To: <54C91333.9000404@bestmx.net> References: <54C91291.2010002@home.nl> <54C91333.9000404@bestmx.net> Message-ID: Nitrogen is a great tool to make a ecommerce shop. There is an example at erlang factory 2009 in London. That is made with nitrogen. Den 28 jan 2015 17:51 skrev "e@REDACTED" : > On 01/28/2015 05:47 PM, Roelof Wobben wrote: > >> Hello, >> >> Can any recommend a good framework for a ecommerce shop I want to make >> later on when I more familiair with Erlang. >> > > as a person who dealt with all sorts of frameworks for more than 10 years, > i strongly recommend you the VOID framework. > > > > I already tried ChicagoBoss but for some reason I cannot start a project. >> >> I have heard of Phoenix but that seems to be for elixer >> >> Regards >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Wed Jan 28 18:15:40 2015 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 28 Jan 2015 11:15:40 -0600 Subject: [erlang-questions] framework for a ecommerce shop In-Reply-To: References: <54C91291.2010002@home.nl> <54C91333.9000404@bestmx.net> Message-ID: Further, if you're interested in the Nitrogen route, I'll be releasing a payment framework for Nitrogen soonish (currently supporting Stripe, Paypal, and Authorize.net, but expandable). I'd be happy to give you access to the repo, if you shoot me a message off-list. It needs a fair bit of cleaning up and refactoring before open-sourcing, but it does work, and has been working in production for about 2 years. On Wed, Jan 28, 2015 at 11:11 AM, Daniel Hallin Widgren wrote: > Nitrogen is a great tool to make a ecommerce shop. There is an example at > erlang factory 2009 in London. That is made with nitrogen. > > Den 28 jan 2015 17:51 skrev "e@REDACTED" : > >> On 01/28/2015 05:47 PM, Roelof Wobben wrote: >>> >>> Hello, >>> >>> Can any recommend a good framework for a ecommerce shop I want to make >>> later on when I more familiair with Erlang. >> >> >> as a person who dealt with all sorts of frameworks for more than 10 years, >> i strongly recommend you the VOID framework. >> >> >> >>> I already tried ChicagoBoss but for some reason I cannot start a project. >>> >>> I have heard of Phoenix but that seems to be for elixer >>> >>> Regards >>> >>> Roelof >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From schneider@REDACTED Wed Jan 28 18:31:46 2015 From: schneider@REDACTED (Frans Schneider) Date: Wed, 28 Jan 2015 18:31:46 +0100 Subject: [erlang-questions] Implement parallel sessions In-Reply-To: References: <54C77B7A.4040401@xs4all.nl> Message-ID: <54C91D02.2010602@xs4all.nl> I did some testing with your suggestions and would love to get some feedback on what I did. The code is here at https://github.com/schnef/tst. Thanks, Frans On 01/27/2015 03:10 PM, Sean Cribbs wrote: > For those processes that are per-user, or otherwise spin up and do > some work and then go away, we use the simple_one_for_one strategy in > the supervisor. In your case, the simple_one_for_one supervisor would > start the master process of the session (whether it be a supervisor or > something else). > If the master process of the user session is a supervisor, you can > create whatever state is needed to be shared (an ETS table perhaps, or > maybe its own pid) and then pass it to the children when starting. > This supervisor would be a one_for_one, one_for_all, or rest_for_one > strategy, depending on the dependencies between its children. > I wonder whether a single process per session will do (or two, one for > user connection and one for the proxy connection), with different > functionality being kept simply in multiple modules rather than FSMs. > For example, if your connection process handling the TCP socket simply > shuttles data to and from, you might consider collapsing it into the > process that's next up the stack. I would like to keep the process separate since each fulfills a very specific function and their implementation is very clean now. Merging modules would make them much harder to maintain / write / understand. Also, using FSM's, especially for the crypt module turns out to be very handy. The modules now are quite small but merging them would make it one huge beast. > > On Tue, Jan 27, 2015 at 6:14 AM, Imants Cekusins > wrote: > > supervisors are meant to supervise only afaik, not to store any state > or do anything else. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Sean Cribbs > > Sr. Software Engineer > Basho Technologies, Inc. > http://basho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jan 28 18:47:26 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 28 Jan 2015 18:47:26 +0100 Subject: [erlang-questions] Implement parallel sessions In-Reply-To: <54C77B7A.4040401@xs4all.nl> References: <54C77B7A.4040401@xs4all.nl> Message-ID: On Tue, Jan 27, 2015 at 12:50 PM, Frans Schneider wrote: > I am implementing a kind of IM proxy which currently handles one single > session for one user. The session is implemented as a stack of processes > with from top to bottom the session control process, a stanza encoder / > decoder process, an en- / decrypt process and finally a process which > handles the tcp connection. Each process is a FSM and holds state such as > RC4 encryption state, tcp data etc. The session is controlled by a > supervisor which is part of a overall supervisor structure. A session will > be open for a long period. > Two immediate hunches from this: * Do you actually need a process chain? You can still write code in modules and chain them together, keeping state separate. But without spawning new processes. Unless you can a distinct advantage where the processes need to do things on their own, it is probably easier to keep them together. * RC4 is broken. Do never ever use it for anything, please. Pick xsalsa20 or chacha20, both derived from the Salsa20 cipher. Or something else. The cryptographic community is widely believing adversaries are able to break RC4, which means it is a ticking bomb for anyone using it. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Wed Jan 28 19:11:14 2015 From: schneider@REDACTED (Frans Schneider) Date: Wed, 28 Jan 2015 19:11:14 +0100 Subject: [erlang-questions] Implement parallel sessions In-Reply-To: References: <54C77B7A.4040401@xs4all.nl> Message-ID: <54C92642.2060306@xs4all.nl> On 01/28/2015 06:47 PM, Jesper Louis Andersen wrote: > > On Tue, Jan 27, 2015 at 12:50 PM, Frans Schneider > wrote: > > I am implementing a kind of IM proxy which currently handles one > single session for one user. The session is implemented as a stack > of processes with from top to bottom the session control process, > a stanza encoder / decoder process, an en- / decrypt process and > finally a process which handles the tcp connection. Each process > is a FSM and holds state such as RC4 encryption state, tcp data > etc. The session is controlled by a supervisor which is part of a > overall supervisor structure. A session will be open for a long > period. > > > Two immediate hunches from this: > > * Do you actually need a process chain? You can still write code in > modules and chain them together, keeping state separate. But without > spawning new processes. Unless you can a distinct advantage where the > processes need to do things on their own, it is probably easier to > keep them together. What do you mean by chaining? I want to keep stuff in separate modules since the code would get too complicated to manage, but I guess it would do no harm if it is only one process. > * RC4 is broken. Do never ever use it for anything, please. Pick > xsalsa20 or chacha20, both derived from the Salsa20 cipher. Or > something else. The cryptographic community is widely believing > adversaries are able to break RC4, which means it is a ticking bomb > for anyone using it. > Yep, RC4 is broken, but that's what some HUGE parties are using... Say What? > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Jan 28 19:37:49 2015 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 28 Jan 2015 10:37:49 -0800 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <20150127190943.GA90397@ferdair.local> <20150128161934.GB90397@ferdair.local> Message-ID: <54C92C7D.2080605@gmail.com> On 01/28/2015 08:34 AM, Roberto Ostinelli wrote: > On Wed, Jan 28, 2015 at 5:19 PM, Fred Hebert > wrote: > > On 01/28, Roberto Ostinelli wrote: > > Here's the erl_crash.dump analysis, done with the nice Fred's script: > > > > analyzing erl_crash.dump, generated on: Wed Jan 28 13:59:36 2015 > > > > Slogan: Received SIGUSR1 > > > > File descriptors open: > > === > > UDP: 0 > > TCP: 180071 > > Files: 6 > > --- > > Total: 180077 > > > > So uh, is 180,000 TCP ports the number you have or expect? Could you be > running out of these and then massive erroring takes place or anything > like that? What are the limits in place for your system? > > > That's exactly the number of long lived connections to the system, so yes it's expected. > ulimit (hard and soft) are set to 1M. > > > Do you see anything wrong there? I honestly don't. > > > > Nothing looks obviously wrong. At this point my number one suspsicion > would be some process(es) that suddenly get lots of messages (links, > monitors, node monitors, general events) and fill their mailboxes. This > then prompts for GCs, which suspends the process, copies the mailbox to > the heap, and then runs again. > > This kind of stuff can sometimes spiral out of control. > > > > So, here's the thing. I got a bigger box, a 30GB. I'm launching the same test, and What I have is the following: > https://cldup.com/1M4qzvbLp_-3000x3000.png > > Basically what happens is that the memory "ramp-up" happens in two phases. The second phase was the one previously making my VM blowup. > With this bigger box, this phase continues but then stabilizes. > > Not sure why this happens in this way, but with a bigger box, as you can see, when 22.3 GB of memory are reached, everything is stable. > This was confusing, but it simply looks like there's some time needed before the memory gets allocated. I'm assuming this has to do with fullsweep_after. With it set to 10, we get to the 22.3GB illustrated; with it set to 5, we get to 19.3GB (3GB less). > > I guess this is it for now. I can only thank you for your time. > > Any additional remarks? > > Best, > r. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions As I mentioned previously, your long-lived processes are generating too much garbage with your current architecture to be able to scale as high as you are attempting. You should be able to see the problem with less load and usage of http://www.erlang.org/doc/man/instrument.html which would then lead to source code changes that split monolithic Erlang processes that generate too much garbage (long-lived) into smaller ones that are able to trigger garbage collection faster (short-lived). The Erlang process message queues are a secondary concern due to the possibility of overloading bottleneck processes, but that problem can be discovered and solved with this approach. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabrek@REDACTED Wed Jan 28 20:02:14 2015 From: mabrek@REDACTED (Anton Lebedevich) Date: Wed, 28 Jan 2015 23:02:14 +0400 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <20150127190943.GA90397@ferdair.local> <20150128161934.GB90397@ferdair.local> Message-ID: On Wed, Jan 28, 2015 at 7:34 PM, Roberto Ostinelli wrote: > So, here's the thing. I got a bigger box, a 30GB. I'm launching the same > test, and What I have is the following: > https://cldup.com/1M4qzvbLp_-3000x3000.png > > Basically what happens is that the memory "ramp-up" happens in two phases. > The second phase was the one previously making my VM blowup. > With this bigger box, this phase continues but then stabilizes. > > Not sure why this happens in this way, but with a bigger box, as you can > see, when 22.3 GB of memory are reached, everything is stable. > This was confusing, but it simply looks like there's some time needed before > the memory gets allocated. I'm assuming this has to do with fullsweep_after. > With it set to 10, we get to the 22.3GB illustrated; with it set to 5, we > get to 19.3GB (3GB less). > > I guess this is it for now. I can only thank you for your time. > > Any additional remarks? The graph looks really weird for me assuming that the load applied to the system is stable. Why does it go down for a short time and then jumps higher than it was? Usually memory graphs increase up to some level and then remain stable under controlled constant load. I've tried to multiply 120000 (number of processes) by 142896 (memory usage of connection handling process) and got 16Gb. So it looks like not all connection handling processes reach size of 142896 after applying load. In this situation I'd: 1) double check that all processes get the same load over time 2) compare process size histograms and allocator stats/fragmentation before/after drop down and before/after jump up Regards, Anton. From erlangerintraining@REDACTED Wed Jan 28 20:09:29 2015 From: erlangerintraining@REDACTED (Kristoffer Brown) Date: Wed, 28 Jan 2015 12:09:29 -0700 Subject: [erlang-questions] Good example of cross-node messaging in cloud Message-ID: I'm trying to develop a simple sample application that I can deploy to ec2 to test cross-node messaging. Currently I am using mnesia for clustering which is far from ideal in this environment. What I would like to do is have each node communicate to the other with some reliability built-in. (If there is a network partition, retry later, etc.) I'm well underway to creating this standalone app that will just read the nodes of the cluster from config and use net_adm:ping to cluster and rpc:multicall to sent the messages (retrying nodes that report failures). This seems far from resilient. And probably not even the best practice for such a situation. Any guidance would be greatly appreciated. If it already exists, a sample of some sort would be ideal. TIA! -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Wed Jan 28 20:09:45 2015 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 28 Jan 2015 20:09:45 +0100 Subject: [erlang-questions] Garbage Collection, BEAM memory and Erlang memory In-Reply-To: References: <20150127190943.GA90397@ferdair.local> <20150128161934.GB90397@ferdair.local> Message-ID: <360A6593-9F4C-4695-B012-9373A404567F@widetag.com> That's a valid point, and I can clarify. During the first phase, all the long lived connections are made. During the second phase, newly-created short lived connections send messages to the long lived ones. The long lived connection are all connected when the first memory increase phase ends, which is when the short-lived connections start sending messages. What is unclear to me is why the second memory increase phase happens quite some time after these short lived processes have started sending messages. > On 28/gen/2015, at 20:02, Anton Lebedevich wrote: > > The graph looks really weird for me assuming that the load applied to > the system is stable. Why does it go down for a short time and then > jumps higher than it was? From nayibor@REDACTED Wed Jan 28 20:56:55 2015 From: nayibor@REDACTED (Nuku Ameyibor) Date: Wed, 28 Jan 2015 11:56:55 -0800 Subject: [erlang-questions] framework for a ecommerce shop Message-ID: <7347329772179846831@unknownmsgid> Any links to the VOID framework . Any links to tutorials would also be nice . From: e@REDACTED Sent: ?28/?01/?2015 16:51 To: erlang-questions@REDACTED Subject: Re: [erlang-questions] framework for a ecommerce shop On 01/28/2015 05:47 PM, Roelof Wobben wrote: > Hello, > > Can any recommend a good framework for a ecommerce shop I want to make > later on when I more familiair with Erlang. as a person who dealt with all sorts of frameworks for more than 10 years, i strongly recommend you the VOID framework. > I already tried ChicagoBoss but for some reason I cannot start a project. > > I have heard of Phoenix but that seems to be for elixer > > Regards > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From matthieu.tourne@REDACTED Thu Jan 29 04:19:24 2015 From: matthieu.tourne@REDACTED (Matthieu Tourne) Date: Wed, 28 Jan 2015 19:19:24 -0800 Subject: [erlang-questions] Hidden node and topic subscription rpc's Message-ID: Hi, I started using py_interface as a layer to do rpc into an Erlang system, receive and send message, and eventually provide functionality to the that Erlang system. One of the functions on the Erlang side would be : handle_call({subscribe, Channel}, From, State) -> %% Saves From as a subscriber of this channel into the State %% When something will arrive on the Channel From will get a message. The problem is when I'm sending an rpc:call via py_interface (a call message on the rex mailbox essentially). It seems that the pid associated to that rpc essentially disappears right when the rpc is completed. As I gathered from this example [1], an alternative way to do this would be to rewrite my handle_call this way : handle_call({subscribe, Channel, SubscribingPid}, _From, State) -> %% Saves SubscribingPid instead of From. And I would pass in the Pid of an active mailbox from my py_interface hidden node. My question is two fold, would there be a way to keep a live Pid in the system (maybe [2] answers this problem as a byproduct), for longer than getting a result to that rpc call. Or would a proper way to do this be to have a proxy process to make the link between rpc's and a live mailbox in my hidden node. Also, is a hidden node the best way of achieving all of this ? Any working example of what I'm trying to achieve would be greatly appreciated, Thank you! Matthieu [1] https://github.com/adamtornhill/tinch_pp/blob/master/test/chat_server.erl [2] https://github.com/cloudant/rexi -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jan 29 06:59:25 2015 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 28 Jan 2015 21:59:25 -0800 Subject: [erlang-questions] Hidden node and topic subscription rpc's In-Reply-To: References: Message-ID: <54C9CC3D.7000504@gmail.com> On 01/28/2015 07:19 PM, Matthieu Tourne wrote: > Hi, > > I started using py_interface as a layer to do rpc into an Erlang system, receive and send message, and eventually provide functionality to the that Erlang system. > > One of the functions on the Erlang side would be : > > handle_call({subscribe, Channel}, From, State) -> > %% Saves From as a subscriber of this channel into the State > %% When something will arrive on the Channel From will get a message. > > The problem is when I'm sending an rpc:call via py_interface (a call message on the rex mailbox essentially). It seems that the pid associated to that rpc essentially disappears right when the rpc is completed. > > As I gathered from this example [1], an alternative way to do this would be to rewrite my handle_call this way : > > handle_call({subscribe, Channel, SubscribingPid}, _From, State) -> > %% Saves SubscribingPid instead of From. > > And I would pass in the Pid of an active mailbox from my py_interface hidden node. > > My question is two fold, would there be a way to keep a live Pid in the system (maybe [2] answers this problem as a byproduct), for longer than getting a result to that rpc call. > Or would a proper way to do this be to have a proxy process to make the link between rpc's and a live mailbox in my hidden node. > Also, is a hidden node the best way of achieving all of this ? py_interface makes the running Python interpreter into a hidden node, similar to a cnode (http://www.erlang.org/doc/tutorial/cnode.html), just using Python source code (like jinterface does with Java source code). You could probably modify the py_interface source code to run as an Erlang port (http://www.erlang.org/doc/tutorial/c_port.html). In both cases, all Python usage will have a single connection bottleneck (a single socket for a cnode and a single pair of pipes for a port). Depending on the Python source code, you could argue that it will never matter due to the global interpreter lock. However, if you do care about Python latency, it is more efficient to use the Python CloudI API. To do RPC you would send a service request to another CloudI service (based on your example, an Erlang CloudI service). An example of a Python service is at http://cloudi.org/#Python to show how a service request can be handled in Python (adding a send_sync CloudI API function call (http://cloudi.org/api.html#1_send_sync) would be the same as doing RPC). The Python CloudI API is implemented only in Python in the cloudi module and with Python/C integration in the cloudi_c module (the quickest Python integration). > > Any working example of what I'm trying to achieve would be greatly appreciated, I don't use py_interface, but the example you are probably looking for is https://github.com/tomas-abrahamsson/py_interface/blob/master/examples/test_remote_exec.py . You specify the remote node, module name, and function name on the command line of the script and it performs the RPC call for you, converting types in the process. > > Thank you! > Matthieu > > [1] https://github.com/adamtornhill/tinch_pp/blob/master/test/chat_server.erl > [2] https://github.com/cloudant/rexi > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomas.abrahamsson@REDACTED Thu Jan 29 09:16:56 2015 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Thu, 29 Jan 2015 09:16:56 +0100 Subject: [erlang-questions] Hidden node and topic subscription rpc's In-Reply-To: References: Message-ID: > I started using py_interface as a layer to do rpc into an Erlang system, > receive and send message, and eventually provide functionality to the that > Erlang system. > > One of the functions on the Erlang side would be : > > handle_call({subscribe, Channel}, From, State) -> > %% Saves From as a subscriber of this channel into the State > %% When something will arrive on the Channel From will get a message. > > The problem is when I'm sending an rpc:call via py_interface (a call message > on the rex mailbox essentially). It seems that the pid associated to that > rpc essentially disappears right when the rpc is completed. I think this is since the serving rex/rpc process spawns off a process to handle the rpc request, in order to not block the rex/rpc server, if I remember correctly. BRs Tomas From andre.graf@REDACTED Thu Jan 29 12:19:28 2015 From: andre.graf@REDACTED (Andre Graf) Date: Thu, 29 Jan 2015 12:19:28 +0100 Subject: [erlang-questions] [ANN] EPMDPXY: A Proxy for the Erlang Port Mapper Deamon Message-ID: <54CA1740.40809@erl.io> Hello list, While testing some distributed Erlang features I wanted to quickly simulate network partitions to see how my distributed Erlang application reacts. I came up with this little project, pretty much alpha I must admit, which simulates the basic functionality of EPMD, just enough to make a local Erlang cluster work. With one exception though, instead of replying the remote listener port when requested, it spawns an internal listener and replies the new (random) port number instead. The newly spawned listener process accepts one single connection and connects to the 'real' remote listener, acting as a proxy between the two Erlang nodes. This enables to 'cut' connections between the nodes (on the TCP level though). Of course this is not a replacement for real hardware damage ;). If this sounds interesting, please check it out on https://github.com/dergraf/epmdpxy Cheers, Andre From serge@REDACTED Thu Jan 29 15:20:25 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 09:20:25 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll Message-ID: Is there a way to hook a file descriptor from within a NIF function to the select/epoll/poll loop managed by the emulator with a callback invoked in the "NIF-land" on activity detected on the file descriptor? I am looking for similar functionality available to NIF functions that is available when writing drivers using driver_select() call (*). More specifically, I have an eventfd file descriptor that I'd like to be notified about without allocating a separate OS thread to poll on that FD and return results by sending a message to a given Erlang Pid. Thanks, Serge (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select -------------- next part -------------- An HTML attachment was scrubbed... URL: From andra.dinu@REDACTED Thu Jan 29 12:59:18 2015 From: andra.dinu@REDACTED (Andra Dinu) Date: Thu, 29 Jan 2015 11:59:18 +0000 Subject: [erlang-questions] Erlang User Conference 2015 - Call for Talks Message-ID: Hi all, The Erlang User Conference 2015 will take place in Stockholm on 11-12 June, with one day of tutorials on the 10th and 3 training days between 8-10 June. The Call For Talks is now open. If you made an interesting innovation, open-source application, tool or product with Erlang/OTP/Elixir or you used Erlang or Elixir in a real-world project and wish to share your work with the community, please submit your talk http://www.erlang-factory.com/euc2015 Best, Andra *Andra Dinu* Community & Social Largest Erlang event in the US: Erlang Factory SF Bay Area 26-27 March . Scholarships available. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Thu Jan 29 15:44:59 2015 From: tony@REDACTED (Tony Rogvall) Date: Thu, 29 Jan 2015 15:44:59 +0100 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: Message-ID: <6B3D62F7-BD13-48F3-AA36-14A8E72105F2@rogvall.se> Why not write a small driver that can watch a number of file descriptors, Get fd from a nif pass it to the driver_select_driver may be overkill but fun. Could be a useful thing. I am working on something that may someday be capable of doing this, ( https://github.com/tonyrog/niffed ) It is a a C library that allow you to call (linked) nif library from driver code, The marshalling code is actually building VM heap compatible data that can be processed by the nif code, only by offsetting some pointers :-) It is even possible to use the same shared object as both nif and driver! /Tony > On 29 jan 2015, at 15:20, Serge Aleynikov wrote: > > Is there a way to hook a file descriptor from within a NIF function to the select/epoll/poll loop managed by the emulator with a callback invoked in the "NIF-land" on activity detected on the file descriptor? > > I am looking for similar functionality available to NIF functions that is available when writing drivers using driver_select() call (*). More specifically, I have an eventfd file descriptor that I'd like to be notified about without allocating a separate OS thread to poll on that FD and return results by sending a message to a given Erlang Pid. > > Thanks, > > Serge > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From serge@REDACTED Thu Jan 29 16:01:07 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 10:01:07 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: <6B3D62F7-BD13-48F3-AA36-14A8E72105F2@rogvall.se> References: <6B3D62F7-BD13-48F3-AA36-14A8E72105F2@rogvall.se> Message-ID: Thanks Tony for the tip. I was hoping there is some other way to call the emulator's internals without resorting to having to implement both a driver and a NIF library for this purpose. It seems like it's somewhat an oversight in the NIF API design that needs to be addressed - there should be a NIF capable of associating an FD with a callback or else, as you mentioned, the implementation of this becomes more complicated than necessary when integrating with external C libraries. On Thu, Jan 29, 2015 at 9:44 AM, Tony Rogvall wrote: > Why not write a small driver that can watch a number of file descriptors, > Get fd from a nif pass it to the driver_select_driver may be overkill but > fun. > > Could be a useful thing. > > I am working on something that may someday be capable of doing this, > ( https://github.com/tonyrog/niffed ) > It is a a C library that allow you to call (linked) nif library from > driver code, > The marshalling code is actually building VM heap compatible data that > can be processed by the nif code, only by offsetting some pointers :-) > It is even possible to use the same shared object as both nif and driver! > > /Tony > > On 29 jan 2015, at 15:20, Serge Aleynikov wrote: > > Is there a way to hook a file descriptor from within a NIF function to the > select/epoll/poll loop managed by the emulator with a callback invoked in > the "NIF-land" on activity detected on the file descriptor? > > I am looking for similar functionality available to NIF functions that is > available when writing drivers using driver_select() call (*). More > specifically, I have an eventfd file descriptor that I'd like to be > notified about without allocating a separate OS thread to poll on that FD > and return results by sending a message to a given Erlang Pid. > > Thanks, > > Serge > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Thu Jan 29 16:10:56 2015 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 29 Jan 2015 09:10:56 -0600 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <6B3D62F7-BD13-48F3-AA36-14A8E72105F2@rogvall.se> Message-ID: You are talking about NIF Native Processes: http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf On Thu, Jan 29, 2015 at 9:01 AM, Serge Aleynikov wrote: > Thanks Tony for the tip. I was hoping there is some other way to call the > emulator's internals without resorting to having to implement both a driver > and a NIF library for this purpose. It seems like it's somewhat an > oversight in the NIF API design that needs to be addressed - there should > be a NIF capable of associating an FD with a callback or else, as you > mentioned, the implementation of this becomes more complicated than > necessary when integrating with external C libraries. > > On Thu, Jan 29, 2015 at 9:44 AM, Tony Rogvall wrote: > >> Why not write a small driver that can watch a number of file descriptors, >> Get fd from a nif pass it to the driver_select_driver may be overkill but >> fun. >> >> Could be a useful thing. >> >> I am working on something that may someday be capable of doing this, >> ( https://github.com/tonyrog/niffed ) >> It is a a C library that allow you to call (linked) nif library from >> driver code, >> The marshalling code is actually building VM heap compatible data that >> can be processed by the nif code, only by offsetting some pointers :-) >> It is even possible to use the same shared object as both nif and driver! >> >> /Tony >> >> On 29 jan 2015, at 15:20, Serge Aleynikov wrote: >> >> Is there a way to hook a file descriptor from within a NIF function to >> the select/epoll/poll loop managed by the emulator with a callback invoked >> in the "NIF-land" on activity detected on the file descriptor? >> >> I am looking for similar functionality available to NIF functions that is >> available when writing drivers using driver_select() call (*). More >> specifically, I have an eventfd file descriptor that I'd like to be >> notified about without allocating a separate OS thread to poll on that FD >> and return results by sending a message to a given Erlang Pid. >> >> Thanks, >> >> Serge >> >> (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Thu Jan 29 17:37:52 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 11:37:52 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <6B3D62F7-BD13-48F3-AA36-14A8E72105F2@rogvall.se> Message-ID: Actually something more simple that only allows for NIF reactivity to file descriptor events... I understand that native processes are quite some time away in the pipeline. On Thu, Jan 29, 2015 at 10:10 AM, Daniel Goertzen wrote: > You are talking about NIF Native Processes: > http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf > > > > On Thu, Jan 29, 2015 at 9:01 AM, Serge Aleynikov > wrote: > >> Thanks Tony for the tip. I was hoping there is some other way to call >> the emulator's internals without resorting to having to implement both a >> driver and a NIF library for this purpose. It seems like it's somewhat an >> oversight in the NIF API design that needs to be addressed - there should >> be a NIF capable of associating an FD with a callback or else, as you >> mentioned, the implementation of this becomes more complicated than >> necessary when integrating with external C libraries. >> >> On Thu, Jan 29, 2015 at 9:44 AM, Tony Rogvall wrote: >> >>> Why not write a small driver that can watch a number of file descriptors, >>> Get fd from a nif pass it to the driver_select_driver may be overkill >>> but fun. >>> >>> Could be a useful thing. >>> >>> I am working on something that may someday be capable of doing this, >>> ( https://github.com/tonyrog/niffed ) >>> It is a a C library that allow you to call (linked) nif library from >>> driver code, >>> The marshalling code is actually building VM heap compatible data that >>> can be processed by the nif code, only by offsetting some pointers :-) >>> It is even possible to use the same shared object as both nif and driver! >>> >>> /Tony >>> >>> On 29 jan 2015, at 15:20, Serge Aleynikov wrote: >>> >>> Is there a way to hook a file descriptor from within a NIF function to >>> the select/epoll/poll loop managed by the emulator with a callback invoked >>> in the "NIF-land" on activity detected on the file descriptor? >>> >>> I am looking for similar functionality available to NIF functions that >>> is available when writing drivers using driver_select() call (*). More >>> specifically, I have an eventfd file descriptor that I'd like to be >>> notified about without allocating a separate OS thread to poll on that FD >>> and return results by sending a message to a given Erlang Pid. >>> >>> Thanks, >>> >>> Serge >>> >>> (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Thu Jan 29 18:05:43 2015 From: egil@REDACTED (=?windows-1252?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 29 Jan 2015 18:05:43 +0100 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: Message-ID: <54CA6867.8020905@erlang.org> On 2015-01-29 15:20, Serge Aleynikov wrote: > Is there a way to hook a file descriptor from within a NIF function to > the select/epoll/poll loop managed by the emulator with a callback > invoked in the "NIF-land" on activity detected on the file descriptor? > > I am looking for similar functionality available to NIF functions that > is available when writing drivers using driver_select() call (*). > More specifically, I have an eventfd file descriptor that I'd like to > be notified about without allocating a separate OS thread to poll on > that FD and return results by sending a message to a given Erlang Pid. I think you have discovered why we have drivers in Erlang. This is exactly the use case for drivers. Normally you don't do much in a driver. You react to an event and transfer the binary to erlang space and to what should be done in Erlang. Don't try to program Erlang in C (like erl_interface), do it in Erlang. And no, we don't have native processes (in which case drivers would be obsolete). And no, it's not an oversight in the NIF API, not really. You could see native processes as an extension to NIFs but it is something completely different. In some specific cases you could transfer a fd (socket) to Erlangs I/O system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that open fd:s in NIFs which then get transferred to gen_tcp in this manner. // Bj?rn-Egil > > Thanks, > > Serge > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthieu.tourne@REDACTED Thu Jan 29 18:15:00 2015 From: matthieu.tourne@REDACTED (Matthieu Tourne) Date: Thu, 29 Jan 2015 09:15:00 -0800 Subject: [erlang-questions] Hidden node and topic subscription rpc's In-Reply-To: <54C9CC3D.7000504@gmail.com> References: <54C9CC3D.7000504@gmail.com> Message-ID: On Wed, Jan 28, 2015 at 9:59 PM, Michael Truog wrote: > On 01/28/2015 07:19 PM, Matthieu Tourne wrote: > > Hi, > > I started using py_interface as a layer to do rpc into an Erlang system, > receive and send message, and eventually provide functionality to the that > Erlang system. > > One of the functions on the Erlang side would be : > > handle_call({subscribe, Channel}, From, State) -> > %% Saves From as a subscriber of this channel into the State > %% When something will arrive on the Channel From will get a message. > > The problem is when I'm sending an rpc:call via py_interface (a call > message on the rex mailbox essentially). It seems that the pid associated > to that rpc essentially disappears right when the rpc is completed. > > As I gathered from this example [1], an alternative way to do this would > be to rewrite my handle_call this way : > > handle_call({subscribe, Channel, SubscribingPid}, _From, State) -> > %% Saves SubscribingPid instead of From. > > And I would pass in the Pid of an active mailbox from my py_interface > hidden node. > > My question is two fold, would there be a way to keep a live Pid in the > system (maybe [2] answers this problem as a byproduct), for longer than > getting a result to that rpc call. > Or would a proper way to do this be to have a proxy process to make the > link between rpc's and a live mailbox in my hidden node. > Also, is a hidden node the best way of achieving all of this ? > > py_interface makes the running Python interpreter into a hidden node, > similar to a cnode (http://www.erlang.org/doc/tutorial/cnode.html), just > using Python source code (like jinterface does with Java source code). You > could probably modify the py_interface source code to run as an Erlang port > (http://www.erlang.org/doc/tutorial/c_port.html). In both cases, all > Python usage will have a single connection bottleneck (a single socket for > a cnode and a single pair of pipes for a port). Depending on the Python > source code, you could argue that it will never matter due to the global > interpreter lock. > > However, if you do care about Python latency, it is more efficient to use > the Python CloudI API. To do RPC you would send a service request to > another CloudI service (based on your example, an Erlang CloudI service). > An example of a Python service is at http://cloudi.org/#Python to show > how a service request can be handled in Python (adding a send_sync CloudI > API function call (http://cloudi.org/api.html#1_send_sync) would be the > same as doing RPC). The Python CloudI API is implemented only in Python in > the cloudi module and with Python/C integration in the cloudi_c module (the > quickest Python integration). > > I was actually looking at CloudI just yesterday! It looks very interesting, and seems to work like magic in my use case [1] :) I would have couple questions about it, what powers the api system ? Reading the code briefly I saw some Erlang external term format, but it wasn't clear which channel the protocol used (socket, pipes, etc ). Also I was wondering if everything would need to become a cloudi_service to be consumed seamlessly by an external API, I'm working with existing code that already has a lot of gen_server patterns, if I have to layer on top this could be a burden. Finally not running on windows could be a problem, but I'm guessing I could rip out just CloudI API part of CloudI (it's a big repo) and make sure that works. [1] https://github.com/CloudI/CloudI/tree/develop/doc#python -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jan 29 18:54:14 2015 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 29 Jan 2015 09:54:14 -0800 Subject: [erlang-questions] Hidden node and topic subscription rpc's In-Reply-To: References: <54C9CC3D.7000504@gmail.com> Message-ID: <54CA73C6.8000903@gmail.com> On 01/29/2015 09:15 AM, Matthieu Tourne wrote: > > > On Wed, Jan 28, 2015 at 9:59 PM, Michael Truog > wrote: > > On 01/28/2015 07:19 PM, Matthieu Tourne wrote: >> Hi, >> >> I started using py_interface as a layer to do rpc into an Erlang system, receive and send message, and eventually provide functionality to the that Erlang system. >> >> One of the functions on the Erlang side would be : >> >> handle_call({subscribe, Channel}, From, State) -> >> %% Saves From as a subscriber of this channel into the State >> %% When something will arrive on the Channel From will get a message. >> >> The problem is when I'm sending an rpc:call via py_interface (a call message on the rex mailbox essentially). It seems that the pid associated to that rpc essentially disappears right when the rpc is completed. >> >> As I gathered from this example [1], an alternative way to do this would be to rewrite my handle_call this way : >> >> handle_call({subscribe, Channel, SubscribingPid}, _From, State) -> >> %% Saves SubscribingPid instead of From. >> >> And I would pass in the Pid of an active mailbox from my py_interface hidden node. >> >> My question is two fold, would there be a way to keep a live Pid in the system (maybe [2] answers this problem as a byproduct), for longer than getting a result to that rpc call. >> Or would a proper way to do this be to have a proxy process to make the link between rpc's and a live mailbox in my hidden node. >> Also, is a hidden node the best way of achieving all of this ? > py_interface makes the running Python interpreter into a hidden node, similar to a cnode (http://www.erlang.org/doc/tutorial/cnode.html), just using Python source code (like jinterface does with Java source code). You could probably modify the py_interface source code to run as an Erlang port (http://www.erlang.org/doc/tutorial/c_port.html). In both cases, all Python usage will have a single connection bottleneck (a single socket for a cnode and a single pair of pipes for a port). Depending on the Python source code, you could argue that it will never matter due to the global interpreter lock. > > However, if you do care about Python latency, it is more efficient to use the Python CloudI API. To do RPC you would send a service request to another CloudI service (based on your example, an Erlang CloudI service). An example of a Python service is at http://cloudi.org/#Python to show how a service request can be handled in Python (adding a send_sync CloudI API function call (http://cloudi.org/api.html#1_send_sync) would be the same as doing RPC). The Python CloudI API is implemented only in Python in the cloudi module and with Python/C integration in the cloudi_c module (the quickest Python integration). > > > I was actually looking at CloudI just yesterday! > It looks very interesting, and seems to work like magic in my use case [1] :) > > I would have couple questions about it, what powers the api system ? > Reading the code briefly I saw some Erlang external term format, but it wasn't clear which channel the protocol used (socket, pipes, etc ). The Erlang external term format is used for sending a CloudI service request to the Erlang VM from an external service. For an external service the service configuration sets the count_thread (number of threads to use for CloudI API objects concurrently) and count_process (number of OS processes for the external service instance, each using the same number of threads). When an external service is started, count_thread * count_process sockets are created, so there is concurrent usage of the CloudI API among all external service threads. The socket type (i.e., 'protocol' described at http://cloudi.org/api.html#2_services_add) is either inet tcp (tcp), unix domain socket tcp (local or default), or inet udp (udp (only really used for testing)). > > Also I was wondering if everything would need to become a cloudi_service to be consumed seamlessly by an external API, I'm working with existing code that already has a lot of gen_server patterns, if I have to layer on top this could be a burden. Anything that handles CloudI service requests needs to be a CloudI service. External Erlang source code can use the cloudi module to send to CloudI services without being services (it can just be a normal Erlang process that keeps the Context data created by cloudi:new/0) > > Finally not running on windows could be a problem, but I'm guessing I could rip out just CloudI API part of CloudI (it's a big repo) and make sure that works. The CloudI source code is portable, so making it work on Windows shouldn't be a problem. However, the main reason I have avoided trying to make Windows work is just due to no one using Windows for anything fault-tolerant (due to Window's historical instability) or any serious server usage. I understand people have tried to (like the London Stock Exchange: http://www.computerworld.com/article/2467082/data-center/london-stock-exchange-to-abandon-failed-windows-platform.html and the Microsoft sponsored work with MPI clusters on Windows). If Windows was somehow important for testing, CloudI could be used on Windows. The simplest approach would likely be with Cygwin, but I have not attempted that yet. > > [1] https://github.com/CloudI/CloudI/tree/develop/doc#python -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Thu Jan 29 20:04:09 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Thu, 29 Jan 2015 20:04:09 +0100 Subject: [erlang-questions] how to add a tuple to a list. Message-ID: <54CA8429.9080708@home.nl> Hello, For a exercise I have to make a sort of database. IM trying to make the write function which has to work like this : Db1 = db:write(francesco, london, Db). [{francesco,london}] So i have to make a tuple of the data and add it in a list. I thought I could do something like this : write(Key, Data, Db) -> [ {key, data} | Db ] but then I see some error messages. Can someone give me a tip how to do this ? Roelof From e@REDACTED Thu Jan 29 20:06:37 2015 From: e@REDACTED (e@REDACTED) Date: Thu, 29 Jan 2015 20:06:37 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8429.9080708@home.nl> References: <54CA8429.9080708@home.nl> Message-ID: <54CA84BD.6090908@bestmx.net> > So i have to make a tuple of the data and add it in a list. > > I thought I could do something like this : > > write(Key, Data, Db) -> > [ {key, data} | Db ] > > but then I see some error messages. what messages? as far as i can _theorize_ the most probably your Db is not a list, and it should be. From r.wobben@REDACTED Thu Jan 29 20:14:14 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Thu, 29 Jan 2015 20:14:14 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA84BD.6090908@bestmx.net> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> Message-ID: <54CA8686.4020701@home.nl> An HTML attachment was scrubbed... URL: From imantc@REDACTED Thu Jan 29 20:14:55 2015 From: imantc@REDACTED (Imants Cekusins) Date: Thu, 29 Jan 2015 20:14:55 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA84BD.6090908@bestmx.net> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> Message-ID: write/3 definition is good. call it for the first time like this: Db1 = db:write(francesco, london, []). [{francesco,london}]. then Db2 = db:write(name2, city2, Db1). From essen@REDACTED Thu Jan 29 20:15:28 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 29 Jan 2015 20:15:28 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8686.4020701@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> Message-ID: <54CA86D0.7010107@ninenines.eu> On 01/29/2015 08:14 PM, Roelof Wobben wrote: > write(Key, Element, Db) -> > [ [{key, data}] | Db ] > > > and this the output of c(db). > > > 3> c(db). > db.erl:12: syntax error before: Tells you there's a missing dot there. -- Lo?c Hoguin http://ninenines.eu From rpettit@REDACTED Thu Jan 29 20:16:48 2015 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 29 Jan 2015 13:16:48 -0600 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8686.4020701@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> Message-ID: <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> Comments inline below. -Rick > On Jan 29, 2015, at 1:14 PM, Roelof Wobben wrote: > > e@REDACTED schreef op 29-1-2015 om 20:06: >>> So i have to make a tuple of the data and add it in a list. >>> >>> I thought I could do something like this : >>> >>> write(Key, Data, Db) -> >>> [ {key, data} | Db ] >>> >>> but then I see some error messages. >> >> what messages? >> >> as far as i can _theorize_ >> the most probably your Db is not a list, and it should be. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > Here is my code so far : > > -module(db). > > -export([new/0, destroy/1]). > > new() -> > []. > > destroy(Db) -> > {ok}. Why are you returning a tuple here instead of simply ?ok? ? > write(Key, Element, Db) -> > [ [{key, data}] | Db ] I?m assuming you want to actually store the Key and Element passed in, and not the atoms ?key? and ?data?, no? Also, looks like you forgot the full-stop / period in that function. -Rick > > > and this the output of c(db). > > > 3> c(db). > db.erl:12: syntax error before: > db.erl:8: Warning: variable 'Db' is unused > error > > Roelof > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Thu Jan 29 20:16:58 2015 From: michael.santos@REDACTED (Michael Santos) Date: Thu, 29 Jan 2015 14:16:58 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: Message-ID: <20150129191658.GA9082@brk> On Thu, Jan 29, 2015 at 09:20:25AM -0500, Serge Aleynikov wrote: > Is there a way to hook a file descriptor from within a NIF function to the > select/epoll/poll loop managed by the emulator with a callback invoked in > the "NIF-land" on activity detected on the file descriptor? > > I am looking for similar functionality available to NIF functions that is > available when writing drivers using driver_select() call (*). More > specifically, I have an eventfd file descriptor that I'd like to be > notified about without allocating a separate OS thread to poll on that FD > and return results by sending a message to a given Erlang Pid. > > Thanks, > > Serge > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select The simplest way is by passing the fd directly into a port: erlang:open_port({fd,Fd,Fd}, [binary,stream]). From e@REDACTED Thu Jan 29 20:23:40 2015 From: e@REDACTED (e@REDACTED) Date: Thu, 29 Jan 2015 20:23:40 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8686.4020701@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> Message-ID: <54CA88BC.5010707@bestmx.net> > Here is my code so far : > > -module(db). > > -export([new/0, destroy/1]). > > new() -> > []. > > destroy(Db) -> > {ok}. > > write(Key, Element, Db) -> > [ [{key, data}] | Db ] > db.erl:12: syntax error before: well, i am not the first to notice that you have simply missed the closing "." for the function 'write' which is a syntax error. and this is exactly why i always put "dots" on a new line: new() -> [] . destroy(Db) -> ok . write(Key, Element, Db) -> [ {Key, Element} | Db ] . I also fixed some other mistakes in your snippet, look for them. And do not forget to export your 'write/3' 'export_all' compiler's option is very nice For debugging: -compile(export_all). From r.wobben@REDACTED Thu Jan 29 20:25:03 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Thu, 29 Jan 2015 20:25:03 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> Message-ID: <54CA890F.7010301@home.nl> An HTML attachment was scrubbed... URL: From serge@REDACTED Thu Jan 29 20:32:06 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 14:32:06 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: <54CA6867.8020905@erlang.org> References: <54CA6867.8020905@erlang.org> Message-ID: I understand your reasoning, though NIF (being the "most recent" successor of C API) is a much cleaner API then the marshaling part of drivers. Having written a fair number of ports/drivers/NIFs I should say that from coding perspective, IMHO, NIFs are the easiest/cleanest to use when it comes to bringing existing C libraries with Erlang. Consequently, I would think that adding FD event awareness to that API (possibly prior to having full-fledged native processes) would only make it more convenient to use as it would cover more possible use cases. Serge On Thu, Jan 29, 2015 at 12:05 PM, Bj?rn-Egil Dahlberg wrote: > On 2015-01-29 15:20, Serge Aleynikov wrote: > > Is there a way to hook a file descriptor from within a NIF function to > the select/epoll/poll loop managed by the emulator with a callback invoked > in the "NIF-land" on activity detected on the file descriptor? > > I am looking for similar functionality available to NIF functions that > is available when writing drivers using driver_select() call (*). More > specifically, I have an eventfd file descriptor that I'd like to be > notified about without allocating a separate OS thread to poll on that FD > and return results by sending a message to a given Erlang Pid. > > > I think you have discovered why we have drivers in Erlang. This is exactly > the use case for drivers. Normally you don't do much in a driver. You react > to an event and transfer the binary to erlang space and to what should be > done in Erlang. Don't try to program Erlang in C (like erl_interface), do > it in Erlang. > > And no, we don't have native processes (in which case drivers would be > obsolete). And no, it's not an oversight in the NIF API, not really. You > could see native processes as an extension to NIFs but it is something > completely different. > > In some specific cases you could transfer a fd (socket) to Erlangs I/O > system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that > open fd:s in NIFs which then get transferred to gen_tcp in this manner. > > // Bj?rn-Egil > > > Thanks, > > Serge > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Thu Jan 29 20:38:09 2015 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 29 Jan 2015 13:38:09 -0600 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA890F.7010301@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> Message-ID: <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> Your db:destroy/1 function doesn?t use the parameter passed in?thus the error about ?Db? being unused. Since nothing is actually being ?destroyed? there, you have a couple options: (1) modify destroy so it takes no arguments (kind of pointless?might as well do away with the function altogether) (2) rewrite the function head like so to inform the compiler you don?t care to use Db in the function body: destroy(_Db) -> ok. -Rick > On Jan 29, 2015, at 1:25 PM, Roelof Wobben wrote: > > Rick Pettit schreef op 29-1-2015 om 20:16: >> Comments inline below. >> >> -Rick >> >>> On Jan 29, 2015, at 1:14 PM, Roelof Wobben > wrote: >>> >>> e@REDACTED schreef op 29-1-2015 om 20:06: >>>>> So i have to make a tuple of the data and add it in a list. >>>>> >>>>> I thought I could do something like this : >>>>> >>>>> write(Key, Data, Db) -> >>>>> [ {key, data} | Db ] >>>>> >>>>> but then I see some error messages. >>>> >>>> what messages? >>>> >>>> as far as i can _theorize_ >>>> the most probably your Db is not a list, and it should be. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> Here is my code so far : >>> >>> -module(db). >>> >>> -export([new/0, destroy/1]). >>> >>> new() -> >>> []. >>> >>> destroy(Db) -> >>> {ok}. >> >> Why are you returning a tuple here instead of simply ?ok? ? >> > > Because of this output : > > 10> c(db). > db.erl:8: Warning: variable 'Db' is unused > {ok,db} > 11> Db1 = db:new(). > [] > 12> Db2 = db:destroy(Db1). > ** exception error: no match of right hand side value ok > > When I do this : > > -module(db). > > -export([new/0, destroy/1, write/3]). > > new() -> > []. > > destroy(Db) -> > ok. > > write(Key, Element, Db) -> > [ {Key, Element} | Db ]. > > Roelof > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Thu Jan 29 20:40:41 2015 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 29 Jan 2015 13:40:41 -0600 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> Message-ID: <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> Oops, meant to add that as for the exception error no match of right hand side value ok, is it possible you previously bound the variable Db2 at the shell? If so, then though what you have appears to be an assignment, it is actually a match, and one which has failed because Db2 has been bound to a value other than ?ok?. You can do the following if you want the shell to ?forget? about Db2?s previous value, effectively unbinding it and allowing for a subsequent assignment to a different value: f(Db2). -Rick > On Jan 29, 2015, at 1:38 PM, Rick Pettit wrote: > > Your db:destroy/1 function doesn?t use the parameter passed in?thus the error about ?Db? being unused. > > Since nothing is actually being ?destroyed? there, you have a couple options: > > (1) modify destroy so it takes no arguments (kind of pointless?might as well do away with the function altogether) > > (2) rewrite the function head like so to inform the compiler you don?t care to use Db in the function body: > > destroy(_Db) -> > ok. > > > > -Rick > >> On Jan 29, 2015, at 1:25 PM, Roelof Wobben > wrote: >> >> Rick Pettit schreef op 29-1-2015 om 20:16: >>> Comments inline below. >>> >>> -Rick >>> >>>> On Jan 29, 2015, at 1:14 PM, Roelof Wobben > wrote: >>>> >>>> e@REDACTED schreef op 29-1-2015 om 20:06: >>>>>> So i have to make a tuple of the data and add it in a list. >>>>>> >>>>>> I thought I could do something like this : >>>>>> >>>>>> write(Key, Data, Db) -> >>>>>> [ {key, data} | Db ] >>>>>> >>>>>> but then I see some error messages. >>>>> >>>>> what messages? >>>>> >>>>> as far as i can _theorize_ >>>>> the most probably your Db is not a list, and it should be. >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> Here is my code so far : >>>> >>>> -module(db). >>>> >>>> -export([new/0, destroy/1]). >>>> >>>> new() -> >>>> []. >>>> >>>> destroy(Db) -> >>>> {ok}. >>> >>> Why are you returning a tuple here instead of simply ?ok? ? >>> >> >> Because of this output : >> >> 10> c(db). >> db.erl:8: Warning: variable 'Db' is unused >> {ok,db} >> 11> Db1 = db:new(). >> [] >> 12> Db2 = db:destroy(Db1). >> ** exception error: no match of right hand side value ok >> >> When I do this : >> >> -module(db). >> >> -export([new/0, destroy/1, write/3]). >> >> new() -> >> []. >> >> destroy(Db) -> >> ok. >> >> write(Key, Element, Db) -> >> [ {Key, Element} | Db ]. >> >> Roelof >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Thu Jan 29 20:52:52 2015 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 29 Jan 2015 14:52:52 -0500 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: <20150129191658.GA9082@brk> References: <20150129191658.GA9082@brk> Message-ID: Though this doesn't preclude the I/O subsystem from actually reading data from the file descriptor. The C library I am bridging with Erlang has it's own reactive paradigm, yet it permits to use an external select/poll event reactor, so I only need to be notified by the emulator that there is data to be read on a given Fd, and pass that Fd to a C function call to do the reading/decoding (same for writing when the Fd is congested). I realize there are many ways to skin the cat, but it looks to me that NIF API for hooking FDs with callbacks into the emulator's I/O loop would be the most beneficial... On Thu, Jan 29, 2015 at 2:16 PM, Michael Santos wrote: > On Thu, Jan 29, 2015 at 09:20:25AM -0500, Serge Aleynikov wrote: > > Is there a way to hook a file descriptor from within a NIF function to > the > > select/epoll/poll loop managed by the emulator with a callback invoked in > > the "NIF-land" on activity detected on the file descriptor? > > > > I am looking for similar functionality available to NIF functions that is > > available when writing drivers using driver_select() call (*). More > > specifically, I have an eventfd file descriptor that I'd like to be > > notified about without allocating a separate OS thread to poll on that FD > > and return results by sending a message to a given Erlang Pid. > > > > Thanks, > > > > Serge > > > > (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select > > The simplest way is by passing the fd directly into a port: > > erlang:open_port({fd,Fd,Fd}, [binary,stream]). > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Thu Jan 29 20:53:34 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Thu, 29 Jan 2015 20:53:34 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> Message-ID: <54CA8FBE.6090007@home.nl> An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Thu Jan 29 21:41:10 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Thu, 29 Jan 2015 21:41:10 +0100 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <54CA6867.8020905@erlang.org> Message-ID: Yes I know. You are well versed in port crafting. =) I don't really like the idea of having third party libs directly involved in Erlangs I/O system, especially through callbacks we have no control over. I don't really like the idea of native processes either but that's beside the point. It's little different with drivers since we have some measure of control there. You pretty much spelled out how we envision NIF usage and reactiveness handling with thread pools, your own select and messaging erlang processes. That's the way to do it. We want to isolate it and mitigate problems of things we have no control over. It's not hard to write a thread-pool-select-message-dispatch-thingy, just some tedious boiler plate, but perhaps the thread pool handling could be done simpler or at least get a helping hand from some NIF framework. Though, I don't see such a framework being implemented by the OTP team any time soon either .. we have far more pressing stuff to deal with. // Bj?rn-Egil 2015-01-29 20:32 GMT+01:00 Serge Aleynikov : > I understand your reasoning, though NIF (being the "most recent" successor > of C API) is a much cleaner API then the marshaling part of drivers. > Having written a fair number of ports/drivers/NIFs I should say that from > coding perspective, IMHO, NIFs are the easiest/cleanest to use when it > comes to bringing existing C libraries with Erlang. Consequently, I would > think that adding FD event awareness to that API (possibly prior to having > full-fledged native processes) would only make it more convenient to use as > it would cover more possible use cases. > > Serge > > On Thu, Jan 29, 2015 at 12:05 PM, Bj?rn-Egil Dahlberg > wrote: > >> On 2015-01-29 15:20, Serge Aleynikov wrote: >> >> Is there a way to hook a file descriptor from within a NIF function to >> the select/epoll/poll loop managed by the emulator with a callback invoked >> in the "NIF-land" on activity detected on the file descriptor? >> >> I am looking for similar functionality available to NIF functions that >> is available when writing drivers using driver_select() call (*). More >> specifically, I have an eventfd file descriptor that I'd like to be >> notified about without allocating a separate OS thread to poll on that FD >> and return results by sending a message to a given Erlang Pid. >> >> >> I think you have discovered why we have drivers in Erlang. This is >> exactly the use case for drivers. Normally you don't do much in a driver. >> You react to an event and transfer the binary to erlang space and to what >> should be done in Erlang. Don't try to program Erlang in C (like >> erl_interface), do it in Erlang. >> >> And no, we don't have native processes (in which case drivers would be >> obsolete). And no, it's not an oversight in the NIF API, not really. You >> could see native processes as an extension to NIFs but it is something >> completely different. >> >> In some specific cases you could transfer a fd (socket) to Erlangs I/O >> system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that >> open fd:s in NIFs which then get transferred to gen_tcp in this manner. >> >> // Bj?rn-Egil >> >> >> Thanks, >> >> Serge >> >> (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpettit@REDACTED Thu Jan 29 22:49:01 2015 From: rpettit@REDACTED (Rick Pettit) Date: Thu, 29 Jan 2015 15:49:01 -0600 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8FBE.6090007@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> Message-ID: <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> Roelof, You don?t need a variable if you don?t care about the return value of the function you are calling. To be clear, your problem with the badmatch is due to Db2 already having a value assigned which is not equal to the atom ?ok?. Try this at the shell to see what that value is: Db2. % don?t forget the period there More than likely it is set to {ok}, which is what you previously had db:destroy/1 returning. Since erlang is a single-assignment language, you cannot simply assign a different value to Db2 once it is bound. You *can* use syntax that appears just like an assignment and have that work, provided the value returned from the function matches exactly the value currently bound to the variable on the left-hand side of the ?=?. For example: Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.2 (abort with ^G) 1> Db2 = {ok}. {ok} 2> Db2. {ok} 3> Db2 = {ok}. {ok} 4> Db2 = ok. ** exception error: no match of right hand side value ok 5> Db2 = {ok}. {ok} 6> f(Db2). ok 7> Db2 = ok. ok 8> Db2 = {ok}. ** exception error: no match of right hand side value {ok} 9> Db2 = ok. ok === Here?s what?s going on there: 1. I bound the value {ok} to the variable Db2 2. I check the value of Db2 3. I ?assign? Db2 the same value ? this isn?t really an assignment now, it is a match, and in this case both the left-hand side and right-hand side have same value, so match succeeds 4. I attempt to assign Db2 a different value?since Db2 was bound to {ok} and the right-hand side now has ?ok?, that match fails?thus the match error 5. I ?assign? Db2 the original value ? again, this isn?t really an assignment, it is a match, which again succeeds for same reason as (3) above 6. I use the f() function at the shell to ?forget? about the value previously bound to Db2, effectively making Db2 unbound to any value (and as such it can now be assigned any value) 7. I assign a different value, this time ?ok?, to Db2 ? no errors that time, because it was an assignment, not a match 8. I attempt to match Db2 with the previous value?and there?s that match error again 9. I ?assign? Db2 the new value again?now it is a match, and the match is good so no error Does that make sense now? -Rick > On Jan 29, 2015, at 1:53 PM, Roelof Wobben wrote: > > I did some trail and error and saw this : > > c(db). > {ok,db} > 14> Db1 = db:new(). > [] > 15> Db2 = db:destroy(Db1). > ** exception error: no match of right hand side value ok > 16> db:destroy(Db1). > ok > > So it seems you do not have to use a variable when destroying. > > Roelof > > > > Rick Pettit schreef op 29-1-2015 om 20:40: >> Oops, meant to add that as for the exception error no match of right hand side value ok, is it possible you previously bound the variable Db2 at the shell? >> >> If so, then though what you have appears to be an assignment, it is actually a match, and one which has failed because Db2 has been bound to a value other than ?ok?. >> >> You can do the following if you want the shell to ?forget? about Db2?s previous value, effectively unbinding it and allowing for a subsequent assignment to a different value: >> >> f(Db2). >> >> -Rick >> >>> On Jan 29, 2015, at 1:38 PM, Rick Pettit > wrote: >>> >>> Your db:destroy/1 function doesn?t use the parameter passed in?thus the error about ?Db? being unused. >>> >>> Since nothing is actually being ?destroyed? there, you have a couple options: >>> >>> (1) modify destroy so it takes no arguments (kind of pointless?might as well do away with the function altogether) >>> >>> (2) rewrite the function head like so to inform the compiler you don?t care to use Db in the function body: >>> >>> destroy(_Db) -> >>> ok. >>> >>> >>> >>> -Rick >>> >>>> On Jan 29, 2015, at 1:25 PM, Roelof Wobben > wrote: >>>> >>>> Rick Pettit schreef op 29-1-2015 om 20:16: >>>>> Comments inline below. >>>>> >>>>> -Rick >>>>> >>>>>> On Jan 29, 2015, at 1:14 PM, Roelof Wobben > wrote: >>>>>> >>>>>> e@REDACTED schreef op 29-1-2015 om 20:06: >>>>>>>> So i have to make a tuple of the data and add it in a list. >>>>>>>> >>>>>>>> I thought I could do something like this : >>>>>>>> >>>>>>>> write(Key, Data, Db) -> >>>>>>>> [ {key, data} | Db ] >>>>>>>> >>>>>>>> but then I see some error messages. >>>>>>> >>>>>>> what messages? >>>>>>> >>>>>>> as far as i can _theorize_ >>>>>>> the most probably your Db is not a list, and it should be. >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> >>>>>> Here is my code so far : >>>>>> >>>>>> -module(db). >>>>>> >>>>>> -export([new/0, destroy/1]). >>>>>> >>>>>> new() -> >>>>>> []. >>>>>> >>>>>> destroy(Db) -> >>>>>> {ok}. >>>>> >>>>> Why are you returning a tuple here instead of simply ?ok? ? >>>>> >>>> >>>> Because of this output : >>>> >>>> 10> c(db). >>>> db.erl:8: Warning: variable 'Db' is unused >>>> {ok,db} >>>> 11> Db1 = db:new(). >>>> [] >>>> 12> Db2 = db:destroy(Db1). >>>> ** exception error: no match of right hand side value ok >>>> >>>> When I do this : >>>> >>>> -module(db). >>>> >>>> -export([new/0, destroy/1, write/3]). >>>> >>>> new() -> >>>> []. >>>> >>>> destroy(Db) -> >>>> ok. >>>> >>>> write(Key, Element, Db) -> >>>> [ {Key, Element} | Db ]. >>>> >>>> Roelof >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jan 30 03:47:43 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 15:47:43 +1300 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8429.9080708@home.nl> References: <54CA8429.9080708@home.nl> Message-ID: On 30/01/2015, at 8:04 am, Roelof Wobben wrote: > Db1 = db:write(francesco, london, Db). > [{francesco,london}] I presume the horrible name is part of the requirements you are working from, not something you came up with. In any programming language, procedure names that are imperative verb (phrases) should be reserved for things that have side effects, which this does not. There is a difference between - INSERTING a record for a NEW key into a data base - REPLACING a record for an EXISTING key into a data base - setting a record for a key that might or might not already exist. You should start by writing a comment. % write(Key, Data, DB0) -> DB1 % ensures that DB1 maps Key to Data and every other key that % is in DB0 to the same value it has in DB0. > So i have to make a tuple of the data and add it in a list. This is a very imperative way of talking and is likely to mislead you. You cannot add anything to a list in Erlang. Once you have a list, that is what it is, forever. You can create a *new* list that has the old one as a tail, but the old list is not changed. > > I thought I could do something like this : > > write(Key, Data, Db) -> > [ {key, data} | Db ] > > but then I see some error messages. It would help enormously if you would say WHAT error messages. I notice that you have Key, Data - - these are variables - - in the argument list but key, data - - these are constants - - in the tuple. Presumably you meant to write write(Key, Data, DB) -> [{Key,Data} | DB]. Like C, C++, C#, Java, JavaScript, XML, . . ., alphabetic case matters in Erlang. ?Key? and ?key? are not the same term. From ok@REDACTED Fri Jan 30 04:02:03 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 16:02:03 +1300 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8686.4020701@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> Message-ID: <8673B08E-908E-4D44-A108-1E6BADD8FF8F@cs.otago.ac.nz> On 30/01/2015, at 8:14 am, Roelof Wobben wrote: > e@REDACTED schreef op 29-1-2015 om 20:06: >>> So i have to make a tuple of the data and add it in a list. >>> >>> I thought I could do something like this : >>> >>> write(Key, Data, Db) -> >>> [ {key, data} | Db ] >>> >>> but then I see some error messages. >> >> what messages? >> >> as far as i can _theorize_ >> the most probably your Db is not a list, and it should be. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > Here is my code so far : > > -module(db). > > -export([new/0, destroy/1]). > > new() -> > []. > > destroy(Db) -> > {ok}. > > write(Key, Element, Db) -> > [ [{key, data}] | Db ] > > > and this the output of c(db). > > > 3> c(db). > db.erl:12: syntax error before: > db.erl:8: Warning: variable 'Db' is unused > error The first message was telling you that there was a syntax error at the very end. There is. You are missing the full stop ?.? at the end of the definition of write/3. The second message was telling you that the variable ?Db? is not used in the body of destroy/1. By the way, destroy/1 does nothing useful and certainly does NOT destroy anything. There is another error: you do not -export write/3. Now it?s time for me to tell you about an efficiency issue. Let?s start by adding another function: -export([write/3, read/2]). read(Key, [{Key,Datum]|_]) -> Datum; read(Key, [_|Db]) -> read(Key, Db). This will raise an exception if you call read/2 with a key that is not present in Db. Now let?s try a little loop. test1() -> test1(1000000000, db:write(foo, 0, db:new())). test1(0, Db) -> db:destroy(Db); test1(N, Db) -> test1(N-1, db:write(foo, db:read(foo, Db) + 1, Db)). This creates a data base with just one maplet foo:->0 and then loops 1,000,000,000 times picking out the value associated with foo, adding 1 to it, and writing that back. What is finally passed to db:destroy/1 is [{foo,1000000000},{foo,999999999},{foo,999999998},?] For real practical use, you should think about using an existing Erlang module like dict (http://www.erlang.org/doc/man/dict.html). If you want to continue to treat this as a programming exercise, then you should think about replacing existing records as well as inserting new ones. The simplest thing to do would be to use a naive simple binary search tree. % Empty tree: ?empty?. % Non-empty tree: {Key, Value, Less, Greater}. % all keys in Less < Key; all keys in Greater > Key. lookup(Key, {K,V,L,R}, Default) -> if Key < K -> lookup(Key, L, Default) ; Key > K -> lookup(Key, R, Default) ; true -> V end; lookup(Key, empty, Default) -> Default. lookup(Key, {K,V,L,R}) -> if Key < K -> lookup(Key, L) ; Key > K -> lookup(Key, R) ; true -> V end. From ok@REDACTED Fri Jan 30 04:07:34 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 16:07:34 +1300 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CA8FBE.6090007@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> Message-ID: <9989E8B9-415A-424D-8AF1-BD904DD63A60@cs.otago.ac.nz> On 30/01/2015, at 8:53 am, Roelof Wobben wrote: > I did some trail and error and saw this : > > c(db). > {ok,db} > 14> Db1 = db:new(). > [] > 15> Db2 = db:destroy(Db1). > ** exception error: no match of right hand side value ok > 16> db:destroy(Db1). > ok > > So it seems you do not have to use a variable when destroying. No. THERE IS NO DESTROYING. Erlang is like Java or JavaScript: there is *NO* way for you to say ?make the memory of this data structure available for reuse RIGHT NOW?. If you don?t want to use a data structure any more, just stop mentioning it, and eventually the run time system will clean up. Why are you even saying Db2 = db:destroy(Db1) instead of just db:destroy(Db1) ? If I saw Db2 = db:destroy(Db1) in a program where I knew that a Db was a list of pairs, I would expect db:destroy/1 to return [], an empty Db. From ok@REDACTED Fri Jan 30 05:06:07 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 17:06:07 +1300 Subject: [erlang-questions] wierd list comprehension outcomes In-Reply-To: <54C4C553.1070105@home.nl> References: <54C4C553.1070105@home.nl> Message-ID: On 25/01/2015, at 11:28 pm, Roelof Wobben wrote: > Hello, > > I follow this book : Erlang programming and I did read chapter 2. > > Now I have to do this exercise : > > C. Recursive list definitions > L = [A|[2,3]]. > [[3,2]|1]. > [H|T] = L. 1. L = [A|[2,3]] Presumably A already has a value. What is it? (Begin transcript.) 1> L = [A|[2,3]]. * 1: variable 'A' is unbound 2> A = 1. 1 3> L = [A|[2,3]]. [1,2,3] (Pause transcript.) 2. [[3,2]|1]. I see no problem here. (Resume transcript.) 4> [[3,2]|1]. [[3,2]|1] (Pause transcript.) Some people will tell you that there is a problem because 1 is not a list and the tail of a list must be a list. They are wrong. In Erlang, one kind of term is a pair [Head | Tail], where Head and Tail can be *absolutely any term whatever*. The programming language Scheme distinguishes clearly between ?pairs? which can be anything and ?lists? which are *made from* pairs as building blocks: a list is empty () or a pair whose tail is a list. Erlang has a list/1 *type* and an is_list/1 *guard test*. is_list([]) succeeds. is_list([H|T]) succeeds *whatever H and T are*. http://erlang.org/doc/reference_manual/typespec.html says of types that List :: list(Type) %% Proper list ([]-terminated) | maybe_improper_list(Type1, Type2) %% Type1=contents, Type2=termination | nonempty_improper_list(Type1, Type2) %% Type1 and Type2 as above | nonempty_list(Type) %% Proper non-empty list so is_list([[3,2]|1]) succeeds, but [[3,2]|1] does not have type list(T) for any type T. Are you confused yet? If not, why not? I?ve always thought that is_list/1 was a bad idea. We can say that ALL [_|_] TERMS ARE DOTTED PAIRS = CONS CELLS, but ONLY SOME CONS CELLS ARE PARTS OF LISTS. You wrote * [[3,2]|1]. * 2: syntax error before: '[' which I expected to see also [ 1 , 2 , 3] As the transcript above shows, there is absolutely no problem with this. The ?[? in question is presumably the first one, which tells me that there was some junk before this line. In any case, there is NO WAY that [[3,2]|1] could possibly give you [1,2,3]. 3. [H|T] = L. I see no problem here. (Resume transcript). 5> [H|T] = L. [1,2,3] 6> H. 1 7> T. [2,3] (Pause transcript.) You wrote * [H|T] = L. Which I see ** exception error: no match of right hand side value [1,2,3] You keep on not showing us everything relevant. ?no match? tells me that either H already has a value that is not 1 or T already has a value that is not [2,3]. where I was expected to see [1] [ 2, 3] You could never have seen [1]. If you write [H|T] = [[1],2,3] _then_ you will get H = [1] and T = [2,3] but not otherwise. You need to be very clear about - extending a list on the left by a single element ?consing? [New_Head | Old_Tail] - extending a list on the left by a list of many elements ?appending? New_Elements ++ Old_Tail. In particular, (Resume transcript.) 8> [0|L]. [0,1,2,3] 9> [0] ++ L. [0,1,2,3] 10> 0 ++ L. ** exception error: bad argument in operator ++/2 called as 0 ++ [1,2,3] 11> [[0]|L]. [[0],1,2,3] (End transcript.) Finally, the subject line of your message says ?weird list COMPREHENSION outcomes?, but the body of the message contains no list comprehensions whatsoever. Did you mean ?list CONSTRUCTION outcomes?? From ok@REDACTED Fri Jan 30 05:32:11 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 17:32:11 +1300 Subject: [erlang-questions] stuck at a exercise In-Reply-To: <54C6509D.5030106@home.nl> References: <54C64A8F.2070603@home.nl> <54C6509D.5030106@home.nl> Message-ID: <4B67F1BC-09E0-4A6D-BF0D-768BAB0BD5D9@cs.otago.ac.nz> On 27/01/2015, at 3:35 am, Roelof Wobben wrote: > Second problem . > > I added a second clause to it like this: > > -module(boolean). > > -export([b_not/1]). > > b_not(true) -> > false. =======^ This ends with a full stop. A full stop says ?this is the absolute and total end of this function definition. There is no more to say about this function. I have done with it. It?s finished. There will never be any more rules for this function in this file. End of story. Done!? > > b_not(false) -> =^^^^^ But whoops! You are trying to define b_not/1, but b_not/1 is already defined. > true. > > but now I see this message ; > > boolean.erl:8: function b_not/1 already defined > error > > Which I do not understand because here the same approach is used : No it is not. BETWEEN clauses you must put SEMICOLONS (;). AT THE END of a function you must put a FULL STOP (.). Now this is an arbitrary fact about Erlang syntax. It did not have to be that way. Here, for example, is a snippet of Prolog code, which is parsed correctly when the ?funnel? preprocessor is loaded first. | data bool ---> false | true. | b_not(false) -> true. | b_not(true) -> false. So it is NOT stupid to put full stops after clauses. It just happens to be wrong in Erlang. In Erlang?s defence it can be said that it follows normal English punctuation. Semicolons separate clauses of a sentence; full stops terminate sentences. Being a Prolog programmer of many years? standing, I find this quirk of Erlang?s a constant ?faux ami?, and I wish the parser would just emit a warning message and carry on. From ok@REDACTED Fri Jan 30 06:00:07 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 18:00:07 +1300 Subject: [erlang-questions] create list quesion In-Reply-To: <54C79D9F.8080607@home.nl> References: <54C79D9F.8080607@home.nl> Message-ID: On 28/01/2015, at 3:15 am, Roelof Wobben wrote: > I try now to solve a challenge where I must make a list from a range, > > So for example if I do create(2) the outcome have to be [ 1, 2] create(N) -> lists:seq(1, N). > So i can do > > create(0) -> > print array Eh? What has *[x]CREATING a [y]LIST* got to do with *[x?]PRINTING an [y?]ARRAY*? > but as far as I know the list is then not in the scope so I cannot use it when I do What list? Which scope? > > create(2) -> > add the 2 to the array. What array? You said list! > > How do I take care that the list is in the scope and still using only create(number) > > Can anyone give me a tip about it ? Let me demonstrate the thinking process. -- A number range 1 to n -- * is a special case of a number range l to u with l = 1 and u = n. -- WHY? We are going to need some sort of induction. -- Maybe it will be easy to drive the upper bound down, -- maybe it will be easier to drive the lower bound up, -- but we can't do that to a bound that is a constant. -- Generalising from 1..N to L..U will give us a CHOICE about -- which bound to do the induction on. -- A number range l to u -- * begins with l and continues with (l+1) to n if l <= u; -- * is empty if l > u. -- WHY? We can think about the INPUT to a function, but we can -- also think about the OUTPUT of a function. If the output is -- a list, it could be empty. When is it empty? It could be -- non-empty. In that case, what are the head and tail? Where -- do they come from? range1(N) -> range(1, N). range(L, U) when L =< U -> [L | range(L+1, U)]; range(L, U) when L > U -> []. Now this uses body recursion, so we might twist it a bit to get tail recursion. -- A number range l to u -- * is a special case of a number range l to u prepended to a list -- r where r = []. -- A number range l to u prepended to a list r -- * is a number range l to (u-1) prepended to (u:r) -- when l <= u; -- * is r when l > u. So range(L, U) -> range(L, U, []). range(L, U, R) when L =< U -> range(L, U-1, [U|R]); range(L, U, R) when L > U -> R. You will notice that the first version of range/2 drives the lower bound UP while the second version drives the upper bound DOWN. Generalising from range1 did pay off. You will also notice that the magic numbers 0 and 2 did not appear anywhere, and that the thought process does not mention "scope" at all. In the questions you've been asking so far, you are still dealing with the "functional programming" aspects of Erlang, where it strongly resembles many other languages in the kind of thinking required to deal with immutable structured data. You will find you need to change mental gears at least twice more: (1) penetrating to the heart of Erlang, namely concurrency. Erlang concurrency has been imitated by some other languages. (2) Getting to grips with the muscle of Erlang, the OTP libraries and the ways of structuring complete systems. This is where Erlang gets to be quite different from other languages. From ok@REDACTED Fri Jan 30 06:04:13 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 18:04:13 +1300 Subject: [erlang-questions] create list quesion In-Reply-To: References: <54C79D9F.8080607@home.nl> <54C79DEA.6020407@ninenines.eu> <54C7A653.4080802@home.nl> <54C7CC14.6070604@home.nl> <54C7D51D.8090202@home.nl> <54C7D777.7040609@home.nl> Message-ID: <37B8FA6D-42AD-48B1-800D-1DEB7CA45B13@cs.otago.ac.nz> On 28/01/2015, at 7:23 am, Imants Cekusins wrote: > this works: > > > -module(create). > > > -export([create_list/2, create/1]). > > create(0) -> > []; > > create(Number) -> > create_list(Number, [] ). > > create_list(0, List) -> List; > > create_list(Number, List) -> > create_list( Number -1, [Number | List] ). In this code, (1) The clause for create(0) is utterly pointless. (2) A call create(-1) results in an infinite loop. (3) create/1 and create_list/2 both create lists, so the distinction between the names is inappropriate. (4) Actually, neither create/1 nor create_list/2 *creates* anything, any more than 1+1 *creates* 2. Programs in OO languages create objects; programs in FP languages compute values. From ok@REDACTED Fri Jan 30 06:16:00 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 18:16:00 +1300 Subject: [erlang-questions] nand problem In-Reply-To: <54C66996.1030109@home.nl> References: <54C66996.1030109@home.nl> Message-ID: <96FEBF9C-F448-4866-9588-85F069EAD2E2@cs.otago.ac.nz> On 27/01/2015, at 5:21 am, Roelof Wobben wrote: > Hello, > > I now have to make a nand boolean with pattern matching. > I have found out that nand is true except when its true and true. > > as hint I get to implement nand using and and or. You can't. If you think of the Booleans as the lattice with carrier {false,true} and false < true, "and" is min and "or" is max, both of which are monotone non-decreasing. Any composition of monotone non-decreasing functions is monotone non-decreasing. But nand(false,false) = true > false = nand(true,true), so nand is NOT monotone non-decreasing and cannot be a composition of ands and ors. It's actually the other way around. Another name for 'nand' is the "Sheffer stroke", and you can make all the 1 or 2 argument Boolean operators from *it*. Look, the simplest possible approach is just to write down the truth table: true nand true = false true nand false = true false nand true = true false nand false = true and change it to Erlang syntax: nand(true, true) -> false; nand(true, false) -> true; nand(false, true) -> true; nand(false, false) -> true. Dollars to doughnuts doing it this way was the point of the exercise. After this, not(X) -> nand(X, X). and(X, Y) -> not(nand(X, Y)). or(X, Y) -> nand(not(X), not(Y)). Now try material implication, where (X imp Y) is true when Y is true or X is false. From ok@REDACTED Fri Jan 30 06:22:23 2015 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 30 Jan 2015 18:22:23 +1300 Subject: [erlang-questions] nand problem In-Reply-To: <54C67B39.30702@home.nl> References: <54C66996.1030109@home.nl> <20150126172632.GH1654@carfax.org.uk> <54C67B39.30702@home.nl> Message-ID: <814BEB5E-1CB1-43DA-8234-3FC729E671D3@cs.otago.ac.nz> On 27/01/2015, at 6:36 am, Roelof Wobben wrote: > I thought I have read somewhere that using _ for defensive programming was nog good practice > but that was on using other on case on the next chapter. It all depends. In languages like SML, Haskell, and F#, the type system ensures that the arguments *cannot* be anything other than the declared type. So b_nand :: Bool -> Bool -> Bool b_nand True True = False b_nand _ _ = True is as safe as it could possibly be. In languages like Prolog and Erlang, the absence of a static type system means that the arguments could be anything, so writing nand(true, true, false). nand(_, _, true). in Prolog would allow the query nand(1, 1, X) to succeed with X = true. However, suppose you have a data type with N possible values. Think of the abstract syntax trees that the Erlang compiler uses, where N is in the low dozens. Now suppose you want to write a function that takes two of these values, but only has O(N) cases that make sense. You *could* write out all N**2 combinations, but you would have to be mad. It's only when N is small (like in your example where N = 2) that avoiding _ begins to make sense. From r.wobben@REDACTED Fri Jan 30 08:15:23 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 08:15:23 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> Message-ID: <54CB2F8B.7000607@home.nl> An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jan 30 09:07:46 2015 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 30 Jan 2015 09:07:46 +0100 Subject: [erlang-questions] Exometer split into several repos Message-ID: <7167066B-BAF4-40A3-B8D3-D720D732C905@feuerlabs.com> For those of you who use Exometer: I just merged a PR splitting Exometer into several repos. It will be tagged as 2.0 soon: - exometer_core [1], including the core Exometer API and some reporters that have no external dependencies; - exometer [2], which includes all functionality previously found in Exometer, in part by making use of exometer_core; - exometer_collectd [3], containing the Collectd reporter. Those of you who need only the basic exometer functionality can use exometer_core directly. A slight change is that if you specify env variables for ?exometer? in your app.config or on the ?erl? command line, you will need to change it to ?exometer_core?. While exometer_core does check for config in exometer, OTP will not recognize the config unless the exometer application is present to be loaded. Those of you who stay with exometer will see some changes in the deps, but API, config and functionality should be compatible. The latest exometer version before the split is 1.1. [1] https://github.com/Feuerlabs/exometer_core [2] https://github.com/Feuerlabs/exometer [3] https://github.com/Feuerlabs/exometer_collectd And, yes, more reporters could be broken out from exometer, but as exometer has support for pruning dependencies via the EXOMETER_PACKAGES environment variable, this could be done on a per-need basis. Questions and feedback are welcome. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From peter.james.morgan@REDACTED Fri Jan 30 09:25:16 2015 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Fri, 30 Jan 2015 08:25:16 +0000 Subject: [erlang-questions] binary_to_float: <<"2e-1">> versus <<"2.0e-1">> Message-ID: I am parsing output from another system, where numbers arrive in a variety of forms which are out of my control. This one has caught me out recently: 1> binary_to_float(<<"2e-1">>). ** exception error: bad argument in function binary_to_float/1 called as binary_to_float(<<"2e-1">>) Whereas: 2> binary_to_float(<<"2.0e-1">>). 0.2 On: erl -version Erlang (SMP,ASYNC_THREADS,HIPE) (BEAM) emulator version 6.3 Any good reason why in binary_to_float with a scientific notation the coefficient must be a float? I'm now pattern matching this case before passing a modified coefficient as a floating point before passing it to binary_to_float. Thanks, Peter. -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Fri Jan 30 11:03:16 2015 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 30 Jan 2015 11:03:16 +0100 Subject: [erlang-questions] binary_to_float: <<"2e-1">> versus <<"2.0e-1">> In-Reply-To: References: Message-ID: <560BD48F-63B6-44D7-90A7-F23B19D98B10@gmail.com> Le 30 janv. 2015 ? 09:25, Peter Morgan a ?crit : > Any good reason why in binary_to_float with a scientific notation the coefficient must be a float? binary_to_float/1 follows the format of floats in Erlang. 2e-1 is illegal syntax in Erlang so binary_to_float/1 crashes on it. Regards. From peter.james.morgan@REDACTED Fri Jan 30 11:17:52 2015 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Fri, 30 Jan 2015 10:17:52 +0000 Subject: [erlang-questions] binary_to_float: <<"2e-1">> versus <<"2.0e-1">> In-Reply-To: <560BD48F-63B6-44D7-90A7-F23B19D98B10@gmail.com> References: <560BD48F-63B6-44D7-90A7-F23B19D98B10@gmail.com> Message-ID: Ah, OK - good to know. I'm using the following as a result: btof(<>) when Coefficient >= $0 andalso Coefficient =< $9 -> binary_to_float(<>); btof(Float) -> binary_to_float(Float). Thanks, Peter. On 30 January 2015 at 10:03, Anthony Ramine wrote: > Le 30 janv. 2015 ? 09:25, Peter Morgan a > ?crit : > > > Any good reason why in binary_to_float with a scientific notation the > coefficient must be a float? > > binary_to_float/1 follows the format of floats in Erlang. 2e-1 is illegal > syntax in Erlang so binary_to_float/1 crashes on it. > > Regards. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists@REDACTED Fri Jan 30 11:52:27 2015 From: lists@REDACTED (Camille Troillard) Date: Fri, 30 Jan 2015 11:52:27 +0100 Subject: [erlang-questions] Using processes to implement business logic Message-ID: Hi, I am looking for opinions about using processes to encapsulate the state of business entities. It looks like, in the context of our problem, we should have more advantages implementing our domain model using processes rather than simple Erlang records. It also appears that processes will act as a nice cache layer in front of the persistent storage. So, what are your experiences? Now another question... given this ?actor? based approach, I am having difficulties to figure out a proper way of dealing with processes lifetime. How would you do this in practice? Manually, or implement simple garbage collection, reference counting, ...? Best, Cam From bm@REDACTED Fri Jan 30 11:38:17 2015 From: bm@REDACTED (Bernd May) Date: Fri, 30 Jan 2015 11:38:17 +0100 Subject: [erlang-questions] cipher negotiation problem in SSL application when using PFS only In-Reply-To: References: <54C17CBC.2060801@dv-team.de> Message-ID: <54CB5F19.80600@dv-team.de> Hi, On 23.01.2015 17:22, Ingela Andin wrote: > You can use > > io:format("~p", [ssl:cipher_suites(openssl)]). > io:format("~p", [ssl:cipher_suites(erlang)]). Thanks, this allowed me to check the cipher suites again but so far I have not found any difference. the output shows that erlang is supposed to support the dhe cipher I want to use. Unfortunately the common cipherlist in the tls_handshake:hello() is empty :-/ I have added a debug trace for further debugging: http://pastebin.com/dayy06L2 Also I have noticed that this seems to be a problem of the R16B03 Erlang. if I use Erlang 17.4 as server and client it works. I had tested it before with an R16B03 server and a 17.4 client , which didn't work. I have also tested it with a 17.4 server now and a R16B03 client and that also works. seems I ahve to find out what changed in the ssl code between these versions and then patch it step by step to track down the problem. Regards, -- Bernd May -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From imantc@REDACTED Fri Jan 30 12:58:11 2015 From: imantc@REDACTED (Imants Cekusins) Date: Fri, 30 Jan 2015 12:58:11 +0100 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: Did you consider gen_fsm? gen_fsm lets you maintain the current state (both stage of the state machine and the state data). It simplifies garbage collection: it is possible to set timeouts for each specific stage Info about which particular stage times out is available. The code can then act accordingly. From zxq9@REDACTED Fri Jan 30 13:27:35 2015 From: zxq9@REDACTED (zxq9) Date: Fri, 30 Jan 2015 21:27:35 +0900 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: <1724567.iz0dqiLFj5@changa> On 2015?1?30? ??? 11:52:27 Camille Troillard wrote: > Hi, > > I am looking for opinions about using processes to encapsulate the state of > business entities. > > It looks like, in the context of our problem, we should have more advantages > implementing our domain model using processes rather than simple Erlang > records. It also appears that processes will act as a nice cache layer in > front of the persistent storage. > > So, what are your experiences? I've found processes to be extremely flexible with regard to representing business entity state. There are a few things to consider before you can gain much from process-based encapsulation of state, though. A determination must be made about what a useful granularity is for your business entities. In the case I deal with I have found it useful to start with a completelty normalized relational data schema as a starting point and build a heirarchy of structures useful to users up from there. It looks something like this: * Elementary record - As low as it gets; a schema of normalized relations. * Record - Practically useful assembly of elementary records and other records. * Document - Wraps whatever level of record the user wants to deal with in display, export and editing rules. This is the essence of a client-side application (regardless what language or paradigm the client is written in -- I've toyed with wxErlang for this, but Qt has sort of been a necessity because of ease of cross platform deployment). One form of business logic is encapsulated by the relational rules and the shape of the relational schema. A choice has to be made whether to make changes cascade at the database level or within application server code. There is no "right" answer to the question of what level to propagate data updates, but the shape of the data being declared to follow a strict normalized relational schema is important if extensibility is a concern (and with business data it always is). My choice has been to propagate notification of changes among record processes according to whatever other processes or external entities are subscribed to update notifications, but have the database schema cascade changes to foreign keys on its own (normalized relations primarily consist of primary keys and foreign keys, though). This choice forces a commitment to having client code (or record processess) calculate derived values, and using the database rules only for data integrity enforcement. Where before I had used materialized views to cache sets of data, I now use mnesia as a denormalized cache. Mnesia cannot store tables larger than 2GB, but this has not been a practical limitation within a single installation/client site (so long as BLOBs are stored as files, and only references to them are stored in the database rows). If this ever does become a limitation a caching strategy other than general memoization will become useful, but I've not hit any walls yet. Records that are "open" or "active" by a user are instantiated as processes. These records subscribe to the records they depend on so they receive/push updates among each other. In this way User A using Client A can update some data element X, and X will notify its underlying record process, which will propagate the change across the system downward to the underlying records and database, and upward to User B on Client B who has a related document open. This can take some getting used to for users who have grown accustomed to the typical "refresh the web page to see updates" form of editing or single-user business applications. (At the moment these living records exist on the application server, but it could be a delegated task if the clients were also Erlang nodes (but not a part of the server's cluster), if each table's owning process managed the subscription system instead of each record. Just haven't gotten that far yet.) This sort of data handling requires a lot of consideration about what "normalization" means, and also care when defining the record schemas. From records, though, it is easy to write OOP GUI code, or process-based wxErlang GUI code (which is easier, but harder to deploy on Windows, and impossible on mobile just now) without your head exploding, and gets you past the "Object- Relational Mismatch" problem. The tradeoff is all that thought that goes into both the relational/elementary record schema and the aggregate record schemas, which turn out to look very different. It requires a considerable amount of time to get folks who have only ever used an ORM framework on track with doing Objects-as-processes/records and elementary records as normalized relations -- I have not found a magic shortcut to this yet. You will not get the schemas right the first time, or the second. Any data that is "just obvious" at first will probably prove to be non-trivial at its root. That is: - tracking people's names in different languages - properly dealing with scripts instead of just "languages" - doing addresses + location properly - making the intuitive leap that families are more like contract organizations which cover a time span instead of a simple {Husband, Wife, [Kids]} tuple - business relationship tracking - event timelines - non-Western units of measure - anything to do with calendars - etc. Even without all this architecture and just beginning with a relatively dirty, denormalized schema in mnesia or ETS tables it is possible to see how much more interesting "live" records defined as processes that are aware of their interdependencies can be. Combining this with a subscribe/publish model is very natural in Erlang. But even with a smallish store of business data you will have to find a way to distinguish between an "active" record and one that needs to reside latent as a collection of rows in tables. If you instantiate everything you can quickly find yourself trying to spawn not a few tens of thousands, but millions of processes (I think this is why you ask your next question below). Making each table or type of record a table- or store-owning process and doing pub/sub at that level may be a golden compromise or might wind up creating bottlenecks. This is part of my thinking behind making the client-side code Erlang also, because it seems like a very smooth method of delegation. The only way to really know is through experimentation. I imagine that there is probably a golden balance somewhere in the middle, but I haven't had to locate it yet, and in any case I am still discovering ways to do things. One thing that is obvious, though, is that my method of writing data definitions could be refined a bit and interpreted to generate much of the simpler record Erlang code, the SQL definitions, and probably the ASN.1 definitions also (btw, it turns out things like JSON are not sufficient for doing business data reliably, and XML is a different sort of nightmare -- boring, old, stodgy ASN.1 is the right tool in this case). Leveraging the information in the data definitions more completely would make experimentation a lot faster. As with anything else, its a time/money tradeoff, and one I am not in a position to make in my favor yet. > Now another question... given this ?actor? based approach, I am having > difficulties to figure out a proper way of dealing with processes lifetime. > How would you do this in practice? Manually, or implement simple garbage > collection, reference counting, ...? Whenever a record's subscription count hits zero, it retires. This is a form of reference counting that is a natural outcome of the subscription "open a document" and "close a document/crash/drop connection" actions. So far this has been entirely adequate. I've written this in a bit of a rush, hopefully I explained more than I confused. There are a million more things to discover about how to make a system like this do more of the heavy lifting and deliver a better user value. -Craig From lists@REDACTED Fri Jan 30 14:11:14 2015 From: lists@REDACTED (Camille Troillard) Date: Fri, 30 Jan 2015 14:11:14 +0100 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: <1724567.iz0dqiLFj5@changa> References: <1724567.iz0dqiLFj5@changa> Message-ID: Hi Craig, Thank you for your answer, it is of incredible value. I foresee more questions, related to distribution... when I get there. Cam On 30 Jan 2015, at 13:27, zxq9 wrote: > On 2015?1?30? ??? 11:52:27 Camille Troillard wrote: >> Hi, >> >> I am looking for opinions about using processes to encapsulate the state of >> business entities. >> >> It looks like, in the context of our problem, we should have more advantages >> implementing our domain model using processes rather than simple Erlang >> records. It also appears that processes will act as a nice cache layer in >> front of the persistent storage. >> >> So, what are your experiences? > > I've found processes to be extremely flexible with regard to representing > business entity state. There are a few things to consider before you can gain > much from process-based encapsulation of state, though. > > A determination must be made about what a useful granularity is for your > business entities. In the case I deal with I have found it useful to start > with a completelty normalized relational data schema as a starting point and > build a heirarchy of structures useful to users up from there. It looks > something like this: > > * Elementary record > - As low as it gets; a schema of normalized relations. > > * Record > - Practically useful assembly of elementary records and other records. > > * Document > - Wraps whatever level of record the user wants to deal with in display, > export and editing rules. This is the essence of a client-side application > (regardless what language or paradigm the client is written in -- I've toyed > with wxErlang for this, but Qt has sort of been a necessity because of ease of > cross platform deployment). > > One form of business logic is encapsulated by the relational rules and the > shape of the relational schema. A choice has to be made whether to make > changes cascade at the database level or within application server code. There > is no "right" answer to the question of what level to propagate data updates, > but the shape of the data being declared to follow a strict normalized > relational schema is important if extensibility is a concern (and with > business data it always is). > > My choice has been to propagate notification of changes among record processes > according to whatever other processes or external entities are subscribed to > update notifications, but have the database schema cascade changes to foreign > keys on its own (normalized relations primarily consist of primary keys and > foreign keys, though). This choice forces a commitment to having client code > (or record processess) calculate derived values, and using the database rules > only for data integrity enforcement. > > Where before I had used materialized views to cache sets of data, I now use > mnesia as a denormalized cache. Mnesia cannot store tables larger than 2GB, > but this has not been a practical limitation within a single > installation/client site (so long as BLOBs are stored as files, and only > references to them are stored in the database rows). If this ever does become > a limitation a caching strategy other than general memoization will become > useful, but I've not hit any walls yet. > > Records that are "open" or "active" by a user are instantiated as processes. > These records subscribe to the records they depend on so they receive/push > updates among each other. In this way User A using Client A can update some > data element X, and X will notify its underlying record process, which will > propagate the change across the system downward to the underlying records and > database, and upward to User B on Client B who has a related document open. > This can take some getting used to for users who have grown accustomed to the > typical "refresh the web page to see updates" form of editing or single-user > business applications. (At the moment these living records exist on the > application server, but it could be a delegated task if the clients were also > Erlang nodes (but not a part of the server's cluster), if each table's owning > process managed the subscription system instead of each record. Just haven't > gotten that far yet.) > > This sort of data handling requires a lot of consideration about what > "normalization" means, and also care when defining the record schemas. From > records, though, it is easy to write OOP GUI code, or process-based wxErlang > GUI code (which is easier, but harder to deploy on Windows, and impossible on > mobile just now) without your head exploding, and gets you past the "Object- > Relational Mismatch" problem. The tradeoff is all that thought that goes into > both the relational/elementary record schema and the aggregate record schemas, > which turn out to look very different. It requires a considerable amount of > time to get folks who have only ever used an ORM framework on track with doing > Objects-as-processes/records and elementary records as normalized relations -- > I have not found a magic shortcut to this yet. > > You will not get the schemas right the first time, or the second. Any data > that is "just obvious" at first will probably prove to be non-trivial at its > root. That is: > - tracking people's names in different languages > - properly dealing with scripts instead of just "languages" > - doing addresses + location properly > - making the intuitive leap that families are more like contract organizations > which cover a time span instead of a simple {Husband, Wife, [Kids]} tuple > - business relationship tracking > - event timelines > - non-Western units of measure > - anything to do with calendars > - etc. > > Even without all this architecture and just beginning with a relatively dirty, > denormalized schema in mnesia or ETS tables it is possible to see how much > more interesting "live" records defined as processes that are aware of their > interdependencies can be. Combining this with a subscribe/publish model is > very natural in Erlang. But even with a smallish store of business data you > will have to find a way to distinguish between an "active" record and one that > needs to reside latent as a collection of rows in tables. If you instantiate > everything you can quickly find yourself trying to spawn not a few tens of > thousands, but millions of processes (I think this is why you ask your next > question below). > > Making each table or type of record a table- or store-owning process and doing > pub/sub at that level may be a golden compromise or might wind up creating > bottlenecks. This is part of my thinking behind making the client-side code > Erlang also, because it seems like a very smooth method of delegation. The > only way to really know is through experimentation. I imagine that there is > probably a golden balance somewhere in the middle, but I haven't had to locate > it yet, and in any case I am still discovering ways to do things. > > One thing that is obvious, though, is that my method of writing data > definitions could be refined a bit and interpreted to generate much of the > simpler record Erlang code, the SQL definitions, and probably the ASN.1 > definitions also (btw, it turns out things like JSON are not sufficient for > doing business data reliably, and XML is a different sort of nightmare -- > boring, old, stodgy ASN.1 is the right tool in this case). Leveraging the > information in the data definitions more completely would make experimentation > a lot faster. As with anything else, its a time/money tradeoff, and one I am > not in a position to make in my favor yet. > >> Now another question... given this ?actor? based approach, I am having >> difficulties to figure out a proper way of dealing with processes lifetime. >> How would you do this in practice? Manually, or implement simple garbage >> collection, reference counting, ...? > > Whenever a record's subscription count hits zero, it retires. This is a form > of reference counting that is a natural outcome of the subscription "open a > document" and "close a document/crash/drop connection" actions. So far this > has been entirely adequate. > > I've written this in a bit of a rush, hopefully I explained more than I > confused. There are a million more things to discover about how to make a > system like this do more of the heavy lifting and deliver a better user value. > > -Craig From lists@REDACTED Fri Jan 30 14:16:24 2015 From: lists@REDACTED (Camille Troillard) Date: Fri, 30 Jan 2015 14:16:24 +0100 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: <0BDB2581-5570-44EE-B0E8-68753FA0A005@tuli.pe> Hi Imants, gen_server also has the timeout mechanism. I am not against using TTL for my processes, I just don?t know if this is the right choice for my particular problem. I would say intuitively that it is too fine grained to keep a specific timeout for each business entity in existence, when they are modelled after processes. Cam On 30 Jan 2015, at 12:58, Imants Cekusins wrote: > Did you consider gen_fsm? > > gen_fsm lets you maintain the current state (both stage of the state > machine and the state data). > > It simplifies garbage collection: it is possible to set timeouts for > each specific stage > > Info about which particular stage times out is available. The code can > then act accordingly. From g@REDACTED Fri Jan 30 15:45:24 2015 From: g@REDACTED (Garrett Smith) Date: Fri, 30 Jan 2015 08:45:24 -0600 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: On Fri, Jan 30, 2015 at 4:52 AM, Camille Troillard wrote: > Hi, > > I am looking for opinions about using processes to encapsulate the state of business entities. By using the term "business entity" you've tipped your hand I'm afraid :) It's hard, and not productive in my experience, to map RDBMS and/or OO modeling concepts into Erlang. People do it, try it, etc. - but I just don't think it's a good fit. If you can forget about "state" and "models" - frankly forget about any "big picture" "paradigm" for "architecting" - and instead just pick one problem you have to solve, and dive in and solve it, you might not miss any other the other stuff. > It looks like, in the context of our problem, we should have more advantages implementing our domain model using processes rather than simple Erlang records. It also appears that processes will act as a nice cache layer in front of the persistent storage. This is a very common method for software construction: Step 1 - Define your domain model - so you starting naming things, classifying them, specifying how they relate to one another - this is what taxonomists do - it's what Adam did. Step 2 - Add behavior to your model - so convert the abstract definition into classes, or into schema with triggers, etc. Step 3 - Optionally add a UI layer, complete with its own domain model (e.g. MVC) This can work - but for me it feels very old timey - back when apps just sat atop a a huge RDBMS. I guess you do still see it in OO camps - I was looking at a Ruby app the other day - and sure enough there was a separate directory containing "domain model" type stuff. > So, what are your experiences? I'd drive your original question far far out of my mind and press on with a laser focused problem definition that's scoped tightly enough that you could possibly solve it in 2 - 3 days. It should be related to your business, assuming that what the original "business entities" are for. Here's what Erlang will offer... You'll get functions, which let you create APIs. A so-called business entity might manifest itself this way: > User = myapp_user:new("Jim"). What is User here? Is it a record? Is it a map? Is it a proplist? Is it a reference? Is it a... It doesn't matter. It's a user. The point is that you get a new user by calling that function. There's no state model here. What about user attributes? Functions... > "Jim" = myapp_user:get_name(User). > AnotherUser = myapp_user:set_name(User, "Jenny"). No model. Just an API. If you're concerned about implementation at this point, that's okay. How would you implement this API? I don't know - use a map as the data structure. I don't personally care - you don't have to care. Early on it probably doesn't matter. Performance, memory, etc. is the least of your concerns starting out. Just get some first-pass solution to your problem working. See it work. See it fail. Then fix it, with the benefit of an actual failure. Erlang also gives you processes. I would not think of a process as an encapsulator of a "business entity". Actually, I wouldn't use this term, ever again, in my life. (I'll use it here, but only a few more times.) A process is an independent thread of execution. So if you have any sort of independent activity in your app, you'll need processes. You in fact might not. If you're building a web app, processes and their lifecycle are handled by the web server - you'll just feed it functions. Where are the business entities here? Don't look for them. Just think about what your application should *do* - then build that using functions. Processes will call those functions in time and space. That's an app in Erlang. > Now another question... given this ?actor? based approach, I am having difficulties to figure out a proper way of dealing with processes lifetime. How would you do this in practice? Manually, or implement simple garbage collection, reference counting, ...? Sticking with "user" as our business entity (sigh) what's its lifetime? Unless you are building a simulator where users come and go quickly, we're certainly not talking about representing a user with a process. For me it's very hard to even think of a "user" as existing at all in a typical businessy app. You have "user state" - and that will be initially created by some *activity* or trigger - e.g. some HTTP request from a web app (someone clicked "Create User" on a web form). Then some other event comes along - maybe a timed cron job - and it checks some state of a "user" and performs some work. Or someone requests a web page and you have to read user state. It's all activities here (processes) and logic (functions). I wish I had a programming model word I could just use here - like "Object Oriented" or "Domain Driven Design" - and then it'd all just fall into place. I think this is "Like the way you'd program in C, but with rich process semantics". LTWYPICWRPS does not roll off the tongue. I personally use "Process Oriented Programming" but it's not used often here. I'd like to see more use of it - I think it's a useful mental model for building apps in Erlang. There's this: http://en.wikipedia.org/wiki/Process-oriented_programming The opening is a good summary of an Erlang app: | Process-oriented programming is a programming paradigm | that separates the concerns of data structures and the concurrent | processes that act upon them. I'd like to add something about functions here, but then this leaks into FP land. That's fine, but it might dilute the model. Smarter people than me will have to help here. Other parts of the article don't apply well - e.g. Erlang is certainly not limited to working with data structures that are "typically persistent, complex, and large scale". The real problem though is when you try to read up on this topic: http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=process+oriented+programming This is utterly tragic - as, at least in my extremely humbly modest opinion, this *is the way* to think of and build software. > Best, > Cam > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Fri Jan 30 15:48:15 2015 From: tony@REDACTED (Tony Rogvall) Date: Fri, 30 Jan 2015 15:48:15 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CB2F8B.7000607@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> <54CB2F8B.7000607@home.nl> Message-ID: Hi Roelof! Have you ever thought about picking up a programming book? There is a very good one that is free and you can read it online. Here is a link: http://learnyousomeerlang.com/content Have fun. /Tony > On 30 jan 2015, at 08:15, Roelof Wobben wrote: > > Rick Pettit schreef op 29-1-2015 om 22:49: >> Roelof, >> >> You don?t need a variable if you don?t care about the return value of the function you are calling. >> >> To be clear, your problem with the badmatch is due to Db2 already having a value assigned which is not equal to the atom ?ok?. >> >> Try this at the shell to see what that value is: >> >> Db2. % don?t forget the period there >> >> More than likely it is set to {ok}, which is what you previously had db:destroy/1 returning. >> >> Since erlang is a single-assignment language, you cannot simply assign a different value to Db2 once it is bound. >> >> You *can* use syntax that appears just like an assignment and have that work, provided the value returned from the function matches exactly the value currently bound to the variable on the left-hand side of the ?=?. >> >> For example: >> >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] >> >> Eshell V6.2 (abort with ^G) >> 1> Db2 = {ok}. >> {ok} >> 2> Db2. >> {ok} >> 3> Db2 = {ok}. >> {ok} >> 4> Db2 = ok. >> ** exception error: no match of right hand side value ok >> 5> Db2 = {ok}. >> {ok} >> 6> f(Db2). >> ok >> 7> Db2 = ok. >> ok >> 8> Db2 = {ok}. >> ** exception error: no match of right hand side value {ok} >> 9> Db2 = ok. >> ok >> >> === >> >> Here?s what?s going on there: >> >> 1. I bound the value {ok} to the variable Db2 >> >> 2. I check the value of Db2 >> >> 3. I ?assign? Db2 the same value ? this isn?t really an assignment now, it is a match, and in this case both the left-hand side and right-hand side have same value, so match succeeds >> >> 4. I attempt to assign Db2 a different value?since Db2 was bound to {ok} and the right-hand side now has ?ok?, that match fails?thus the match error >> >> 5. I ?assign? Db2 the original value ? again, this isn?t really an assignment, it is a match, which again succeeds for same reason as (3) above >> >> 6. I use the f() function at the shell to ?forget? about the value previously bound to Db2, effectively making Db2 unbound to any value (and as such it can now be assigned any value) >> >> 7. I assign a different value, this time ?ok?, to Db2 ? no errors that time, because it was an assignment, not a match >> >> 8. I attempt to match Db2 with the previous value?and there?s that match error again >> >> 9. I ?assign? Db2 the new value again?now it is a match, and the match is good so no error >> >> >> Does that make sense now? >> >> -Rick >> >>> On Jan 29, 2015, at 1:53 PM, Roelof Wobben > wrote: >>> >>> I did some trail and error and saw this : >>> >>> c(db). >>> {ok,db} >>> 14> Db1 = db:new(). >>> [] >>> 15> Db2 = db:destroy(Db1). >>> ** exception error: no match of right hand side value ok >>> 16> db:destroy(Db1). >>> ok >>> >>> So it seems you do not have to use a variable when destroying. >>> >>> Roelof >>> >>> >>> >>> Rick Pettit schreef op 29-1-2015 om 20:40: >>>> Oops, meant to add that as for the exception error no match of right hand side value ok, is it possible you previously bound the variable Db2 at the shell? >>>> >>>> If so, then though what you have appears to be an assignment, it is actually a match, and one which has failed because Db2 has been bound to a value other than ?ok?. >>>> >>>> You can do the following if you want the shell to ?forget? about Db2?s previous value, effectively unbinding it and allowing for a subsequent assignment to a different value: >>>> >>>> f(Db2). >>>> >>>> -Rick >>>> >>>>> On Jan 29, 2015, at 1:38 PM, Rick Pettit > wrote: >>>>> >>>>> Your db:destroy/1 function doesn?t use the parameter passed in?thus the error about ?Db? being unused. >>>>> >>>>> Since nothing is actually being ?destroyed? there, you have a couple options: >>>>> >>>>> (1) modify destroy so it takes no arguments (kind of pointless?might as well do away with the function altogether) >>>>> >>>>> (2) rewrite the function head like so to inform the compiler you don?t care to use Db in the function body: >>>>> >>>>> destroy(_Db) -> >>>>> ok. >>>>> >>>>> >>>>> >>>>> -Rick >>>>> >>>>>> On Jan 29, 2015, at 1:25 PM, Roelof Wobben > wrote: >>>>>> >>>>>> Rick Pettit schreef op 29-1-2015 om 20:16: >>>>>>> Comments inline below. >>>>>>> >>>>>>> -Rick >>>>>>> >>>>>>>> On Jan 29, 2015, at 1:14 PM, Roelof Wobben > wrote: >>>>>>>> >>>>>>>> e@REDACTED schreef op 29-1-2015 om 20:06: >>>>>>>>>> So i have to make a tuple of the data and add it in a list. >>>>>>>>>> >>>>>>>>>> I thought I could do something like this : >>>>>>>>>> >>>>>>>>>> write(Key, Data, Db) -> >>>>>>>>>> [ {key, data} | Db ] >>>>>>>>>> >>>>>>>>>> but then I see some error messages. >>>>>>>>> >>>>>>>>> what messages? >>>>>>>>> >>>>>>>>> as far as i can _theorize_ >>>>>>>>> the most probably your Db is not a list, and it should be. >>>>>>>>> _______________________________________________ >>>>>>>>> erlang-questions mailing list >>>>>>>>> erlang-questions@REDACTED >>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>> >>>>>>>> >>>>>>>> Here is my code so far : >>>>>>>> >>>>>>>> -module(db). >>>>>>>> >>>>>>>> -export([new/0, destroy/1]). >>>>>>>> >>>>>>>> new() -> >>>>>>>> []. >>>>>>>> >>>>>>>> destroy(Db) -> >>>>>>>> {ok}. >>>>>>> >>>>>>> Why are you returning a tuple here instead of simply ?ok? ? >>>>>>> >>>>>> >>>>>> Because of this output : >>>>>> >>>>>> 10> c(db). >>>>>> db.erl:8: Warning: variable 'Db' is unused >>>>>> {ok,db} >>>>>> 11> Db1 = db:new(). >>>>>> [] >>>>>> 12> Db2 = db:destroy(Db1). >>>>>> ** exception error: no match of right hand side value ok >>>>>> >>>>>> When I do this : >>>>>> >>>>>> -module(db). >>>>>> >>>>>> -export([new/0, destroy/1, write/3]). >>>>>> >>>>>> new() -> >>>>>> []. >>>>>> >>>>>> destroy(Db) -> >>>>>> ok. >>>>>> >>>>>> write(Key, Element, Db) -> >>>>>> [ {Key, Element} | Db ]. >>>>>> >>>>>> Roelof >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > Thanks all, this problem is also solved. > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From r.wobben@REDACTED Fri Jan 30 16:11:30 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 16:11:30 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> <54CB2F8B.7000607@home.nl> Message-ID: <54CB9F22.1060005@home.nl> An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Jan 30 16:23:05 2015 From: tony@REDACTED (Tony Rogvall) Date: Fri, 30 Jan 2015 16:23:05 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: <54CB9F22.1060005@home.nl> References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> <54CB2F8B.7000607@home.nl> <54CB9F22.1060005@home.nl> Message-ID: Perfect! Happy hacking. /Tony > On 30 jan 2015, at 16:11, Roelof Wobben wrote: > > I read now the erlamg programming book. > > Roelof > > > > Tony Rogvall schreef op 30-1-2015 om 15:48: >> Hi Roelof! >> >> Have you ever thought about picking up a programming book? >> >> There is a very good one that is free and you can read it online. >> >> Here is a link: http://learnyousomeerlang.com/content >> >> Have fun. >> >> /Tony >> >> >>> On 30 jan 2015, at 08:15, Roelof Wobben > wrote: >>> >>> Rick Pettit schreef op 29-1-2015 om 22:49: >>>> Roelof, >>>> >>>> You don?t need a variable if you don?t care about the return value of the function you are calling. >>>> >>>> To be clear, your problem with the badmatch is due to Db2 already having a value assigned which is not equal to the atom ?ok?. >>>> >>>> Try this at the shell to see what that value is: >>>> >>>> Db2. % don?t forget the period there >>>> >>>> More than likely it is set to {ok}, which is what you previously had db:destroy/1 returning. >>>> >>>> Since erlang is a single-assignment language, you cannot simply assign a different value to Db2 once it is bound. >>>> >>>> You *can* use syntax that appears just like an assignment and have that work, provided the value returned from the function matches exactly the value currently bound to the variable on the left-hand side of the ?=?. >>>> >>>> For example: >>>> >>>> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] >>>> >>>> Eshell V6.2 (abort with ^G) >>>> 1> Db2 = {ok}. >>>> {ok} >>>> 2> Db2. >>>> {ok} >>>> 3> Db2 = {ok}. >>>> {ok} >>>> 4> Db2 = ok. >>>> ** exception error: no match of right hand side value ok >>>> 5> Db2 = {ok}. >>>> {ok} >>>> 6> f(Db2). >>>> ok >>>> 7> Db2 = ok. >>>> ok >>>> 8> Db2 = {ok}. >>>> ** exception error: no match of right hand side value {ok} >>>> 9> Db2 = ok. >>>> ok >>>> >>>> === >>>> >>>> Here?s what?s going on there: >>>> >>>> 1. I bound the value {ok} to the variable Db2 >>>> >>>> 2. I check the value of Db2 >>>> >>>> 3. I ?assign? Db2 the same value ? this isn?t really an assignment now, it is a match, and in this case both the left-hand side and right-hand side have same value, so match succeeds >>>> >>>> 4. I attempt to assign Db2 a different value?since Db2 was bound to {ok} and the right-hand side now has ?ok?, that match fails?thus the match error >>>> >>>> 5. I ?assign? Db2 the original value ? again, this isn?t really an assignment, it is a match, which again succeeds for same reason as (3) above >>>> >>>> 6. I use the f() function at the shell to ?forget? about the value previously bound to Db2, effectively making Db2 unbound to any value (and as such it can now be assigned any value) >>>> >>>> 7. I assign a different value, this time ?ok?, to Db2 ? no errors that time, because it was an assignment, not a match >>>> >>>> 8. I attempt to match Db2 with the previous value?and there?s that match error again >>>> >>>> 9. I ?assign? Db2 the new value again?now it is a match, and the match is good so no error >>>> >>>> >>>> Does that make sense now? >>>> >>>> -Rick >>>> >>>>> On Jan 29, 2015, at 1:53 PM, Roelof Wobben > wrote: >>>>> >>>>> I did some trail and error and saw this : >>>>> >>>>> c(db). >>>>> {ok,db} >>>>> 14> Db1 = db:new(). >>>>> [] >>>>> 15> Db2 = db:destroy(Db1). >>>>> ** exception error: no match of right hand side value ok >>>>> 16> db:destroy(Db1). >>>>> ok >>>>> >>>>> So it seems you do not have to use a variable when destroying. >>>>> >>>>> Roelof >>>>> >>>>> >>>>> >>>>> Rick Pettit schreef op 29-1-2015 om 20:40: >>>>>> Oops, meant to add that as for the exception error no match of right hand side value ok, is it possible you previously bound the variable Db2 at the shell? >>>>>> >>>>>> If so, then though what you have appears to be an assignment, it is actually a match, and one which has failed because Db2 has been bound to a value other than ?ok?. >>>>>> >>>>>> You can do the following if you want the shell to ?forget? about Db2?s previous value, effectively unbinding it and allowing for a subsequent assignment to a different value: >>>>>> >>>>>> f(Db2). >>>>>> >>>>>> -Rick >>>>>> >>>>>>> On Jan 29, 2015, at 1:38 PM, Rick Pettit > wrote: >>>>>>> >>>>>>> Your db:destroy/1 function doesn?t use the parameter passed in?thus the error about ?Db? being unused. >>>>>>> >>>>>>> Since nothing is actually being ?destroyed? there, you have a couple options: >>>>>>> >>>>>>> (1) modify destroy so it takes no arguments (kind of pointless?might as well do away with the function altogether) >>>>>>> >>>>>>> (2) rewrite the function head like so to inform the compiler you don?t care to use Db in the function body: >>>>>>> >>>>>>> destroy(_Db) -> >>>>>>> ok. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -Rick >>>>>>> >>>>>>>> On Jan 29, 2015, at 1:25 PM, Roelof Wobben > wrote: >>>>>>>> >>>>>>>> Rick Pettit schreef op 29-1-2015 om 20:16: >>>>>>>>> Comments inline below. >>>>>>>>> >>>>>>>>> -Rick >>>>>>>>> >>>>>>>>>> On Jan 29, 2015, at 1:14 PM, Roelof Wobben > wrote: >>>>>>>>>> >>>>>>>>>> e@REDACTED schreef op 29-1-2015 om 20:06: >>>>>>>>>>>> So i have to make a tuple of the data and add it in a list. >>>>>>>>>>>> >>>>>>>>>>>> I thought I could do something like this : >>>>>>>>>>>> >>>>>>>>>>>> write(Key, Data, Db) -> >>>>>>>>>>>> [ {key, data} | Db ] >>>>>>>>>>>> >>>>>>>>>>>> but then I see some error messages. >>>>>>>>>>> >>>>>>>>>>> what messages? >>>>>>>>>>> >>>>>>>>>>> as far as i can _theorize_ >>>>>>>>>>> the most probably your Db is not a list, and it should be. >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> erlang-questions mailing list >>>>>>>>>>> erlang-questions@REDACTED >>>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here is my code so far : >>>>>>>>>> >>>>>>>>>> -module(db). >>>>>>>>>> >>>>>>>>>> -export([new/0, destroy/1]). >>>>>>>>>> >>>>>>>>>> new() -> >>>>>>>>>> []. >>>>>>>>>> >>>>>>>>>> destroy(Db) -> >>>>>>>>>> {ok}. >>>>>>>>> >>>>>>>>> Why are you returning a tuple here instead of simply ?ok? ? >>>>>>>>> >>>>>>>> >>>>>>>> Because of this output : >>>>>>>> >>>>>>>> 10> c(db). >>>>>>>> db.erl:8: Warning: variable 'Db' is unused >>>>>>>> {ok,db} >>>>>>>> 11> Db1 = db:new(). >>>>>>>> [] >>>>>>>> 12> Db2 = db:destroy(Db1). >>>>>>>> ** exception error: no match of right hand side value ok >>>>>>>> >>>>>>>> When I do this : >>>>>>>> >>>>>>>> -module(db). >>>>>>>> >>>>>>>> -export([new/0, destroy/1, write/3]). >>>>>>>> >>>>>>>> new() -> >>>>>>>> []. >>>>>>>> >>>>>>>> destroy(Db) -> >>>>>>>> ok. >>>>>>>> >>>>>>>> write(Key, Element, Db) -> >>>>>>>> [ {Key, Element} | Db ]. >>>>>>>> >>>>>>>> Roelof >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> Thanks all, this problem is also solved. >>> >>> Roelof >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From r.wobben@REDACTED Fri Jan 30 16:25:02 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 16:25:02 +0100 Subject: [erlang-questions] how to add a tuple to a list. In-Reply-To: References: <54CA8429.9080708@home.nl> <54CA84BD.6090908@bestmx.net> <54CA8686.4020701@home.nl> <55690AB8-242D-4061-BC51-97846B0E76DB@vailsys.com> <54CA890F.7010301@home.nl> <1C71940C-BECE-4126-8942-0B665155EA36@vailsys.com> <20C06585-CD06-4404-9EB3-D5D5529580BD@vailsys.com> <54CA8FBE.6090007@home.nl> <8EBFB5ED-2253-413B-B1BB-B13F020D2376@vailsys.com> <54CB2F8B.7000607@home.nl> <54CB9F22.1060005@home.nl> Message-ID: <54CBA24E.9010603@home.nl> An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Fri Jan 30 17:36:20 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 17:36:20 +0100 Subject: [erlang-questions] array search problem Message-ID: <54CBB304.2030103@home.nl> Hello, Im still struggeling to make the database exercise working. I have to implement a read method which outputs as this : 5> db:read(francesco, Db2). {ok,london} 6> Db3 = db:write(joern, stockholm, Db2). [{joern,stockholm},{lelle,stockholm},{francesco,london}] 7> db:read(ola, Db3). {error,instance} To achieve this do I need to use a try catch or can I achieve this with only pattern matching. Roelof From lists@REDACTED Fri Jan 30 18:03:34 2015 From: lists@REDACTED (Camille Troillard) Date: Fri, 30 Jan 2015 18:03:34 +0100 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: <9CED760F-3A9F-4C83-AFFD-F013962E76B9@tuli.pe> Hi Garret, Thank you for taking the time to answer my email. On 30 Jan 2015, at 15:45, Garrett Smith wrote: >> I am looking for opinions about using processes to encapsulate the state of business entities. > > By using the term "business entity" you've tipped your hand I'm afraid :) > > It's hard, and not productive in my experience, to map RDBMS and/or OO > modeling concepts into Erlang. People do it, try it, etc. - but I just > don't think it's a good fit. Sorry if the term entity was confusing, I was not using it this way. As I did not want to go into the discussion of Object Oriented Programming, I tried to find a less opinionated word for a ?thing?, part of the domain model. You?ll noticed that I did not talk about mapping a RDBMS either, just persistence. [...] > A process is an independent thread of execution. So if you have any > sort of independent activity in your app, you'll need processes. You > in fact might not. If you're building a web app, processes and their > lifecycle are handled by the web server - you'll just feed it > functions. Where are the business entities here? Don't look for them. I don?t see how the implementation of the business logic as to do with the external interface (web server). [...] >> Now another question... given this ?actor? based approach, I am having difficulties to figure out a proper way of dealing with processes lifetime. How would you do this in practice? Manually, or implement simple garbage collection, reference counting, ...? > > Sticking with "user" as our business entity (sigh) what's its > lifetime? Unless you are building a simulator where users come and go > quickly, we're certainly not talking about representing a user with a > process. For me it's very hard to even think of a "user" as existing > at all in a typical businessy app. You have "user state" - and that > will be initially created by some *activity* or trigger - e.g. some > HTTP request from a web app (someone clicked "Create User" on a web > form). Then some other event comes along - maybe a timed cron job - > and it checks some state of a "user" and performs some work. Or > someone requests a web page and you have to read user state. It's all > activities here (processes) and logic (functions). Again, I don?t understand why the web server would have anything to do with the of implementing a user as a process or a record. That does not rule out the possibility to modelling a user entity (or actor, or object, whatever you feel comfortable with) as a process. Cam From mononcqc@REDACTED Fri Jan 30 18:11:08 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 30 Jan 2015 12:11:08 -0500 Subject: [erlang-questions] array search problem In-Reply-To: <54CBB304.2030103@home.nl> References: <54CBB304.2030103@home.nl> Message-ID: <20150130171107.GD90397@ferdair.local> Hi Roelof. I recognize these exercises from the Francesco Cesarini and Simon Thompson book (published by O'Reilly). If you cannot work through these exercises, I strongly recommend you go back to the chapter that precedes them and re-read it again until it clicks, or possibly go back to a chapter you may have skipped. It will be a better use of your time to go through it and understand it properly rather than come to the mailing list and ask people in here to do that part on your behalf. That being said, the trick for this exercise is to go through recursion. The database you mention uses *lists* (*arrays* are a different data type and have a module to that name, too). Lists are a data structure defined recursively: [3, 2, 1] [3 | [2, 1]] [3 | [2 | [1]]] [3 | [2 | [1|[]]]] Those 4 lists are equivalent. So when you traverse a list by going [Head | Tail], on each of these, you take one element and then are left with the rest. The definition is therefore [FirstElement | RestOfListWhichIsAlsoAList]. The last element of a list is necessarily [], the empty list. Recursive functions have two main kinds of clauses (I'm going with an informal definition here): base cases, and the regular case. The base case is whenever recursion cannot proceed further. The base case is when you can proceed further. To search elements in a list, your base case will therefore be '[]', the empty list, where you can't search further. The base case will be the other [Element | Rest]. So your function to search in a DB? To avoid giving you the answer, you know that if you can't look further, you haven't found the element. Therefore, lookup(Element, []) -> {error, not_found}; lookup(Element, [??? | RestOfList]) -> ???. Can you fill in the blanks? If not, go back a few chapters. There's no shame in doing that and making sure you understand things right before moving on to more difficult topics. Regards, Fred. On 01/30, Roelof Wobben wrote: > Hello, > > Im still struggeling to make the database exercise working. > > I have to implement a read method which outputs as this : > > 5> db:read(francesco, Db2). > {ok,london} > 6> Db3 = db:write(joern, stockholm, Db2). > [{joern,stockholm},{lelle,stockholm},{francesco,london}] > 7> db:read(ola, Db3). > {error,instance} > > To achieve this do I need to use a try catch or can I achieve this with > only pattern matching. > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Fri Jan 30 18:16:02 2015 From: g@REDACTED (Garrett Smith) Date: Fri, 30 Jan 2015 11:16:02 -0600 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: <9CED760F-3A9F-4C83-AFFD-F013962E76B9@tuli.pe> References: <9CED760F-3A9F-4C83-AFFD-F013962E76B9@tuli.pe> Message-ID: On Fri, Jan 30, 2015 at 11:03 AM, Camille Troillard wrote: > Hi Garret, > > Thank you for taking the time to answer my email. > > > On 30 Jan 2015, at 15:45, Garrett Smith wrote: > >>> I am looking for opinions about using processes to encapsulate the state of business entities. >> >> By using the term "business entity" you've tipped your hand I'm afraid :) >> >> It's hard, and not productive in my experience, to map RDBMS and/or OO >> modeling concepts into Erlang. People do it, try it, etc. - but I just >> don't think it's a good fit. > > Sorry if the term entity was confusing, I was not using it this way. > As I did not want to go into the discussion of Object Oriented Programming, I tried to find a less opinionated word for a ?thing?, part of the domain model. You?ll noticed that I did not talk about mapping a RDBMS either, just persistence. > > > [...] > >> A process is an independent thread of execution. So if you have any >> sort of independent activity in your app, you'll need processes. You >> in fact might not. If you're building a web app, processes and their >> lifecycle are handled by the web server - you'll just feed it >> functions. Where are the business entities here? Don't look for them. > > I don?t see how the implementation of the business logic as to do with the external interface (web server). It doesn't go with it at all. I'm just saying here that a process should be used to implement an independent thread of execution. In many web apps, the only independent thread of execution is the HTTP transaction. So it's possible, in your case, provided that you're just serving HTTP requests, that you don't start a single process yourself. The implementation of business logic is via functions. But you were originally going down the path of modeling. This is what I'm suggesting you not worry about. The glaring exception to this of course is if you're building an actual model - like a simulator - where you want to see "users" coming to life and behaving various ways. Here you would certainly represent the users as processes. In a typical OLTP application, you would not do this - you'd handle an inbound request, perform some operations - usually against a database - and return a result. The *process* here is the act of handling that request, not in modeling some abstract "entity". > [...] > >>> Now another question... given this ?actor? based approach, I am having difficulties to figure out a proper way of dealing with processes lifetime. How would you do this in practice? Manually, or implement simple garbage collection, reference counting, ...? >> >> Sticking with "user" as our business entity (sigh) what's its >> lifetime? Unless you are building a simulator where users come and go >> quickly, we're certainly not talking about representing a user with a >> process. For me it's very hard to even think of a "user" as existing >> at all in a typical businessy app. You have "user state" - and that >> will be initially created by some *activity* or trigger - e.g. some >> HTTP request from a web app (someone clicked "Create User" on a web >> form). Then some other event comes along - maybe a timed cron job - >> and it checks some state of a "user" and performs some work. Or >> someone requests a web page and you have to read user state. It's all >> activities here (processes) and logic (functions). > > Again, I don?t understand why the web server would have anything to do with the of implementing a user as a process or a record. That does not rule out the possibility to modelling a user entity (or actor, or object, whatever you feel comfortable with) as a process. It does not rule it out. I guess I'd ask, what is motivating you to consider using a process here? From mononcqc@REDACTED Fri Jan 30 18:18:56 2015 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 30 Jan 2015 12:18:56 -0500 Subject: [erlang-questions] array search problem In-Reply-To: <20150130171107.GD90397@ferdair.local> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> Message-ID: <20150130171856.GE90397@ferdair.local> Agh, I ended up repeating 'base case' everywhere in my recursion explanation. My bad. Here's the fixed version: On 01/30, Fred Hebert wrote: > Recursive functions have two main kinds of clauses (I'm going with an > informal definition here): base cases, and the regular case. The base > case is whenever recursion cannot proceed further. The regular case is when > you can proceed further. > > To search elements in a list, your base case will therefore be '[]', the > empty list, where you can't search further. > The regular case will be the other [Element | Rest]. > > So your function to search in a DB? To avoid giving you the answer, you > know that if you can't look further, you haven't found the element. > Therefore, > > lookup(Element, []) -> > {error, not_found}; > lookup(Element, [??? | RestOfList]) -> > ???. > > Can you fill in the blanks for the regular case? If not, go back a few > chapters. There's no shame in doing that and making sure you > understand things right before moving on to more difficult topics. > Sorry about that. From r.wobben@REDACTED Fri Jan 30 18:23:59 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 18:23:59 +0100 Subject: [erlang-questions] array search problem In-Reply-To: <20150130171107.GD90397@ferdair.local> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> Message-ID: <54CBBE2F.8020209@home.nl> Thanks, I was thinking to far. I was trying to solve if someone was mistyping the database Db2 instead of Db1. Roelof Fred Hebert schreef op 30-1-2015 om 18:11: > Hi Roelof. > > I recognize these exercises from the Francesco Cesarini and Simon > Thompson book (published by O'Reilly). > > If you cannot work through these exercises, I strongly recommend you go > back to the chapter that precedes them and re-read it again until it > clicks, or possibly go back to a chapter you may have skipped. > > It will be a better use of your time to go through it and understand it > properly rather than come to the mailing list and ask people in here to > do that part on your behalf. > > That being said, the trick for this exercise is to go through recursion. > > The database you mention uses *lists* (*arrays* are a different data > type and have a module to that name, too). Lists are a data structure > defined recursively: > > [3, 2, 1] > [3 | [2, 1]] > [3 | [2 | [1]]] > [3 | [2 | [1|[]]]] > > Those 4 lists are equivalent. So when you traverse a list by going > [Head | Tail], on each of these, you take one element and then are left > with the rest. > > The definition is therefore [FirstElement | RestOfListWhichIsAlsoAList]. > The last element of a list is necessarily [], the empty list. > > Recursive functions have two main kinds of clauses (I'm going with an > informal definition here): base cases, and the regular case. The base > case is whenever recursion cannot proceed further. The base case is when > you can proceed further. > > To search elements in a list, your base case will therefore be '[]', the > empty list, where you can't search further. > The base case will be the other [Element | Rest]. > > So your function to search in a DB? To avoid giving you the answer, you > know that if you can't look further, you haven't found the element. > Therefore, > > lookup(Element, []) -> > {error, not_found}; > lookup(Element, [??? | RestOfList]) -> > ???. > > Can you fill in the blanks? If not, go back a few chapters. There's no > shame in doing that and making sure you understand things right before > moving on to more difficult topics. > > Regards, > Fred. > > On 01/30, Roelof Wobben wrote: >> Hello, >> >> Im still struggeling to make the database exercise working. >> >> I have to implement a read method which outputs as this : >> >> 5> db:read(francesco, Db2). >> {ok,london} >> 6> Db3 = db:write(joern, stockholm, Db2). >> [{joern,stockholm},{lelle,stockholm},{francesco,london}] >> 7> db:read(ola, Db3). >> {error,instance} >> >> To achieve this do I need to use a try catch or can I achieve this with >> only pattern matching. >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From e@REDACTED Fri Jan 30 19:25:58 2015 From: e@REDACTED (e@REDACTED) Date: Fri, 30 Jan 2015 19:25:58 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: <54CBCCB6.8020602@bestmx.net> Hi, all. SSL: certify: ssl_alert.erl:92:Fatal error: unknown ca I know this issue generates thousands of "hits" in google-search yet google does not reveal a consistent explanation (not a recipe!) first of all: Unknown TO WHOM??? secondly: What CA will be considered known? what properties of CA are required? may we assume that "CA" and "a certificate file" are synonyms in the current context? otherwise, what is CA and how is it represented? and last but not least: Might be this error induced by some lower-level reason, unrelated to "CA familiarity", for example unacceptable certificate format? My config is: {cacertfile, Dir ++ "ca.crt"} % self-signed {certfile, Dir ++ "server.crt"} % signed by ca.crt {keyfile, Dir ++ "server.key"} % no other options are explicitly specified where files are produced by the following procedure: openssl genrsa -des3 -out ca.key 1024 openssl req -new -key ca.key -out ca.csr cp ca.key ca.key.orig openssl rsa -in ca.key.orig -out ca.key openssl x509 -req -days 3650 -in ca.csr -signkey ca.key -out ca.crt openssl genrsa -des3 -out server.key 1024 openssl req -new -key server.key -out server.csr openssl x509 -req -days 3650 -in server.csr -signkey ca.key -out server.crt cp server.key server.key.orig openssl rsa -in server.key.orig -out server.key From r.wobben@REDACTED Fri Jan 30 19:34:42 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 19:34:42 +0100 Subject: [erlang-questions] array search problem In-Reply-To: <20150130171107.GD90397@ferdair.local> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> Message-ID: <54CBCEC2.5080707@home.nl> Hello, Im stuck again, I have now this : read(Key, []) -> {error, Instance}; read(Key, [Key, _] )-> Key; read(Key, [_, Tail] ) -> read(Key, Tail). I see this output : 10> Db1 = db:new(). [] 11> Db2 = db:write(name,roelof,Db1). [{name,roelof}] 12> Db3 = db:read(name, Db2). ** exception error: no function clause matching db:read(name,[{name,roelof}]) (db.erl, line 14) Fred Hebert schreef op 30-1-2015 om 18:11: > Hi Roelof. > > I recognize these exercises from the Francesco Cesarini and Simon > Thompson book (published by O'Reilly). > > If you cannot work through these exercises, I strongly recommend you go > back to the chapter that precedes them and re-read it again until it > clicks, or possibly go back to a chapter you may have skipped. > > It will be a better use of your time to go through it and understand it > properly rather than come to the mailing list and ask people in here to > do that part on your behalf. > > That being said, the trick for this exercise is to go through recursion. > > The database you mention uses *lists* (*arrays* are a different data > type and have a module to that name, too). Lists are a data structure > defined recursively: > > [3, 2, 1] > [3 | [2, 1]] > [3 | [2 | [1]]] > [3 | [2 | [1|[]]]] > > Those 4 lists are equivalent. So when you traverse a list by going > [Head | Tail], on each of these, you take one element and then are left > with the rest. > > The definition is therefore [FirstElement | RestOfListWhichIsAlsoAList]. > The last element of a list is necessarily [], the empty list. > > Recursive functions have two main kinds of clauses (I'm going with an > informal definition here): base cases, and the regular case. The base > case is whenever recursion cannot proceed further. The base case is when > you can proceed further. > > To search elements in a list, your base case will therefore be '[]', the > empty list, where you can't search further. > The base case will be the other [Element | Rest]. > > So your function to search in a DB? To avoid giving you the answer, you > know that if you can't look further, you haven't found the element. > Therefore, > > lookup(Element, []) -> > {error, not_found}; > lookup(Element, [??? | RestOfList]) -> > ???. > > Can you fill in the blanks? If not, go back a few chapters. There's no > shame in doing that and making sure you understand things right before > moving on to more difficult topics. > > Regards, > Fred. > > On 01/30, Roelof Wobben wrote: >> Hello, >> >> Im still struggeling to make the database exercise working. >> >> I have to implement a read method which outputs as this : >> >> 5> db:read(francesco, Db2). >> {ok,london} >> 6> Db3 = db:write(joern, stockholm, Db2). >> [{joern,stockholm},{lelle,stockholm},{francesco,london}] >> 7> db:read(ola, Db3). >> {error,instance} >> >> To achieve this do I need to use a try catch or can I achieve this with >> only pattern matching. >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Fri Jan 30 19:38:23 2015 From: hugo@REDACTED (Hugo Mills) Date: Fri, 30 Jan 2015 18:38:23 +0000 Subject: [erlang-questions] array search problem In-Reply-To: <54CBCEC2.5080707@home.nl> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> <54CBCEC2.5080707@home.nl> Message-ID: <20150130183823.GA5950@carfax.org.uk> On Fri, Jan 30, 2015 at 07:34:42PM +0100, Roelof Wobben wrote: > Hello, > > Im stuck again, > > I have now this : > > read(Key, []) -> > {error, Instance}; This clause matches a list with no members as the second parameter of the function. > read(Key, [Key, _] )-> > Key; > > read(Key, [_, Tail] ) -> > read(Key, Tail). These last two clauses match a list with exactly two members as the second parameter of the function. > I see this output : > > 10> Db1 = db:new(). > [] > 11> Db2 = db:write(name,roelof,Db1). > [{name,roelof}] > 12> Db3 = db:read(name, Db2). > ** exception error: no function clause matching > db:read(name,[{name,roelof}]) (db.erl, line 14) Here, you're passing a list with exactly one member (a 2-tuple) as the second parameter of the function. Hugo. > > > > > Fred Hebert schreef op 30-1-2015 om 18:11: > >Hi Roelof. > > > >I recognize these exercises from the Francesco Cesarini and Simon > >Thompson book (published by O'Reilly). > > > >If you cannot work through these exercises, I strongly recommend you go > >back to the chapter that precedes them and re-read it again until it > >clicks, or possibly go back to a chapter you may have skipped. > > > >It will be a better use of your time to go through it and understand it > >properly rather than come to the mailing list and ask people in here to > >do that part on your behalf. > > > >That being said, the trick for this exercise is to go through recursion. > > > >The database you mention uses *lists* (*arrays* are a different data > >type and have a module to that name, too). Lists are a data structure > >defined recursively: > > > > [3, 2, 1] > > [3 | [2, 1]] > > [3 | [2 | [1]]] > > [3 | [2 | [1|[]]]] > > > >Those 4 lists are equivalent. So when you traverse a list by going > >[Head | Tail], on each of these, you take one element and then are left > >with the rest. > > > >The definition is therefore [FirstElement | RestOfListWhichIsAlsoAList]. > >The last element of a list is necessarily [], the empty list. > > > >Recursive functions have two main kinds of clauses (I'm going with an > >informal definition here): base cases, and the regular case. The base > >case is whenever recursion cannot proceed further. The base case is when > >you can proceed further. > > > >To search elements in a list, your base case will therefore be '[]', the > >empty list, where you can't search further. > >The base case will be the other [Element | Rest]. > > > >So your function to search in a DB? To avoid giving you the answer, you > >know that if you can't look further, you haven't found the element. > >Therefore, > > > > lookup(Element, []) -> > > {error, not_found}; > > lookup(Element, [??? | RestOfList]) -> > > ???. > > > >Can you fill in the blanks? If not, go back a few chapters. There's no > >shame in doing that and making sure you understand things right before > >moving on to more difficult topics. > > > >Regards, > >Fred. > > > >On 01/30, Roelof Wobben wrote: > >>Hello, > >> > >>Im still struggeling to make the database exercise working. > >> > >>I have to implement a read method which outputs as this : > >> > >>5> db:read(francesco, Db2). > >>{ok,london} > >>6> Db3 = db:write(joern, stockholm, Db2). > >>[{joern,stockholm},{lelle,stockholm},{francesco,london}] > >>7> db:read(ola, Db3). > >>{error,instance} > >> > >>To achieve this do I need to use a try catch or can I achieve this with > >>only pattern matching. > >> > >>Roelof > >> > >>_______________________________________________ > >>erlang-questions mailing list > >>erlang-questions@REDACTED > >>http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | For months now, we have been making triumphant hugo@REDACTED carfax.org.uk | retreats before a demoralised enemy who is advancing http://carfax.org.uk/ | in utter disorder. PGP: 65E74AC0 | Wasp -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From garry@REDACTED Fri Jan 30 19:44:23 2015 From: garry@REDACTED (Garry Hodgson) Date: Fri, 30 Jan 2015 13:44:23 -0500 Subject: [erlang-questions] array search problem In-Reply-To: <54CBCEC2.5080707@home.nl> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> <54CBCEC2.5080707@home.nl> Message-ID: <54CBD107.8080207@research.att.com> [] is a list with 0 elements. [ Key, Tail ] is a list with 2 elements. [ Key | Tail ] is a list with Key as its first element, and Tail as the rest of the list (possibly [] ). On 1/30/15 1:34 PM, Roelof Wobben wrote: > Hello, > > Im stuck again, > > I have now this : > > read(Key, []) -> > {error, Instance}; > > read(Key, [Key, _] )-> > Key; > > read(Key, [_, Tail] ) -> > read(Key, Tail). > > I see this output : > > 10> Db1 = db:new(). > [] > 11> Db2 = db:write(name,roelof,Db1). > [{name,roelof}] > 12> Db3 = db:read(name, Db2). > ** exception error: no function clause matching > db:read(name,[{name,roelof}]) (db.erl, line 14) > > > > > > > Fred Hebert schreef op 30-1-2015 om 18:11: >> Hi Roelof. >> >> I recognize these exercises from the Francesco Cesarini and Simon >> Thompson book (published by O'Reilly). >> >> If you cannot work through these exercises, I strongly recommend you go >> back to the chapter that precedes them and re-read it again until it >> clicks, or possibly go back to a chapter you may have skipped. >> >> It will be a better use of your time to go through it and understand it >> properly rather than come to the mailing list and ask people in here to >> do that part on your behalf. >> >> That being said, the trick for this exercise is to go through recursion. >> >> The database you mention uses *lists* (*arrays* are a different data >> type and have a module to that name, too). Lists are a data structure >> defined recursively: >> >> [3, 2, 1] >> [3 | [2, 1]] >> [3 | [2 | [1]]] >> [3 | [2 | [1|[]]]] >> >> Those 4 lists are equivalent. So when you traverse a list by going >> [Head | Tail], on each of these, you take one element and then are left >> with the rest. >> >> The definition is therefore [FirstElement | RestOfListWhichIsAlsoAList]. >> The last element of a list is necessarily [], the empty list. >> >> Recursive functions have two main kinds of clauses (I'm going with an >> informal definition here): base cases, and the regular case. The base >> case is whenever recursion cannot proceed further. The base case is when >> you can proceed further. >> >> To search elements in a list, your base case will therefore be '[]', the >> empty list, where you can't search further. >> The base case will be the other [Element | Rest]. >> >> So your function to search in a DB? To avoid giving you the answer, you >> know that if you can't look further, you haven't found the element. >> Therefore, >> >> lookup(Element, []) -> >> {error, not_found}; >> lookup(Element, [??? | RestOfList]) -> >> ???. >> >> Can you fill in the blanks? If not, go back a few chapters. There's no >> shame in doing that and making sure you understand things right before >> moving on to more difficult topics. >> >> Regards, >> Fred. >> >> On 01/30, Roelof Wobben wrote: >>> Hello, >>> >>> Im still struggeling to make the database exercise working. >>> >>> I have to implement a read method which outputs as this : >>> >>> 5> db:read(francesco, Db2). >>> {ok,london} >>> 6> Db3 = db:write(joern, stockholm, Db2). >>> [{joern,stockholm},{lelle,stockholm},{francesco,london}] >>> 7> db:read(ola, Db3). >>> {error,instance} >>> >>> To achieve this do I need to use a try catch or can I achieve this >>> with >>> only pattern matching. >>> >>> Roelof >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From r.wobben@REDACTED Fri Jan 30 19:59:50 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Fri, 30 Jan 2015 19:59:50 +0100 Subject: [erlang-questions] array search problem In-Reply-To: <54CBD107.8080207@research.att.com> References: <54CBB304.2030103@home.nl> <20150130171107.GD90397@ferdair.local> <54CBCEC2.5080707@home.nl> <54CBD107.8080207@research.att.com> Message-ID: <54CBD4A6.3010809@home.nl> Thanks, I see now a output but the wrong one. It seems that [key| Value] is not working because the second argument is a tuple. So back to the book and reading. Roelof Garry Hodgson schreef op 30-1-2015 om 19:44: > [] is a list with 0 elements. > [ Key, Tail ] is a list with 2 elements. > [ Key | Tail ] is a list with Key as its first element, > and Tail as the rest of the list (possibly [] ). > > > On 1/30/15 1:34 PM, Roelof Wobben wrote: >> Hello, >> >> Im stuck again, >> >> I have now this : >> >> read(Key, []) -> >> {error, Instance}; >> >> read(Key, [Key, _] )-> >> Key; >> >> read(Key, [_, Tail] ) -> >> read(Key, Tail). >> >> I see this output : >> >> 10> Db1 = db:new(). >> [] >> 11> Db2 = db:write(name,roelof,Db1). >> [{name,roelof}] >> 12> Db3 = db:read(name, Db2). >> ** exception error: no function clause matching >> db:read(name,[{name,roelof}]) (db.erl, line 14) >> >> >> >> >> >> >> Fred Hebert schreef op 30-1-2015 om 18:11: >>> Hi Roelof. >>> >>> I recognize these exercises from the Francesco Cesarini and Simon >>> Thompson book (published by O'Reilly). >>> >>> If you cannot work through these exercises, I strongly recommend you go >>> back to the chapter that precedes them and re-read it again until it >>> clicks, or possibly go back to a chapter you may have skipped. >>> >>> It will be a better use of your time to go through it and understand it >>> properly rather than come to the mailing list and ask people in here to >>> do that part on your behalf. >>> >>> That being said, the trick for this exercise is to go through >>> recursion. >>> >>> The database you mention uses *lists* (*arrays* are a different data >>> type and have a module to that name, too). Lists are a data structure >>> defined recursively: >>> >>> [3, 2, 1] >>> [3 | [2, 1]] >>> [3 | [2 | [1]]] >>> [3 | [2 | [1|[]]]] >>> >>> Those 4 lists are equivalent. So when you traverse a list by going >>> [Head | Tail], on each of these, you take one element and then are left >>> with the rest. >>> >>> The definition is therefore [FirstElement | >>> RestOfListWhichIsAlsoAList]. >>> The last element of a list is necessarily [], the empty list. >>> >>> Recursive functions have two main kinds of clauses (I'm going with an >>> informal definition here): base cases, and the regular case. The base >>> case is whenever recursion cannot proceed further. The base case is >>> when >>> you can proceed further. >>> >>> To search elements in a list, your base case will therefore be '[]', >>> the >>> empty list, where you can't search further. >>> The base case will be the other [Element | Rest]. >>> >>> So your function to search in a DB? To avoid giving you the answer, you >>> know that if you can't look further, you haven't found the element. >>> Therefore, >>> >>> lookup(Element, []) -> >>> {error, not_found}; >>> lookup(Element, [??? | RestOfList]) -> >>> ???. >>> >>> Can you fill in the blanks? If not, go back a few chapters. There's no >>> shame in doing that and making sure you understand things right before >>> moving on to more difficult topics. >>> >>> Regards, >>> Fred. >>> >>> On 01/30, Roelof Wobben wrote: >>>> Hello, >>>> >>>> Im still struggeling to make the database exercise working. >>>> >>>> I have to implement a read method which outputs as this : >>>> >>>> 5> db:read(francesco, Db2). >>>> {ok,london} >>>> 6> Db3 = db:write(joern, stockholm, Db2). >>>> [{joern,stockholm},{lelle,stockholm},{francesco,london}] >>>> 7> db:read(ola, Db3). >>>> {error,instance} >>>> >>>> To achieve this do I need to use a try catch or can I achieve this >>>> with >>>> only pattern matching. >>>> >>>> Roelof >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ingela.andin@REDACTED Fri Jan 30 21:18:02 2015 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 30 Jan 2015 21:18:02 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CBCCB6.8020602@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> Message-ID: Hi! 2015-01-30 19:25 GMT+01:00 e@REDACTED : > Hi, all. > > SSL: certify: ssl_alert.erl:92:Fatal error: unknown ca > > I know this issue generates thousands of "hits" in google-search > yet google does not reveal a consistent explanation (not a recipe!) > > first of all: Unknown TO WHOM??? > To the client or server trying to verify its peer certificate. > secondly: What CA will be considered known? > > The root CA must be present in the verifiers CA database (cacertfile or corresponding option for that client/server). > what properties of CA are required? > may we assume that "CA" and "a certificate file" are synonyms in the > current context? otherwise, what is CA and how is it represented? > > Certificates and CA certificates are defined in RFC 5280. The are defined by as ASN-1 specifications and can normaly be inputed as ASN-1 DER (binary format) or as a PEM file (a text file representaion of the "DER-blob"). > and last but not least: Might be this error induced by some lower-level > reason, unrelated to "CA familiarity", for example unacceptable certificate > format? > > That would result in a diffrent error. > My config is: > {cacertfile, Dir ++ "ca.crt"} % self-signed > {certfile, Dir ++ "server.crt"} % signed by ca.crt > {keyfile, Dir ++ "server.key"} > % no other options are explicitly specified > > This is only the options of the server. The client needs to have the ca.crt in its configuration to be able to verify the servers cert. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Fri Jan 30 21:37:51 2015 From: e@REDACTED (e@REDACTED) Date: Fri, 30 Jan 2015 21:37:51 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: <54CBCCB6.8020602@bestmx.net> Message-ID: <54CBEB9F.1080405@bestmx.net> On 01/30/2015 09:18 PM, Ingela Andin wrote: > Hi! > > 2015-01-30 19:25 GMT+01:00 e@REDACTED : > >> Hi, all. >> >> SSL: certify: ssl_alert.erl:92:Fatal error: unknown ca >> >> I know this issue generates thousands of "hits" in google-search >> yet google does not reveal a consistent explanation (not a recipe!) >> >> first of all: Unknown TO WHOM??? >> > > > To the client or server trying to verify its peer certificate. since the error appears on the server side, may i deduce that the server (Erlang's ssl application) is trying to verify the client (a browser)? in this case i want to know how to disable this feature. (i only need to verify the server by the client) >> secondly: What CA will be considered known? >> >> > The root CA must be present in the verifiers CA database (cacertfile or > corresponding option for that client/server). my 'cacertfile' (as given to the 'ssl' application) contains one and only one certificate which is self-signed. >> what properties of CA are required? >> may we assume that "CA" and "a certificate file" are synonyms in the >> current context? otherwise, what is CA and how is it represented? > Certificates and CA certificates are defined in RFC 5280. The are defined > by as ASN-1 specifications and can normaly be inputed as ASN-1 DER (binary > format) or > as a PEM file (a text file representaion of the "DER-blob"). I was asking something else. When 'ssl' application complains about a "CA" does it mean a corresponding certfile that represents a CA or something else? (Does it consider any other data sources besides those files provided by me?) >> My config is: >> {cacertfile, Dir ++ "ca.crt"} % self-signed >> {certfile, Dir ++ "server.crt"} % signed by ca.crt >> {keyfile, Dir ++ "server.key"} >> % no other options are explicitly specified >> >> > This is only the options of the server. but the error appears on the server not on the client > The client needs to have the ca.crt > in its configuration to be able > to verify the servers cert. how to ensure that both certificates are available (or transferred) to a client? isn't it implicitly implemented by 'ssl' application? if not, then the bigger problems arise, such as "how client knows where to look for a missing cert?" From eric.pailleau@REDACTED Fri Jan 30 23:00:27 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Fri, 30 Jan 2015 23:00:27 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CBEB9F.1080405@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> Message-ID: <54CBFEFB.9030001@wanadoo.fr> > > my 'cacertfile' (as given to the 'ssl' application) contains one and > only one certificate which is self-signed. > If it is self-signed, it is a root CA cert. (issuer=subject) Depending X509 version, you may also have an attribute CA=true or CA=false (version = 3 if I remember). openssl x509 printing may help you. For SSL you must have some KeyUsages : serverAuth SSL/TLS Web Server Authentication. clientAuth SSL/TLS Web Client Authentication. Regards From e@REDACTED Sat Jan 31 01:38:20 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 01:38:20 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CBFEFB.9030001@wanadoo.fr> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> Message-ID: <54CC23FC.30108@bestmx.net> On 01/30/2015 11:00 PM, PAILLEAU Eric wrote: >> my 'cacertfile' (as given to the 'ssl' application) contains one and >> only one certificate which is self-signed. >> > > If it is self-signed, it is a root CA cert. (issuer=subject) > Depending X509 version, you may also have an attribute CA=true or > CA=false (version = 3 if I remember). > openssl x509 printing may help you. > For SSL you must have some KeyUsages : > > serverAuth SSL/TLS Web Server Authentication. > clientAuth SSL/TLS Web Client Authentication. sorry, i am completely lost. who is gonna read this info from my certificate and when? also it is still completely unclear where is the ORIGIN of the error: "unknown ca" -- who has thrown it? From eric.pailleau@REDACTED Sat Jan 31 01:59:39 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 31 Jan 2015 01:59:39 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC23FC.30108@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> <54CC23FC.30108@bestmx.net> Message-ID: <54CC28FB.9060705@wanadoo.fr> > > > sorry, i am completely lost. > who is gonna read this info from my certificate and when? > > also it is still completely unclear where is the ORIGIN of the error: > "unknown ca" -- who has thrown it? Read NOTES chapter of this link : http://www.openssl.org/docs/ssl/SSL_CTX_set_client_CA_list.html You need a trusted list of CA in a bundle file. you can check your openssl settings with : openssl verify -verbose -purpose sslserver -CAfile /path/to/tls/certs/cabundle.pem /path/to/tls/certs/your_server.pem Hope it can help. Regards From e@REDACTED Sat Jan 31 02:02:31 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 02:02:31 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC28FB.9060705@wanadoo.fr> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> <54CC23FC.30108@bestmx.net> <54CC28FB.9060705@wanadoo.fr> Message-ID: <54CC29A7.5060604@bestmx.net> >> sorry, i am completely lost. >> who is gonna read this info from my certificate and when? >> >> also it is still completely unclear where is the ORIGIN of the error: >> "unknown ca" -- who has thrown it? > > Read NOTES chapter of this link : > > http://www.openssl.org/docs/ssl/SSL_CTX_set_client_CA_list.html > > You need a trusted list of CA in a bundle file. trusted by WHOM? what particular application makes a decision to throw me an error? From eric.pailleau@REDACTED Sat Jan 31 02:09:00 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 31 Jan 2015 02:09:00 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC29A7.5060604@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> <54CC23FC.30108@bestmx.net> <54CC28FB.9060705@wanadoo.fr> <54CC29A7.5060604@bestmx.net> Message-ID: <54CC2B2C.9060500@wanadoo.fr> > > trusted by WHOM? > what particular application makes a decision to throw me an error? Trusted by you for sure. The error is raised by openssl. Accepting any SSL connections would be the same as not doing SSL at all. All the security is based on trust chain from a root you considere sure to the leave certificate signed by the upper CA(s). Regards From e@REDACTED Sat Jan 31 02:13:39 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 02:13:39 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC2B2C.9060500@wanadoo.fr> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> <54CC23FC.30108@bestmx.net> <54CC28FB.9060705@wanadoo.fr> <54CC29A7.5060604@bestmx.net> <54CC2B2C.9060500@wanadoo.fr> Message-ID: <54CC2C43.8020203@bestmx.net> On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: > >> >> trusted by WHOM? >> what particular application makes a decision to throw me an error? > > Trusted by you for sure. pardon me, i think puns are not very productive. > The error is raised by openssl. well, i guess there MUST BE a way to suppress this "wise" behavior. is there any docs, describing relations between erlang's "ssl" and openssl? how is it called? when? and what options are fed to openssl? maybe there is plain and simple switch "do not verify"? From zxq9@REDACTED Sat Jan 31 02:37:19 2015 From: zxq9@REDACTED (zxq9) Date: Sat, 31 Jan 2015 10:37:19 +0900 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC2C43.8020203@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <54CC2B2C.9060500@wanadoo.fr> <54CC2C43.8020203@bestmx.net> Message-ID: <1485757.DlpYLdlh5y@changa> On 2015?1?31? ??? 02:13:39 e@REDACTED wrote: > On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: > >> trusted by WHOM? > >> what particular application makes a decision to throw me an error? > > > > Trusted by you for sure. > > pardon me, i think puns are not very productive. > > > The error is raised by openssl. > > well, i guess there MUST BE a way to suppress this "wise" behavior. > is there any docs, describing relations between erlang's "ssl" and > openssl? how is it called? when? and what options are fed to openssl? > > maybe there is plain and simple switch "do not verify"? This has been the reality sand in the CA pudding since the beginning. You have a choice: verify every CA yourself (which pretty much relegates you to only using CAs you or people you actually know generate), or trust the general bundle that groups like OpenSSL, Mozilla, Google, Microsoft, etc. generally trust together. It usability issue. Every time you hit an HTTPS site or use TLS you are trusting a CA in one of those bundles that came with your OS or distro or browser. You are almost definitely doing that right now to retrieve and read this message. If, for example, you remove all the CAs from your browser's cache because you don't trust them, then your view of the web will break -- because nothing that requires 3rd party verification will work anymore. No more HTTPS, no more TLS. Poof! While it is indeed possible to comb through the CAs by hand and cull the ones you don't trust, this quickly becomes a practically futile demonstration of stubborness instead of an activity that will enhance security. Not everyone even understands what a CA is, so its not an option open to any other than the technically inclined -- which is at the heart of the (every?) security conundrum. Which brings us back to usability. In the case of specific business applications or infrastructure you or people you actually know have set up, creating your own constellation of CAs is entirely reasonable and indeed more secure than trusting a giant bundle of hundreds of CAs, the provenance of which nobody can actually prove. But this is CAs for your own little world, your private, self-secured part of the network. That is not a solution for anything outside of your control, and that is most of the internet. Its a tradeoff. In any case, you are better equipped to make decisions for yourself if you understand that certificates do not by themselves actually equate to security of the form people commonly believe. That's not a reason to stop using the internet, but it is something to bear in mind. From e@REDACTED Sat Jan 31 02:41:35 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 02:41:35 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <1485757.DlpYLdlh5y@changa> References: <54CBCCB6.8020602@bestmx.net> <54CC2B2C.9060500@wanadoo.fr> <54CC2C43.8020203@bestmx.net> <1485757.DlpYLdlh5y@changa> Message-ID: <54CC32CF.8030609@bestmx.net> On 01/31/2015 02:37 AM, zxq9 wrote: > On 2015?1?31? ??? 02:13:39 e@REDACTED wrote: >> On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: >>>> trusted by WHOM? >>>> what particular application makes a decision to throw me an error? >>> >>> Trusted by you for sure. >> >> pardon me, i think puns are not very productive. >> >>> The error is raised by openssl. >> >> well, i guess there MUST BE a way to suppress this "wise" behavior. >> is there any docs, describing relations between erlang's "ssl" and >> openssl? how is it called? when? and what options are fed to openssl? >> >> maybe there is plain and simple switch "do not verify"? > > This has been the reality sand in the CA pudding since the beginning. > > You have a choice: verify every CA yourself (which pretty much relegates you > to only using CAs you or people you actually know generate), or trust the > general bundle that groups like OpenSSL, Mozilla, Google, Microsoft, etc. > generally trust together. You missed one important option: I shall trust MYSELF and it is exactly my case. i do not want my own certificate to be validated against my own authority. you see? From zxq9@REDACTED Sat Jan 31 02:54:54 2015 From: zxq9@REDACTED (zxq9) Date: Sat, 31 Jan 2015 10:54:54 +0900 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC32CF.8030609@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <1485757.DlpYLdlh5y@changa> <54CC32CF.8030609@bestmx.net> Message-ID: <1871739.YQv9n8zsUf@changa> On 2015?1?31? ??? 02:41:35 e@REDACTED wrote: > On 01/31/2015 02:37 AM, zxq9 wrote: > > On 2015?1?31? ??? 02:13:39 e@REDACTED wrote: > >> On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: > >>>> trusted by WHOM? > >>>> what particular application makes a decision to throw me an error? > >>> > >>> Trusted by you for sure. > >> > >> pardon me, i think puns are not very productive. > >> > >>> The error is raised by openssl. > >> > >> well, i guess there MUST BE a way to suppress this "wise" behavior. > >> is there any docs, describing relations between erlang's "ssl" and > >> openssl? how is it called? when? and what options are fed to openssl? > >> > >> maybe there is plain and simple switch "do not verify"? > > > > This has been the reality sand in the CA pudding since the beginning. > > > > You have a choice: verify every CA yourself (which pretty much relegates > > you to only using CAs you or people you actually know generate), or trust > > the general bundle that groups like OpenSSL, Mozilla, Google, Microsoft, > > etc. generally trust together. > > You missed one important option: I shall trust MYSELF and it is exactly > my case. > i do not want my own certificate to be validated against my own authority. > you see? lol wut? These two sentences are so perfectly misconstructed in the context of this discussion that I'm having serious doubts as to your intent. From e@REDACTED Sat Jan 31 03:24:23 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 03:24:23 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <1871739.YQv9n8zsUf@changa> References: <54CBCCB6.8020602@bestmx.net> <1485757.DlpYLdlh5y@changa> <54CC32CF.8030609@bestmx.net> <1871739.YQv9n8zsUf@changa> Message-ID: <54CC3CD7.7080800@bestmx.net> On 01/31/2015 02:54 AM, zxq9 wrote: > On 2015?1?31? ??? 02:41:35 e@REDACTED wrote: >> On 01/31/2015 02:37 AM, zxq9 wrote: >>> On 2015?1?31? ??? 02:13:39 e@REDACTED wrote: >>>> On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: >>>>>> trusted by WHOM? >>>>>> what particular application makes a decision to throw me an error? >>>>> >>>>> Trusted by you for sure. >>>> >>>> pardon me, i think puns are not very productive. >>>> >>>>> The error is raised by openssl. >>>> >>>> well, i guess there MUST BE a way to suppress this "wise" behavior. >>>> is there any docs, describing relations between erlang's "ssl" and >>>> openssl? how is it called? when? and what options are fed to openssl? >>>> >>>> maybe there is plain and simple switch "do not verify"? >>> >>> This has been the reality sand in the CA pudding since the beginning. >>> >>> You have a choice: verify every CA yourself (which pretty much relegates >>> you to only using CAs you or people you actually know generate), or trust >>> the general bundle that groups like OpenSSL, Mozilla, Google, Microsoft, >>> etc. generally trust together. >> >> You missed one important option: I shall trust MYSELF and it is exactly >> my case. >> i do not want my own certificate to be validated against my own authority. >> you see? > > lol wut? let's revert to the beginning of the thread. I have launched a server, i have fed it with all the certificates needed. i try to connect to the server by a client and *the server* (NOT THE CLIENT!!!) throws an error "unknown ca". trying to figure out the reason behind this laconic formula, i came up with the following questions: what particular application throws it? what particular entity on *MY* server is referred to as "CA"? to whom it is "unknown", and what is considered "knowledge" in this context? From r.wobben@REDACTED Sat Jan 31 09:43:39 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 09:43:39 +0100 Subject: [erlang-questions] case not equal to Message-ID: <54CC95BB.6010901@home.nl> Hello, I have now this : read(Key, [Head | List]) -> case element(1, Head) of Key -> {ok, element(2, Head)}; _ -> read(Key, List). end The second clause must be thay element(1,Head) is not equal to Key, How do I take care of this. This gives this error message ; db.erl:20: syntax error before: '.' and !Key gives the same error message. Roelof From bengt.kleberg@REDACTED Sat Jan 31 09:53:48 2015 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Sat, 31 Jan 2015 08:53:48 +0000 Subject: [erlang-questions] case not equal to In-Reply-To: <54CC95BB.6010901@home.nl> References: <54CC95BB.6010901@home.nl> Message-ID: <1E6D4726-E225-479F-ACA0-2ED2EF1302A9@ericsson.com> Move '.' to after end. Bengt Sent from my iPhone > On 31 Jan 2015, at 09:43, Roelof Wobben wrote: > > Hello, > > I have now this : > > read(Key, [Head | List]) -> > case element(1, Head) of > Key -> {ok, element(2, Head)}; > _ -> read(Key, List). > end > > > The second clause must be thay element(1,Head) is not equal to Key, > > How do I take care of this. > > This gives this error message ; db.erl:20: syntax error before: '.' > > and !Key gives the same error message. > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From r.wobben@REDACTED Sat Jan 31 09:56:35 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 09:56:35 +0100 Subject: [erlang-questions] case not equal to In-Reply-To: References: <54CC95BB.6010901@home.nl> Message-ID: <54CC98C3.5060304@home.nl> Thanks. Finnally solved this one. Now delete and match and this exercise is done. Roelof Gordon Guthrie schreef op 31-1-2015 om 9:46: > R > > In your mind replace ?;? with ?OR? and ?,? with ?AND? and ?.? with STOP > >> read(Key, [Head | List]) -> >> case element(1, Head) of >> Key -> {ok, element(2, Head)} OR >> _ -> read(Key, List) STOP >> end > > There should be nothing at the end of the second clause not a ?.? (STOP) > There should be a ?,? (AND) or a ?.? (STOP) if it is the end of the function. > > G > >> Le 31 janv. 2015 ? 08:43, Roelof Wobben a ?crit : >> >> Hello, >> >> I have now this : >> >> read(Key, [Head | List]) -> >> case element(1, Head) of >> Key -> {ok, element(2, Head)}; >> _ -> read(Key, List). >> end >> >> >> The second clause must be thay element(1,Head) is not equal to Key, >> >> How do I take care of this. >> >> This gives this error message ; db.erl:20: syntax error before: '.' >> >> and !Key gives the same error message. >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From dch@REDACTED Sat Jan 31 10:02:45 2015 From: dch@REDACTED (Dave Cottlehuber) Date: Sat, 31 Jan 2015 10:02:45 +0100 Subject: [erlang-questions] FreeBSD dtrace-enabled builds Message-ID: Hi, On FreeBSD 10.1R amd64, after building a modified lang/erlang port, the following alarming warning is displayed. This is the same in the newer lang/erlang-runtime17 flavour as well BTW. WARNING: number of probes fixed does not match the number of defined probes (1001 != 1088, respectively) WARNING: some probes might not fire or your program might crash Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:64] [hipe] [kernel-poll:true] [dtrace] Eshell V6.3 ?(abort with ^G) 1> No doubt this is in reference the forthcoming zombie apocalypse? How serious is this ? does anybody have practical experience to relate? Any idea on what the problem might be or what I might need to do to fix it? Assuming this is resolved, what is the practical impact of using a dtrace-enabled build in production? Is there likely to be a significant performance hit, or is this even foolish from a stability perspective? Handwavey answers are entirely appropriate here. A+, Dave ? sent from my Couch From hugo@REDACTED Sat Jan 31 11:28:34 2015 From: hugo@REDACTED (Hugo Mills) Date: Sat, 31 Jan 2015 10:28:34 +0000 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC3CD7.7080800@bestmx.net> References: <54CBCCB6.8020602@bestmx.net> <1485757.DlpYLdlh5y@changa> <54CC32CF.8030609@bestmx.net> <1871739.YQv9n8zsUf@changa> <54CC3CD7.7080800@bestmx.net> Message-ID: <20150131102834.GB5950@carfax.org.uk> On Sat, Jan 31, 2015 at 03:24:23AM +0100, e@REDACTED wrote: > I have launched a server, i have fed it with all the certificates needed. > i try to connect to the server by a client and *the server* (NOT THE > CLIENT!!!) throws an error "unknown ca". > > trying to figure out the reason behind this laconic formula, i came > up with the following questions: > > what particular application throws it? I can't answer that question, but hopefully the answers below will help you work it out. > what particular entity on *MY* server is referred to as "CA"? The CA is the Certificate Authority. It's the thing that signs certificates. The CA issues one or more certificates of its own, which can be used to verify that a public certificate that it has signed is actually valid, according to that CA's internal rules and regulations. If you are signing your own certificates, you are acting as your own CA, and will have access to both the CA's private key and public certificate. Most commercial CAs these days have a hierarchy of certificates -- a top level (or "root") one for which the key is kept highly secure (i.e. in a strong-room, never connected to the internet), and which is only used to sign their next level down, which are the key/cert pairs used to actually sign the certificates that they issue for their customers to use. This sequence of certificates (root -> signing cert -> user cert) is called a certificate chain, and in order to fully verify the validity of a user certificate, you need to have all of the other certificates available. > to whom it is "unknown", and what is considered "knowledge" in this context? When a TLS connection is made, both sides of the connection get the opportunity to pass a certificate to each other. Those certificates are then validated using the certificate chain I spoke about above. Now, the entity which passes over its certificate to the other side to be checked can (must) also pass the certificate chain that signed that certificate. This is fine, except that then they could be passing a complete certificate chain that they've just invented, and actually they're intending on doing Bad Things. So, the receiving side of this conversation typically keeps a list of the top level CA certificates that are trusted to sign certificates in a reliable way. This list may be the common list of CAs that are checked by the browser manufacturers, or for your own application, it may simply contain your own private CA cert that you used to sign your own certificates. It's that list of "trusted" CAs that constitutes "knowledge". Hugo. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | Turning, pages turning in the widening bath, hugo@REDACTED carfax.org.uk | The spine cannot bear the humidity. http://carfax.org.uk/ | Books fall apart; the binding cannot hold. PGP: 65E74AC0 | Page 129 is loosed upon the world. Zarf -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From jon@REDACTED Sat Jan 31 11:35:30 2015 From: jon@REDACTED (Jon Schneider) Date: Sat, 31 Jan 2015 10:35:30 +0000 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CC2B2C.9060500@wanadoo.fr> References: <54CBCCB6.8020602@bestmx.net> <54CBEB9F.1080405@bestmx.net> <54CBFEFB.9030001@wanadoo.fr> <54CC23FC.30108@bestmx.net> <54CC28FB.9060705@wanadoo.fr> <54CC29A7.5060604@bestmx.net> <54CC2B2C.9060500@wanadoo.fr> Message-ID: > Accepting any SSL connections would be the same as not doing SSL at all. I disagree with this. Without significant resources and the ability to man-in-the-middle reading SSL traffic is still very difficult. In some ways self-signed certificates you have to accept once especially if you check the fingerprint are waaaay better than relying on the integrity of N CAs. Jon From zxq9@REDACTED Sat Jan 31 12:04:26 2015 From: zxq9@REDACTED (zxq9) Date: Sat, 31 Jan 2015 20:04:26 +0900 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: <54CBCCB6.8020602@bestmx.net> <54CC2B2C.9060500@wanadoo.fr> Message-ID: <4490285.JFCDUsRdah@changa> On 2015?1?31? ??? 10:35:30 Jon Schneider wrote: > > Accepting any SSL connections would be the same as not doing SSL at all. > > I disagree with this. Without significant resources and the ability to > man-in-the-middle reading SSL traffic is still very difficult. In some ways > self-signed certificates you have to accept once especially if you check > the fingerprint are waaaay better than relying on the integrity of N CAs. In an actively supported business data system this is The Right Way to deal with verification. That is not the case most people are familiar with, though, be they users or developers. From eric.pailleau@REDACTED Sat Jan 31 12:40:42 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 31 Jan 2015 12:40:42 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: I meant that you cannot authenticate a self signed cert unless you stored it your self as secure. unless this, anybody can forge a cert claiming to be somebody else. much simpler than decrypt the ssl session and do MIM.... ? Envoy? depuis mon mobile ? Eric zxq9 a ?crit?: >On 2015?1?31? ??? 10:35:30 Jon Schneider wrote: >> > Accepting any SSL connections would be the same as not doing SSL at all. >> >> I disagree with this. Without significant resources and the ability to >> man-in-the-middle reading SSL traffic is still very difficult. In some ways >> self-signed certificates you have to accept once especially if you check >> the fingerprint are waaaay better than relying on the integrity of N CAs. > >In an actively supported business data system this is The Right Way to deal >with verification. > >That is not the case most people are familiar with, though, be they users or >developers. >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From eric.pailleau@REDACTED Sat Jan 31 13:07:45 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 31 Jan 2015 13:07:45 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: Hi, not a pun, I felt the answer obvious. the important is not the trust of others, but what you considere trusted. I suppose you can add your self signed cert in the CA bundle as a trusted CA root. This will probably solve the problem. but this may imply the attribute CA=true in it, depending openssl version, if my memory is good... ? Envoy? depuis mon mobile ? Eric e@REDACTED a ?crit?: >On 01/31/2015 02:09 AM, PAILLEAU Eric wrote: >> >>> >>> trusted by WHOM? >>> what particular application makes a decision to throw me an error? >> >> Trusted by you for sure. > >pardon me, i think puns are not very productive. > > >> The error is raised by openssl. > >well, i guess there MUST BE a way to suppress this "wise" behavior. >is there any docs, describing relations between erlang's "ssl" and >openssl? how is it called? when? and what options are fed to openssl? > >maybe there is plain and simple switch "do not verify"? > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From eric.meadows.jonsson@REDACTED Sat Jan 31 14:52:52 2015 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Sat, 31 Jan 2015 14:52:52 +0100 Subject: [erlang-questions] Retrieve public key from DER encoded certificate Message-ID: I have a DER encoded certificate that i decode to #'Certificate' with public_key:pkix_decode_cert(Cert, plain). From the certificate I retrieve #'SubjectPublicKeyInfo' which I pass to publick_key:pem_entry_decode/1 to get rsa_public_key() or dsa_public_key(). But this function call fails because pem_entry_decode/1 does not seem to expect #'SubjectPublicKeyInfo' to be already decoded. It already has the structure that it tries to decode to here https://github.com/erlang/otp/blob/382943f9a943eb4215a297f9445e21fb9b5c0633/lib/public_key/src/public_key.erl#L119-L120 . How should I retrieve rsa_public_key() | dsa_public_key() from a DER encoded certificate? -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sat Jan 31 14:59:44 2015 From: per@REDACTED (Per Hedeland) Date: Sat, 31 Jan 2015 14:59:44 +0100 (CET) Subject: [erlang-questions] binary_to_float: <<"2e-1">> versus <<"2.0e-1">> In-Reply-To: Message-ID: <201501311359.t0VDxi9G070814@pluto.hedeland.org> Peter Morgan wrote: > >Ah, OK - good to know. I'm using the following as a result: > > >btof(<>) when Coefficient >= $0 >andalso Coefficient =< $9 -> > binary_to_float(<>); >btof(Float) -> > binary_to_float(Float). If you really don't know anything about the format of the numbers you receive, that seems like a very limited solution. E.g. I suspect that you want to handle also at least some of <<"2e1">>,<<"-2e-1">>,<<"+2e-1">>,<<"22e-1">>,<<"2">>,<<".2">>,<<"2.">> - all of which will result in badarg from binary_to_float/1 with the above code. Since I happened to have to deal with this "strictness" recently, I cobbled up the below. I believe it should handle all "reasonable" (and some not so reasonable) formats that a float can be "given" in - and (intentionally) crash for anything else. Takes an iodata() argument, since re(3) does. --Per Hedeland to_float(Val) -> RE = "^\\s*([+-])?(\\d+)?(\\.)?(\\d+)?([Ee][+-]?\\d+)?\\s*$", {match, [Sign, Int, Point, Fract, Exp]} = re:run(Val, RE, [{capture, [1, 2, 3, 4, 5], list}]), list_to_float(Sign ++ float_str(Int, Point, Fract) ++ Exp). %% make list_to_float/1 happy float_str(Int, "", "") -> Int ++ ".0"; float_str("", _, [_|_] = Fract) -> "0." ++ Fract; float_str([_|_] = Int, _, "") -> Int ++ ".0"; float_str([_|_] = Int, _, [_|_] = Fract) -> Int ++ "." ++ Fract. From r.wobben@REDACTED Sat Jan 31 15:11:57 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 15:11:57 +0100 Subject: [erlang-questions] wierd outome on my match method Message-ID: <54CCE2AD.8090804@home.nl> Hello, I have made this match method : match_key( _Key, [] , List) -> List ; match_key( Key, [Head | Tail], List ) -> case element(1, Head) of Key -> match_key(Key, Tail, [element(2,Head) | List] ); _ -> match_key(Key, Tail, List ) end. match( Key, [] ) -> {error, "Instance"}; match(Key, List) -> delete_key (Key, List, []). But when I have this database. Db5 : [{name2,tamara},{name,chantal},{name,roelof}] and I do this : Db11 = db:match(name,Db5). I see this outcome : [{name2,tamara}] and when I do this : Db10 = db:match(name2,Db5). I see this outcome: [{name,roelof},{name,chantal}] So it schould be the other way around. Can someone help me figure out what went wrong ? Roelof From essen@REDACTED Sat Jan 31 15:23:19 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 31 Jan 2015 15:23:19 +0100 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: <54CCE2AD.8090804@home.nl> References: <54CCE2AD.8090804@home.nl> Message-ID: <54CCE557.5090804@ninenines.eu> On 01/31/2015 03:11 PM, Roelof Wobben wrote: > match( Key, [] ) -> > {error, "Instance"}; > > match(Key, List) -> > delete_key (Key, List, []). This above is the function you call. It does not perform a match. > But when I have this database. > > Db5 : [{name2,tamara},{name,chantal},{name,roelof}] > > and I do this : > > Db11 = db:match(name,Db5). > > I see this outcome : > > [{name2,tamara}] > > and when I do this : > > Db10 = db:match(name2,Db5). > > I see this outcome: > > [{name,roelof},{name,chantal}] > > So it schould be the other way around. > > Can someone help me figure out what went wrong ? > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin http://ninenines.eu From hugo@REDACTED Sat Jan 31 15:29:57 2015 From: hugo@REDACTED (Hugo Mills) Date: Sat, 31 Jan 2015 14:29:57 +0000 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: <54CCE2AD.8090804@home.nl> References: <54CCE2AD.8090804@home.nl> Message-ID: <20150131142956.GC5950@carfax.org.uk> On Sat, Jan 31, 2015 at 03:11:57PM +0100, Roelof Wobben wrote: > Hello, > > I have made this match method : > > match_key( _Key, [] , List) -> > List ; > > match_key( Key, [Head | Tail], List ) -> > case element(1, Head) of > Key -> match_key(Key, Tail, [element(2,Head) | List] ); > _ -> match_key(Key, Tail, List ) > end. > > match( Key, [] ) -> > {error, "Instance"}; > > match(Key, List) -> > delete_key (Key, List, []). Is this function really what you mean to do? Hugo. > But when I have this database. > > Db5 : [{name2,tamara},{name,chantal},{name,roelof}] > > and I do this : > > Db11 = db:match(name,Db5). > > I see this outcome : > > [{name2,tamara}] > > and when I do this : > > Db10 = db:match(name2,Db5). > > I see this outcome: > > [{name,roelof},{name,chantal}] > > So it schould be the other way around. > > Can someone help me figure out what went wrong ? > > Roelof > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | O tempura! O moresushi! hugo@REDACTED carfax.org.uk | http://carfax.org.uk/ | PGP: 65E74AC0 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From r.wobben@REDACTED Sat Jan 31 15:31:35 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 15:31:35 +0100 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: <54CCE557.5090804@ninenines.eu> References: <54CCE2AD.8090804@home.nl> <54CCE557.5090804@ninenines.eu> Message-ID: <54CCE747.8040804@home.nl> Thanks, Everything is working now. Finnaly I have solved the whole exercise like this : -module(db). -export([new/0, destroy/1, write/3, read/2, delete_key/3, delete/2, match/2, match_key/3]). new() -> []. destroy(_Db) -> ok. write(Key, Element, Db) -> [ {Key, Element} | Db ]. read(_Key, []) -> {error, "Instance"}; read(Key, [Head | List]) -> case element(1, Head) of Key -> {ok, element(2, Head)}; _ -> read(Key, List) end. delete_key( _Key, [] , List2) -> List2 ; delete_key( Key, [Head | Tail], List3 ) -> case element(1, Head) of Key -> delete_key(Key, Tail, List3 ); _ -> delete_key(Key, Tail, [Head | List3] ) end. delete( _Key, [] ) -> {error, "Instance"}; delete(Key, List) -> match_key (Key, List, []). match_key( _Key, [] , List) -> List ; match_key( Key, [Head | Tail], List ) -> case element(1, Head) of Key -> match_key(Key, Tail, [element(2,Head) | List] ); _ -> match_key(Key, Tail, List ) end. match( Key, [] ) -> {error, "Instance"}; match(Key, List) -> match_key (Key, List, []). Now I have 2 questions : 1) Is this a good solution. 2) Can I somewhere hide my helper functions (match_key, delete_key) and do not get a warning about unused. Roelof Lo?c Hoguin schreef op 31-1-2015 om 15:23: > On 01/31/2015 03:11 PM, Roelof Wobben wrote: >> match( Key, [] ) -> >> {error, "Instance"}; >> >> match(Key, List) -> >> delete_key (Key, List, []). > > This above is the function you call. It does not perform a match. > >> But when I have this database. >> >> Db5 : [{name2,tamara},{name,chantal},{name,roelof}] >> >> and I do this : >> >> Db11 = db:match(name,Db5). >> >> I see this outcome : >> >> [{name2,tamara}] >> >> and when I do this : >> >> Db10 = db:match(name2,Db5). >> >> I see this outcome: >> >> [{name,roelof},{name,chantal}] >> >> So it schould be the other way around. >> >> Can someone help me figure out what went wrong ? >> >> Roelof >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From imantc@REDACTED Sat Jan 31 15:49:52 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 31 Jan 2015 15:49:52 +0100 Subject: [erlang-questions] binary_to_float: <<"2e-1">> versus <<"2.0e-1">> In-Reply-To: <201501311359.t0VDxi9G070814@pluto.hedeland.org> References: <201501311359.t0VDxi9G070814@pluto.hedeland.org> Message-ID: Thank you very much, Per! From jesper.louis.andersen@REDACTED Sat Jan 31 16:23:05 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 31 Jan 2015 16:23:05 +0100 Subject: [erlang-questions] Using processes to implement business logic In-Reply-To: References: Message-ID: On Fri, Jan 30, 2015 at 11:52 AM, Camille Troillard wrote: > It looks like, in the context of our problem, we should have more > advantages implementing our domain model using processes rather than simple > Erlang records. It also appears that processes will act as a nice cache > layer in front of the persistent storage. In the words of the Subject/Object dichotomy[0]: do you want your business entities to "act" or be "acted upon" ? If the former is true, then make them into processes. This is often the case if your entities are "alive" and needs to query databases, have timeouts, needs to manage persistence and so on. Or if the processing of an entity is complex with many different advanced code paths. If the latter is true, then make them into records. This is usually good if your data can be batch-processed and is just "dead" data, where other systems are using that data to work. This often leads to a pipeline processing chain where workers (subjects) operate on the entities. This is also the only viable model in many programming languages. [0] http://en.wikipedia.org/wiki/Subject%E2%80%93object_problem -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Jan 31 16:40:55 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 31 Jan 2015 16:40:55 +0100 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: <54CCE2AD.8090804@home.nl> References: <54CCE2AD.8090804@home.nl> Message-ID: On Sat, Jan 31, 2015 at 3:11 PM, Roelof Wobben wrote: > match_key( Key, [Head | Tail], List ) -> > case element(1, Head) of > Key -> match_key(Key, Tail, [element(2,Head) | List] ); > _ -> match_key(Key, Tail, List ) > end. > This is a very imperative solution to the matcher. If you have a tuple, T = {X, Y} you can use element(1, T) and element(2, T) to project the X and Y elements "out of" the tuple. The equation rule is: T = {element(1, T), element(2, T)}, That is, if we split the tuple and reform it, we obtain the same tuple again. But there is another semantics possible, namely matching. Most main-stream languages did not add this form since it was not commonly known at the time, but for some problems, it is a far more succinct: T = case T of {X, Y} -> {X, Y} end, which, we can use to rewrite the code: match_key(K, [H|T], L) -> case H of {K, V} -> match_key(K, T, [V | L]); _ -> match_key(Key, T, L) end. Note how this is more direct[0], since we don't have to worry about projecting out of `Head` anymore. We simply match on H and depending on the match, we do the right thing. If the key matches, then the value, V, is already deconstructed, so we can avoid having to write that. Now, we can refactor some more. The match can be carried out at the top level of the function: match_key(_K, [], L) -> L; match_key(K, [{K, V} | T], L) -> match_key(K, T, [V | L]); match_key(K, [_ | T], L) -> match_key(K, T, L). which should remove a lot of clutter and make the function operate on the structure of the 2nd argument directly. [0] It avoids what is called boolean blindness. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Sat Jan 31 16:52:04 2015 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Sat, 31 Jan 2015 09:52:04 -0600 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <54CA6867.8020905@erlang.org> Message-ID: What is the measure of control that the VM has over drivers that it would not have over native processes? I've always interpreted native processes as being simpler and more capable than drivers but with exactly the same hazards (executing for too long, corrupting/crashing, etc). I've worked with NIFs but not drivers; perhaps there's some obvious aspect of drivers that I've missed. On Thu, Jan 29, 2015 at 2:41 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > Yes I know. You are well versed in port crafting. =) > > I don't really like the idea of having third party libs directly involved > in Erlangs I/O system, especially through callbacks we have no control > over. I don't really like the idea of native processes either but that's > beside the point. > It's little different with drivers since we have some measure of control > there. > > You pretty much spelled out how we envision NIF usage and reactiveness > handling with thread pools, your own select and messaging erlang processes. > That's the way to do it. We want to isolate it and mitigate problems of > things we have no control over. > It's not hard to write a thread-pool-select-message-dispatch-thingy, just > some tedious boiler plate, but perhaps the thread pool handling could be > done simpler or at least get a helping hand from some NIF framework. > Though, I don't see such a framework being implemented by the OTP team any > time soon either .. we have far more pressing stuff to deal with. > > // Bj?rn-Egil > > > 2015-01-29 20:32 GMT+01:00 Serge Aleynikov : > >> I understand your reasoning, though NIF (being the "most recent" >> successor of C API) is a much cleaner API then the marshaling part of >> drivers. Having written a fair number of ports/drivers/NIFs I should say >> that from coding perspective, IMHO, NIFs are the easiest/cleanest to use >> when it comes to bringing existing C libraries with Erlang. Consequently, >> I would think that adding FD event awareness to that API (possibly prior to >> having full-fledged native processes) would only make it more convenient to >> use as it would cover more possible use cases. >> >> Serge >> >> On Thu, Jan 29, 2015 at 12:05 PM, Bj?rn-Egil Dahlberg >> wrote: >> >>> On 2015-01-29 15:20, Serge Aleynikov wrote: >>> >>> Is there a way to hook a file descriptor from within a NIF function to >>> the select/epoll/poll loop managed by the emulator with a callback invoked >>> in the "NIF-land" on activity detected on the file descriptor? >>> >>> I am looking for similar functionality available to NIF functions that >>> is available when writing drivers using driver_select() call (*). More >>> specifically, I have an eventfd file descriptor that I'd like to be >>> notified about without allocating a separate OS thread to poll on that FD >>> and return results by sending a message to a given Erlang Pid. >>> >>> >>> I think you have discovered why we have drivers in Erlang. This is >>> exactly the use case for drivers. Normally you don't do much in a driver. >>> You react to an event and transfer the binary to erlang space and to what >>> should be done in Erlang. Don't try to program Erlang in C (like >>> erl_interface), do it in Erlang. >>> >>> And no, we don't have native processes (in which case drivers would be >>> obsolete). And no, it's not an oversight in the NIF API, not really. You >>> could see native processes as an extension to NIFs but it is something >>> completely different. >>> >>> In some specific cases you could transfer a fd (socket) to Erlangs I/O >>> system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that >>> open fd:s in NIFs which then get transferred to gen_tcp in this manner. >>> >>> // Bj?rn-Egil >>> >>> >>> Thanks, >>> >>> Serge >>> >>> (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sat Jan 31 16:52:12 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 16:52:12 +0100 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: References: <54CCE2AD.8090804@home.nl> Message-ID: <54CCFA2C.9030506@home.nl> An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Jan 31 16:59:58 2015 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 31 Jan 2015 16:59:58 +0100 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <54CA6867.8020905@erlang.org> Message-ID: On Sat, Jan 31, 2015 at 4:52 PM, Daniel Goertzen wrote: > What is the measure of control that the VM has over drivers that it would > not have over native processes? I've always interpreted native processes > as being simpler and more capable than drivers but with exactly the same > hazards (executing for too long, corrupting/crashing, etc). I've worked > with NIFs but not drivers; perhaps there's some obvious aspect of drivers > that I've missed. Your observation is correct. Once you have a native process API, you are essentially on the path to eradicate drivers from the Erlang system by making them an extension on top of native processes. Of course, there are details to be hashed out, but this is a possible path to take. The fun thing is that this will also make the Erlang and Go runtimes a bit more like each other. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sat Jan 31 17:12:10 2015 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sat, 31 Jan 2015 17:12:10 +0100 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <54CA6867.8020905@erlang.org> Message-ID: Yes native processes would probably make drivers obsolete. We don't exactly know what native processes will become but we could certainly envision a better solution than the current drivers. Ideally we would have a mechanism that could schedule native code flawlessly in user space but I don't see that happening. Now, exactly what is the problem with making a thread and handling select yourself? Is it perceived as worse performance or perceived as just cumbersome coding? Do you need the fd to interact with other Erlang ports? // Bj?rn-Egil 2015-01-31 16:59 GMT+01:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > > On Sat, Jan 31, 2015 at 4:52 PM, Daniel Goertzen < > daniel.goertzen@REDACTED> wrote: > >> What is the measure of control that the VM has over drivers that it would >> not have over native processes? I've always interpreted native processes >> as being simpler and more capable than drivers but with exactly the same >> hazards (executing for too long, corrupting/crashing, etc). I've worked >> with NIFs but not drivers; perhaps there's some obvious aspect of drivers >> that I've missed. > > > Your observation is correct. Once you have a native process API, you are > essentially on the path to eradicate drivers from the Erlang system by > making them an extension on top of native processes. Of course, there are > details to be hashed out, but this is a possible path to take. > > The fun thing is that this will also make the Erlang and Go runtimes a bit > more like each other. > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Sat Jan 31 17:18:04 2015 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 31 Jan 2015 11:18:04 -0500 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: <54CCE747.8040804@home.nl> References: <54CCE2AD.8090804@home.nl> <54CCE557.5090804@ninenines.eu> <54CCE747.8040804@home.nl> Message-ID: On Sat, Jan 31, 2015 at 9:31 AM, Roelof Wobben wrote: > Thanks, > > Everything is working now. > Finnaly I have solved the whole exercise like this : > > -module(db). > > -export([new/0, destroy/1, write/3, read/2, delete_key/3, delete/2, > match/2, match_key/3]). > > new() -> > []. > > destroy(_Db) -> > ok. > > write(Key, Element, Db) -> > [ {Key, Element} | Db ]. > > read(_Key, []) -> > {error, "Instance"}; > > read(Key, [Head | List]) -> > case element(1, Head) of > Key -> {ok, element(2, Head)}; > _ -> read(Key, List) > end. > > delete_key( _Key, [] , List2) -> > List2 ; > > delete_key( Key, [Head | Tail], List3 ) -> > case element(1, Head) of > Key -> delete_key(Key, Tail, List3 ); > _ -> delete_key(Key, Tail, [Head | List3] ) > end. > > delete( _Key, [] ) -> > {error, "Instance"}; > > delete(Key, List) -> > match_key (Key, List, []). > > match_key( _Key, [] , List) -> > List ; > > match_key( Key, [Head | Tail], List ) -> > case element(1, Head) of > Key -> match_key(Key, Tail, [element(2,Head) | List] ); > _ -> match_key(Key, Tail, List ) > end. > > match( Key, [] ) -> > {error, "Instance"}; > > match(Key, List) -> > match_key (Key, List, []). > > > Now I have 2 questions : > > 1) Is this a good solution. > Does it work? If it works, then it's potentially a good solution. But you should think about including test code so you can refactor with confidence, and to make it easier for others to provide the help you're seeking. > 2) Can I somewhere hide my helper functions (match_key, delete_key) and do > not get a warning about unused. If you're getting warnings about unused functions, there's a chance the way you're calling them doesn't match their definitions. Assuming you're working from example 3-4 of "Erlang Programming", it wants new/0, destroy/1, write/3, delete/2, read/2, and match/2 to comprise the API. So let's make that change: -export([new/0, destroy/1, write/3, delete/2, read/2, match/2]). Compiling your code with this change yields: /private/tmp/db.erl:23: Warning: function delete_key/3 is unused /private/tmp/db.erl:47: Warning: variable 'Key' is unused Why is this delete_key/3 unused? The reason is that the only place that calls it is the delete_key/3 function itself. If it's not exported, then only functions in the same module can call it, and of no other functions in the module call it, it's unused. As pointed out previously, the bug is here: delete(Key, List) -> match_key (Key, List, []). which should be: delete(Key, List) -> delete_key (Key, List, []). If we make that change, and add a leading underscore to Key on line 47, the code compiles without warning. But does it work? Let's add an exported test/0 function based on the sample session in the book to see: test() -> Db = db:new(), Db1 = db:write(francesco, london, Db), Db2 = db:write(lelle, stockholm, Db1), {ok, london} = db:read(francesco, Db2), Db3 = db:write(joern, stockholm, Db2), {error, instance} = db:read(ola, Db3), [joern, lelle] = db:match(stockholm, Db3), Db4 = db:delete(lelle, Db3), [joern] = db:match(stockholm, Db4), ok. Then run it: 5> db:test(). ** exception error: no match of right hand side value {error,"Instance"} in function db:test/0 (/private/tmp/db.erl, line 59) Line 59 is: {error, instance} = db:read(ola, Db3), The error is a failure to match the return value of read/2 against the expected {error, instance}. The problem is here: read(_Key, []) -> {error, "Instance"}; Let's change that: read(_Key, []) -> {error, instance}; Also change the failing clauses for match/2 and delete/2 the same way. Now recompile and rerun the test: 7> db:test(). ** exception error: no match of right hand side value [] in function db:test/0 (/private/tmp/db.erl, line 60) Line 60 is: [joern, lelle] = db:match(stockholm, Db3), Looks like match/2 doesn't work, since it's returning an empty list. Why? match/2 just calls match_key/3: match_key( _Key, [] , List) -> List ; match_key( Key, [Head | Tail], List ) -> case element(1, Head) of Key -> match_key(Key, Tail, [element(2,Head) | List] ); _ -> match_key(Key, Tail, List ) end. The problem is the call to element(1, Head) returns the Name element of the tuple, not the Location. You have your calls to element(1, Head) and element(2, Head) in the wrong places in this function. Let's swap them: match_key( Key, [Head | Tail], List ) -> case element(2, Head) of Key -> match_key(Key, Tail, [element(1,Head) | List] ); _ -> match_key(Key, Tail, List ) end. Recompile and re-test: 9> db:test(). ** exception error: no match of right hand side value [lelle,joern] in function db:test/0 (/private/tmp/db.erl, line 60) The test expects [joern,lelle] but match/2 now returns [lelle,joern]. Why? The reason is here, on line 43: Key -> match_key(Key, Tail, [element(1,Head) | List] ); Every time we find a match, we push the result onto the head of our accumulator list passed to the next iteration. That means the last result we find will be first in the results. This is a common issue with accumulator-based recursive functions, and normally you'd fix it by calling lists:reverse/1 to reverse the final result at line 39: match_key( _Key, [] , List) -> lists:reverse(List); But the book says no calls to the lists module are allowed, so we can't do this. Another approach is to append to the accumulator list instead, at line 43: Key -> match_key(Key, Tail, List ++ [element(1,Head)]); With this change in place, let's recompile and rerun the tests: 11> db:test(). ok Yay, the test works. Now, can we improve the code? One thing that sticks out are the calls to element/2. Those can be replaced with tuple pattern matching. For example, read/2 looks like this: read(Key, [Head | List]) -> case element(1, Head) of Key -> {ok, element(2, Head)}; _ -> read(Key, List) end. Let's get rid of those element/2 calls. The case is simply doing a match of the first element of each tuple against Key. We can do that right in the function head, like this: read(Key, [{Key,Location} | List]) -> {ok, element(2, Head)}; And since element(2,Head) is just Location, we can just make it this: read(Key, [{Key,Location} | _List]) -> {ok, Location}; OK, that works if Key matches the name in the head tuple. But what if Key doesn't match? We need another read/2 clause, originally handled by the second clause of the case statement: read(Key, [_Head | List]) -> read(Key, List). With these changes in place, do our tests still pass? 16> db:test(). ok Yep. Now, where else do we call element/2? In delete_key/3: delete_key( Key, [Head | Tail], List3 ) -> case element(1, Head) of Key -> delete_key(Key, Tail, List3 ); _ -> delete_key(Key, Tail, [Head | List3] ) end. We can change this roughly the same way we changed read/2, like this: delete_key(Key, [{Key,_Location} | Tail], List3) -> delete_key(Key, Tail, List3); delete_key(Key, [Head | Tail], List3) -> delete_key(Key, Tail, [Head | List3]). and rerun the tests: 18> db:test(). ok BTW, notice that delete_key/3 also reverses the result list. For consistency, change it to use append instead, like we did earlier for match_key/3: delete_key(Key, [{Key,_Location} | Tail], List3) -> delete_key(Key, Tail, List3); delete_key(Key, [Head | Tail], List3) -> delete_key(Key, Tail, List3 ++ [Head]). And the tests still pass. The other use of element/2 is in match_key/3, so let's fix that the same way: match_key(Key, [{Name,Key} | Tail], List) -> match_key(Key, Tail, List ++ [Name]); match_key(Key, [_Head | Tail], List) -> match_key(Key, Tail, List). The difference here from previous matches is the Key matches element 2 rather than element 1. The tests still pass with this change: 22> db:test(). ok One problem with the code is that the write/2 function unconditionally adds the name/location without checking to see if name is already there. Most people can't be in multiple locations at once, so get rid of any matching name/location first: write(Name, Location, Db) -> NewDb = case read(Name, Db) of {ok,_Location} -> delete(Name, Db); {error,instance} -> Db end, [{Name, Location} | NewDb]. We first read to see if the name is present; if so, we delete it and used the modified db to write. If not found, we just use the db passed in to write. And the tests still pass with this change, which isn't too surprising since we're not specifically testing this case. At this stage, another improvement is to rename the *_key functions -- there's no need for the _key suffix, they can instead just be named delete and match because they have arity 3, unlike their exported counterparts, which have arity 2. The arity difference prevents them from clashing. I would also stop using Key and Element as parameter names and use Name and Location instead, as they're more descriptive for the problem. With these final changes in place, the whole refactored module looks like this: -module(db). -export([new/0, destroy/1, write/3, delete/2, read/2, match/2, test/0]). new() -> []. destroy(_Db) -> ok. write(Name, Location, Db) -> NewDb = case read(Name, Db) of {ok,_Location} -> delete(Name, Db); {error,instance} -> Db end, [{Name, Location} | NewDb]. read(_Name, []) -> {error, instance}; read(Name, [{Name,Location} | _List]) -> {ok, Location}; read(Name, [_Head | List]) -> read(Name, List). delete(_Name, [] , List) -> List; delete(Name, [{Name,_Location} | Tail], List) -> delete(Name, Tail, List); delete(Name, [Head | Tail], List) -> delete(Name, Tail, List ++ [Head]). delete(_Name, []) -> {error, instance}; delete(Name, List) -> delete(Name, List, []). match( _Name, [] , List) -> List; match(Location, [{Name,Location} | Tail], List) -> match(Location, Tail, List ++ [Name]); match(Location, [_Head | Tail], List) -> match(Location, Tail, List). match(_Location, []) -> {error, instance}; match(Location, List) -> match(Location, List, []). test() -> Db = db:new(), Db1 = db:write(francesco, london, Db), Db2 = db:write(lelle, stockholm, Db1), {ok, london} = db:read(francesco, Db2), Db3 = db:write(joern, stockholm, Db2), {error, instance} = db:read(ola, Db3), [joern, lelle] = db:match(stockholm, Db3), Db4 = db:delete(lelle, Db3), [joern] = db:match(stockholm, Db4), ok. Study all these changes and make sure you understand the reasoning behind them. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From r.wobben@REDACTED Sat Jan 31 17:27:29 2015 From: r.wobben@REDACTED (Roelof Wobben) Date: Sat, 31 Jan 2015 17:27:29 +0100 Subject: [erlang-questions] wierd outome on my match method In-Reply-To: References: <54CCE2AD.8090804@home.nl> <54CCE557.5090804@ninenines.eu> <54CCE747.8040804@home.nl> Message-ID: <54CD0271.8000806@home.nl> An HTML attachment was scrubbed... URL: From e@REDACTED Sat Jan 31 17:46:51 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 17:46:51 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: Message-ID: <54CD06FB.6050608@bestmx.net> On 01/31/2015 12:40 PM, Eric Pailleau wrote: > I meant that you cannot authenticate a self signed cert true. i can not and i do not want to. and an important part of the confusion raised by the error message ids "why it allegedly tried to verify the certificate i already claimed as trustworthy as myself?" this error seems to be caused by the chain verification procedure, but *i didn't ask to run this procedure* so the pragmatic aspect of my question is "how to suppress this unwanted behavior", but i prefer to understand the implementation and the error messages, in order to prevent similar errors in the future. > unless you stored it your self as secure. that's exactly what i did with openssl last night. now openssl (being called from the command line) "trusts" all my certs but the erlang's error _persists_. From daniel.goertzen@REDACTED Sat Jan 31 18:24:04 2015 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Sat, 31 Jan 2015 11:24:04 -0600 Subject: [erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll In-Reply-To: References: <54CA6867.8020905@erlang.org> Message-ID: I don't have a particular problem using a reactor thread for fds, but it is the flow of data in the other direction, from Erlang to a thread, that causes more friction for me. A NIF thread can easily message an Erlang process with enif_send(), but the opposite is harder. Native processes easily allow for that through the message callback (assuming you've also eliminated your thread and pushed your reactor onto Erlang.) Perhaps there is some alternative approach where you could attach a pid to a pipe() and have ErlNifEnv/ERL_NIF_TERM pairs arrive at a reactor thread through that pipe? On Sat, Jan 31, 2015 at 10:12 AM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > Yes native processes would probably make drivers obsolete. We don't > exactly know what native processes will become but we could certainly > envision a better solution than the current drivers. Ideally we would have > a mechanism that could schedule native code flawlessly in user space but I > don't see that happening. > > Now, exactly what is the problem with making a thread and handling select > yourself? Is it perceived as worse performance or perceived as just > cumbersome coding? Do you need the fd to interact with other Erlang ports? > > // Bj?rn-Egil > > 2015-01-31 16:59 GMT+01:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > >> >> On Sat, Jan 31, 2015 at 4:52 PM, Daniel Goertzen < >> daniel.goertzen@REDACTED> wrote: >> >>> What is the measure of control that the VM has over drivers that it >>> would not have over native processes? I've always interpreted native >>> processes as being simpler and more capable than drivers but with exactly >>> the same hazards (executing for too long, corrupting/crashing, etc). I've >>> worked with NIFs but not drivers; perhaps there's some obvious aspect of >>> drivers that I've missed. >> >> >> Your observation is correct. Once you have a native process API, you are >> essentially on the path to eradicate drivers from the Erlang system by >> making them an extension on top of native processes. Of course, there are >> details to be hashed out, but this is a possible path to take. >> >> The fun thing is that this will also make the Erlang and Go runtimes a >> bit more like each other. >> >> >> -- >> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sat Jan 31 18:48:08 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 31 Jan 2015 18:48:08 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: From Erlang dicumentation : Do the ssl handshake. 5 server> {ok, SSLSocket} = ssl:ssl_accept(Socket, [{cacertfile, "cacerts.pem"}, {certfile, "cert.pem"}, {keyfile, "key.pem"}]). {ok,{sslsocket,[...]}} did you supplied the cacertfile tuple ? ? Envoy? depuis mon mobile ? Eric e@REDACTED a ?crit?: >On 01/31/2015 12:40 PM, Eric Pailleau wrote: >> I meant that you cannot authenticate a self signed cert > >true. i can not and i do not want to. >and an important part of the confusion raised by the error message ids >"why it allegedly tried to verify the certificate i already claimed as >trustworthy as myself?" > >this error seems to be caused by the chain verification procedure, >but *i didn't ask to run this procedure* > >so the pragmatic aspect of my question is "how to suppress this unwanted >behavior", but i prefer to understand the implementation and the error >messages, in order to prevent similar errors in the future. > > >> unless you stored it your self as secure. > >that's exactly what i did with openssl last night. >now openssl (being called from the command line) "trusts" all my certs >but the erlang's error _persists_. > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From eric.pailleau@REDACTED Sat Jan 31 19:06:56 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 31 Jan 2015 19:06:56 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: for your other remarks, you did not ask to run this procedure, but it is the normal procedure if you do ssl client authentication. SSL is nice to hide to listeners exchanges between a client and server, but if you do not authenticate the client, you can hide exchanges between your server and a hacker. disabling the normal procedure, by coding your own openssl verification callback, is possible but not recommanded unless a strong experience. Btw, I do recommand using CA with long life, and certificates with shorter lifes. otherwise you will have to update your CA bundle with your self signed, each time your self signed cert expires. ? Envoy? depuis mon mobile ? Eric From ivan@REDACTED Sat Jan 31 18:52:02 2015 From: ivan@REDACTED (Ivan Carmenates Garcia) Date: Sat, 31 Jan 2015 12:52:02 -0500 Subject: [erlang-questions] Unity and Erlang Message-ID: Hi guys, it is nice to be back, to this emailing list, I was wondering about an idea, what about a framework to communicate Unity and Erlang. In the past two or three years, I was posting about a framework I made, called EVO Extended Visual Otp, which integrated a way to connect C# and Erlang, very simple and powerful, but unfortunately nobody liked C# by those days, and I abandoned the project working fine. Now you cannot imagine my joy when I found out about Unity, I never though that something like it were even possible to exists, I always dreamed with making games, and game servers, that why I learned Erlang in the first place, but didn't knew that game engines with that power even exist!, then I found unity by accident, and was awesome when I figure out that you can program in it with C#, my another beloved language, was like a dream came true. Now I decided that I want to make EVO for Unity, since I love also Unity and I'm learn it full, but rebuilding it from scratch since in the old one, I use otp.net for internal connection between both languages, but now I want to make it simpler and using pure tcp socket connections to build the nice EVO layers on top. So I will appreciate any help regard any ways of low level connections between this two nice languages. I saw this https://coderwall.com/p/gvderg/connecting-unity3d-to-an-erlang-game-server and I like, but an extra advice would be appreciated. Best regards, Ivan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Sat Jan 31 19:20:32 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 19:20:32 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: Message-ID: <54CD1CF0.5050502@bestmx.net> On 01/31/2015 07:06 PM, Eric Pailleau wrote: > you did not ask to run this procedure, but it is the normal procedure if you do ssl client authentication. Exactly! But i hoped i did not. I do not need client auth, and i beg you to tell me how to disable it (if it is enabled by-default). i am using cowboy, and perhaps this question should be redirected to them... From eric.pailleau@REDACTED Sat Jan 31 19:25:55 2015 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 31 Jan 2015 19:25:55 +0100 Subject: [erlang-questions] SSL: "unknown ca" Message-ID: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> Maybe Lo?c have a tip... ? Envoy? depuis mon mobile ? Eric e@REDACTED a ?crit?: >On 01/31/2015 07:06 PM, Eric Pailleau wrote: >> you did not ask to run this procedure, but it is the normal procedure if you do ssl client authentication. > >Exactly! >But i hoped i did not. >I do not need client auth, and i beg you to tell me how to disable it >(if it is enabled by-default). > >i am using cowboy, and perhaps this question should be redirected to them... From essen@REDACTED Sat Jan 31 19:28:24 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 31 Jan 2015 19:28:24 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> Message-ID: <54CD1EC8.2040104@ninenines.eu> Don't look at me, we just have a wrapper on top of SSL, most SSL issues are out of my league. :-) On 01/31/2015 07:25 PM, Eric Pailleau wrote: > Maybe Lo?c have a tip... > > ? Envoy? depuis mon mobile ? Eric > > e@REDACTED a ?crit : > >> On 01/31/2015 07:06 PM, Eric Pailleau wrote: >>> you did not ask to run this procedure, but it is the normal procedure if you do ssl client authentication. >> >> Exactly! >> But i hoped i did not. >> I do not need client auth, and i beg you to tell me how to disable it >> (if it is enabled by-default). >> >> i am using cowboy, and perhaps this question should be redirected to them... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From e@REDACTED Sat Jan 31 19:31:57 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 19:31:57 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD1EC8.2040104@ninenines.eu> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> Message-ID: <54CD1F9D.7000707@bestmx.net> On 01/31/2015 07:28 PM, Lo?c Hoguin wrote: > Don't look at me, we just have a wrapper on top of SSL, most SSL issues > are out of my league. :-) this wrapper is supposed to know everything about arguments it passes, may be there is one i am looking for is among them (hiding behind an incomprehensible name, for example) From essen@REDACTED Sat Jan 31 19:35:22 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 31 Jan 2015 19:35:22 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD1F9D.7000707@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> Message-ID: <54CD206A.6060607@ninenines.eu> On 01/31/2015 07:31 PM, e@REDACTED wrote: > On 01/31/2015 07:28 PM, Lo?c Hoguin wrote: >> Don't look at me, we just have a wrapper on top of SSL, most SSL issues >> are out of my league. :-) > > this wrapper is supposed to know everything about arguments it passes, > may be there is one i am looking for is among them (hiding behind an > incomprehensible name, for example) I don't know if this is what you need as I'm not sure what your issue is exactly, but there is an option to define the verification fun (verify_fun) as others have already pointed out in this thread. Have you tried it yet? -- Lo?c Hoguin http://ninenines.eu From e@REDACTED Sat Jan 31 19:36:58 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 19:36:58 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD206A.6060607@ninenines.eu> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> Message-ID: <54CD20CA.3030509@bestmx.net> On 01/31/2015 07:35 PM, Lo?c Hoguin wrote: > On 01/31/2015 07:31 PM, e@REDACTED wrote: >> On 01/31/2015 07:28 PM, Lo?c Hoguin wrote: >>> Don't look at me, we just have a wrapper on top of SSL, most SSL issues >>> are out of my league. :-) >> >> this wrapper is supposed to know everything about arguments it passes, >> may be there is one i am looking for is among them (hiding behind an >> incomprehensible name, for example) > > I don't know if this is what you need as I'm not sure what your issue is > exactly, but there is an option to define the verification fun > (verify_fun) as others have already pointed out in this thread. Have you > tried it yet? no, it sounds to me like a last resort solution. if i am to define this function i then have very little need in the SSL itself. shouldn't it work by-default? From essen@REDACTED Sat Jan 31 21:36:20 2015 From: essen@REDACTED (=?UTF-8?Q?Lo=C3=AFc_Hoguin?=) Date: Sat, 31 Jan 2015 21:36:20 +0100 (CET) Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD20CA.3030509@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> Message-ID: <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> Replying through webmail because SMTP decided to stop working. Hope it's not too terrible when it gets into your mailboxes. > Le 31 janvier 2015 ? 19:36, "e@REDACTED" a ?crit : > > > On 01/31/2015 07:35 PM, Lo?c Hoguin wrote: > > On 01/31/2015 07:31 PM, e@REDACTED wrote: > >> On 01/31/2015 07:28 PM, Lo?c Hoguin wrote: > >>> Don't look at me, we just have a wrapper on top of SSL, most SSL issues > >>> are out of my league. :-) > >> > >> this wrapper is supposed to know everything about arguments it passes, > >> may be there is one i am looking for is among them (hiding behind an > >> incomprehensible name, for example) > > > > I don't know if this is what you need as I'm not sure what your issue is > > exactly, but there is an option to define the verification fun > > (verify_fun) as others have already pointed out in this thread. Have you > > tried it yet? > > no, it sounds to me like a last resort solution. > if i am to define this function i then have very little need in the SSL > itself. > shouldn't it work by-default? I think you are missing the point of SSL. I will try to dumb it down. SSL gives you both secrecy through encryption and authentication of the endpoint you communicate with. The latter is only possible precisely because of the certificate chain. It is used to verify that the server's certificate has been signed by a trusted entity or by an intermediary of the trusted entity. Therefore we can assume that the server is who they say they are. If you self sign a certificate then you lose this authentication aspect. Sure you still have secrecy through encryption, but anyone between you and the server can decide to decrypt and read what you send. This is called a man in the middle attack. By default SSL gives you secure connections, that means both encryption and authentication. A browser or client that tries to connect to a server that uses a self signed certificate will get an error until further steps are taken by the user to confirm they really want to connect to it. It works as intended by default. Now I am not sure what you do, what certificate you have, what kind of client you use, what options, so it is very hard to help. If it is the client that rejects your server's self signed certificate though, there's nothing you can do except getting a certificate from a trusted CA, or use an option in the client to disable the certificate verification. -- Lo?c Hoguin http://ninenines.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From e@REDACTED Sat Jan 31 21:56:03 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 21:56:03 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> Message-ID: <54CD4163.2020802@bestmx.net> On 01/31/2015 09:36 PM, Lo?c Hoguin wrote: > I think you are missing the point of SSL. I will try to dumb it down. please, don't. the only point i am missing is how to interpret the error message i have got. what is the source of this error? what entities does this error message refer to? these questions are very clear, and they do not depend on the purpose of ssl. by any chance does anyone understand this error message? (without prolix sermons) From t@REDACTED Sat Jan 31 22:00:47 2015 From: t@REDACTED (Tristan Sloughter) Date: Sat, 31 Jan 2015 15:00:47 -0600 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD4163.2020802@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> Message-ID: <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> https://www.openssl.org/docs/ssl/SSL_alert_type_string.html -- Tristan Sloughter t@REDACTED On Sat, Jan 31, 2015, at 02:56 PM, e@REDACTED wrote: > On 01/31/2015 09:36 PM, Lo?c Hoguin wrote: > > I think you are missing the point of SSL. I will try to dumb it down. > > please, don't. the only point i am missing is how to interpret the error > message i have got. > what is the source of this error? > what entities does this error message refer to? > > these questions are very clear, and they do not depend on the purpose of > ssl. > > by any chance does anyone understand this error message? > (without prolix sermons) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From imantc@REDACTED Sat Jan 31 22:06:00 2015 From: imantc@REDACTED (Imants Cekusins) Date: Sat, 31 Jan 2015 22:06:00 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD4163.2020802@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> Message-ID: > By default SSL gives you secure connections, that means both encryption and authentication. does this authentication feature necessitate a third party CA? would encryption alone work without a third party CA? From e@REDACTED Sat Jan 31 22:07:43 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:07:43 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> Message-ID: <54CD441F.4050009@bestmx.net> On 01/31/2015 10:00 PM, Tristan Sloughter wrote: > > https://www.openssl.org/docs/ssl/SSL_alert_type_string.html A valid certificate chain or partial chain was received, but the certificate was not accepted because the CA certificate could not be located or couldn't be matched with a known, trusted CA. WHICH certificate? The certificates i supplied to ssl as: 'cacertfile' 'certfile' are both valid according to the system's 'openssl': > openssl verify -CApath /etc/ssl/certs server.crt server.crt: OK From e@REDACTED Sat Jan 31 22:14:28 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:14:28 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> Message-ID: <54CD45B4.9070205@bestmx.net> On 01/31/2015 10:06 PM, Imants Cekusins wrote: >> By default SSL gives you secure connections, that means both encryption and authentication. > > does this authentication feature necessitate a third party CA? no. (and this is why i am puzzled) it involves my own CA and one cert signed by this CA and both of these certs are perfectly verifiable by openssl (as called from the command line) (i even bothered to ensure trustworthiness of my CA with the system's openssl) and i supply these certs to the erlang's ssl the following way: {cacertfile, Dir ++ "ca.crt"}, {certfile, Dir ++ "server.crt"}, {keyfile, Dir ++ "server.key"} % No other options where: server.crt = concatenation of ca.crt my.crt From t@REDACTED Sat Jan 31 22:17:54 2015 From: t@REDACTED (Tristan Sloughter) Date: Sat, 31 Jan 2015 15:17:54 -0600 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD441F.4050009@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> <54CD441F.4050009@bestmx.net> Message-ID: <1422739074.1594821.221426685.17414AB4@webmail.messagingengine.com> So whatever ca file path you are providing to the Erlang module is wrong. You asked what the error was, people have continually told you what it is. -- Tristan Sloughter t@REDACTED On Sat, Jan 31, 2015, at 03:07 PM, e@REDACTED wrote: > On 01/31/2015 10:00 PM, Tristan Sloughter wrote: > > > > https://www.openssl.org/docs/ssl/SSL_alert_type_string.html > > A valid certificate chain or partial chain was received, but the > certificate was not accepted because the CA certificate could not be > located or couldn't be matched with a known, trusted CA. > > WHICH certificate? > > The certificates i supplied to ssl as: > 'cacertfile' > 'certfile' > > are both valid according to the system's 'openssl': > > > openssl verify -CApath /etc/ssl/certs server.crt > server.crt: OK > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Sat Jan 31 22:15:10 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 31 Jan 2015 22:15:10 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> Message-ID: <54CD45DE.6050408@ninenines.eu> On 01/31/2015 10:06 PM, Imants Cekusins wrote: >> By default SSL gives you secure connections, that means both encryption and authentication. > > does this authentication feature necessitate a third party CA? It depends on everyone agreeing which root CAs should be trusted and keeping that trust list up to date as it changes over time. Note that by "everyone" here I mean "all interested parties", aka the clients and servers involved. You can have your own personal CA in the list of the trusted CAs if it is all your clients and servers. > would encryption alone work without a third party CA? Yes, but the connections will be rejected by default unless the client is specifically configured to either trust this CA or not perform the verification. -- Lo?c Hoguin http://ninenines.eu From e@REDACTED Sat Jan 31 22:19:57 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:19:57 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <1422739074.1594821.221426685.17414AB4@webmail.messagingengine.com> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> <54CD441F.4050009@bestmx.net> <1422739074.1594821.221426685.17414AB4@webmail.messagingengine.com> Message-ID: <54CD46FD.90000@bestmx.net> On 01/31/2015 10:17 PM, Tristan Sloughter wrote: > So whatever ca file path you are providing to the Erlang module is > wrong. it is unlikely, because, i once have set wrong access perms to these files and i have imeddiately got 'eaccess' instead of the "unknown ca" > You asked what the error was no. i have asked "WHAT ENTITIES DOES THIS ERROR REFER TO?" From eric.pailleau@REDACTED Sat Jan 31 22:24:13 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 31 Jan 2015 22:24:13 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD45B4.9070205@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <54CD45B4.9070205@bestmx.net> Message-ID: <54CD47FD.1070601@wanadoo.fr> Le 31/01/2015 22:14, e@REDACTED a ?crit : > {cacertfile, Dir ++ "ca.crt"}, > {certfile, Dir ++ "server.crt"}, > {keyfile, Dir ++ "server.key"} What is Dir ? BTW you should use filename:join/2 instead ++ ... From eric.pailleau@REDACTED Sat Jan 31 22:27:19 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 31 Jan 2015 22:27:19 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD47FD.1070601@wanadoo.fr> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <54CD45B4.9070205@bestmx.net> <54CD47FD.1070601@wanadoo.fr> Message-ID: <54CD48B7.2070100@wanadoo.fr> Le 31/01/2015 22:24, PAILLEAU Eric a ?crit : > Le 31/01/2015 22:14, e@REDACTED a ?crit : >> {cacertfile, Dir ++ "ca.crt"}, >> {certfile, Dir ++ "server.crt"}, >> {keyfile, Dir ++ "server.key"} > > What is Dir ? > > BTW you should use filename:join/2 instead ++ ... I precise my question : - I understood you use cowboy as server, but are you using an Erlang client ? - Are you using same directories/cert files for server and client ? From essen@REDACTED Sat Jan 31 22:16:27 2015 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 31 Jan 2015 22:16:27 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD441F.4050009@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> <54CD441F.4050009@bestmx.net> Message-ID: <54CD462B.8030406@ninenines.eu> On 01/31/2015 10:07 PM, e@REDACTED wrote: > On 01/31/2015 10:00 PM, Tristan Sloughter wrote: >> >> https://www.openssl.org/docs/ssl/SSL_alert_type_string.html > > A valid certificate chain or partial chain was received, but the > certificate was not accepted because the CA certificate could not be > located or couldn't be matched with a known, trusted CA. > > WHICH certificate? > > The certificates i supplied to ssl as: > 'cacertfile' > 'certfile' > > are both valid according to the system's 'openssl': > > > openssl verify -CApath /etc/ssl/certs server.crt > server.crt: OK I answered that in my previous email. The client and the server must agree on the list of trusted CAs. Did you also provide the same cacertfile file to the client? -- Lo?c Hoguin http://ninenines.eu From e@REDACTED Sat Jan 31 22:32:14 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:32:14 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD47FD.1070601@wanadoo.fr> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <54CD45B4.9070205@bestmx.net> <54CD47FD.1070601@wanadoo.fr> Message-ID: <54CD49DE.2090608@bestmx.net> On 01/31/2015 10:24 PM, PAILLEAU Eric wrote: > Le 31/01/2015 22:14, e@REDACTED a ?crit : >> {cacertfile, Dir ++ "ca.crt"}, >> {certfile, Dir ++ "server.crt"}, >> {keyfile, Dir ++ "server.key"} > > What is Dir ? Dir = code:priv_dir(online37) ++ "/../../../../etc/ssl/" and yes this dir is readable: -r--r--r-- 1 online37 root 887 Jan 31 03:28 ca.crt -r--r--r-- 1 online37 root 1766 Jan 31 03:28 server.crt -r-------- 1 online37 root 891 Jan 31 03:28 server.key From e@REDACTED Sat Jan 31 22:36:33 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:36:33 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD462B.8030406@ninenines.eu> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <1422738047.1591823.221422849.06EAB432@webmail.messagingengine.com> <54CD441F.4050009@bestmx.net> <54CD462B.8030406@ninenines.eu> Message-ID: <54CD4AE1.5020407@bestmx.net> On 01/31/2015 10:16 PM, Lo?c Hoguin wrote: > On 01/31/2015 10:07 PM, e@REDACTED wrote: >> On 01/31/2015 10:00 PM, Tristan Sloughter wrote: >>> >>> https://www.openssl.org/docs/ssl/SSL_alert_type_string.html >> >> A valid certificate chain or partial chain was received, but the >> certificate was not accepted because the CA certificate could not be >> located or couldn't be matched with a known, trusted CA. >> >> WHICH certificate? >> >> The certificates i supplied to ssl as: >> 'cacertfile' >> 'certfile' >> >> are both valid according to the system's 'openssl': >> >> > openssl verify -CApath /etc/ssl/certs server.crt >> server.crt: OK > > I answered that in my previous email. > > The client and the server must agree on the list of trusted CAs. Did you > also provide the same cacertfile file to the client? yes, besides it is included in the server.crt i also manually import both certs into my browser, one as an 'authority' and another as a 'server' From e@REDACTED Sat Jan 31 22:38:53 2015 From: e@REDACTED (e@REDACTED) Date: Sat, 31 Jan 2015 22:38:53 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD48B7.2070100@wanadoo.fr> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <54CD45B4.9070205@bestmx.net> <54CD47FD.1070601@wanadoo.fr> <54CD48B7.2070100@wanadoo.fr> Message-ID: <54CD4B6D.20104@bestmx.net> On 01/31/2015 10:27 PM, PAILLEAU Eric wrote: > Le 31/01/2015 22:24, PAILLEAU Eric a ?crit : >> Le 31/01/2015 22:14, e@REDACTED a ?crit : >>> {cacertfile, Dir ++ "ca.crt"}, >>> {certfile, Dir ++ "server.crt"}, >>> {keyfile, Dir ++ "server.key"} >> >> What is Dir ? >> >> BTW you should use filename:join/2 instead ++ ... > > I precise my question : > > - I understood you use cowboy as server, but are you using an Erlang > client ? > - Are you using same directories/cert files for server and client ? client is a browser (iceweasel) protocol is "wss" damit, i ought to try the same config with the different protocol, say https (which a have successfully brought up numerous times before) From eric.pailleau@REDACTED Sat Jan 31 22:52:55 2015 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 31 Jan 2015 22:52:55 +0100 Subject: [erlang-questions] SSL: "unknown ca" In-Reply-To: <54CD4B6D.20104@bestmx.net> References: <3h1xqxrq1xeoikq5qoulydau.1422728755306@email.android.com> <54CD1EC8.2040104@ninenines.eu> <54CD1F9D.7000707@bestmx.net> <54CD206A.6060607@ninenines.eu> <54CD20CA.3030509@bestmx.net> <126074499.136820.1422736581055.JavaMail.open-xchange@oxbaltgw35.schlund.de> <54CD4163.2020802@bestmx.net> <54CD45B4.9070205@bestmx.net> <54CD47FD.1070601@wanadoo.fr> <54CD48B7.2070100@wanadoo.fr> <54CD4B6D.20104@bestmx.net> Message-ID: <54CD4EB7.4080605@wanadoo.fr> You are mixing OSI layers. If the SSL handshake fails, there is nothing to do with the protocol that want to go into the SSL tunnel. Regard