From himars@REDACTED Sat Apr 1 05:35:54 2017 From: himars@REDACTED (Jack Tang) Date: Sat, 1 Apr 2017 11:35:54 +0800 Subject: [erlang-questions] How can I release beam process memory? In-Reply-To: References: Message-ID: After setting up erlang memory visualization, we find etc allocator does not release the memory during some historical datum are remove from mnesia tables. Can I release the memory on the fly rather than restart the mnesia application? Thanks! BR On Sun, Jan 15, 2017 at 4:47 AM, D?niel Szoboszlay wrote: > Hi Jack, > > I guess the 9 GB is lost due to memory fragmentation. Erlang allocates > memory in large chunks called carriers from the OS, then places the blocks > your program actually needs on these carriers. A carrier can only be > returned to the OS once all the blocks on it have been freed (and even > then, the memory allocator may decide to keep it around for a while in case > more memory is needed). > > You can check with recon_alloc > how much unused memory is > lost due to fragmentation in the various allocators. > > The bad news is that you cannot defragment the carriers, and if the > selected memory allocator strategy doesn't work well for your application, > you cannot change it either without restarting the emulator. > > However, if the memory is wasted in the eheap_alloc, you may try to force > a GC on all processes a couple of times. As the GC copies the memory, it > will allocate new blocks and free up the old heap blocks. So there's a > chance the allocators can compact the blocks together on fewer segments. > But that's just a guess, it may or may not work at all. > > Cheers, > Daniel > > On Sat, 14 Jan 2017 at 08:04 Jack Tang wrote: > >> Hello list, >> >> I run one Erlang application on Debian server and today I find the beam >> process consumes around 35G memory by `top` command. >> >> ``` >> KiB Mem: 99194912 total, 61682656 used, 37512252 free, 397380 buffers >> KiB Swap: 0 total, 0 used, 0 free. 18684864 cached >> Mem >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ >> COMMAND >> 11858 usr1 20 0 36.850g 0.032t 6220 S 73.5 34.4 8038:49 beam.smp >> ``` >> >> I connect to the Erlang application using remote shell and find the >> mem-leaked supervisor tree and run gc on the whole tree. Code looks like >> blow: >> >> ``` >> lists:foreach(fun(E) -> PId = element(2, E), erlang:garbage_collect(PId) >> end, supervisor:which_children(some_thing_sup)). >> ``` >> >> and erlang:memory() decreases from 32G to 23G. >> ``` >> [{total,22982011544}, >> {processes,12884182336}, >> {processes_used,12884170336}, >> {system,10097829208}, >> {atom,13828705}, >> {atom_used,13796692}, >> {binary,170530288}, >> {code,16450626}, >> {ets,9637717576}] >> ``` >> >> However, when I input `top` command, the beam process still takes 35G >> memory. What can I do to release the 9G memory? Thanks >> >> BR >> -Jack >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -- Jack Tang http://www.linkedin.com/in/jacktang -------------- next part -------------- An HTML attachment was scrubbed... URL: From ted.ml.erlang@REDACTED Sat Apr 1 06:40:37 2017 From: ted.ml.erlang@REDACTED (Ted Burghart) Date: Sat, 1 Apr 2017 00:40:37 -0400 Subject: [erlang-questions] Implementing enif_whereis In-Reply-To: <4369DB3F-5B30-494D-AB73-A26CC7B05473@tedb.net> References: <4369DB3F-5B30-494D-AB73-A26CC7B05473@tedb.net> Message-ID: <9B547DD2-E80F-43D7-9B7A-CCBE3D830196@tedb.net> Now available as a PR at https://github.com/erlang/otp/pull/1400 I've tested it fairly heavily, and all of the tests that are environment-neutral are included in the CT suites. The PR comment *should* explain my rationale, but I'd be happy to field questions or comments. ? Ted > On 30-Mar 2017, at 11:01 AM, Ted Burghart wrote: > > My working code is at https://github.com/tburghart/otp/tree/trb/erts/enif_whereis > > I'm adding tests for behavior on dirty schedulers and background threads, which may result in checks in enif_whereis to bail out in contexts where it can't execute properly. > > Comments are welcome! > >> On 30-Mar 2017, at 7:41 AM, Ted Burghart > wrote: >> >> Hello, >> >> I've implemented an enif_whereis function on a branch of master (OTP-20), and I have a couple of questions. >> >> This turned out to be pretty simple, and I'm wondering if something this small needs to go through the full EEP process or if it can be PR'd in, and does it need a Jira issue? >> >> The current function implementation (in erl_nif.c) is as follows: >> >> int enif_whereis(ErlNifEnv *env, ERL_NIF_TERM name, ERL_NIF_TERM *out) >> { >> if (is_atom(name)) { >> Process* c_p; >> Eterm res; >> >> execution_state(env, &c_p, NULL); >> res = erts_whereis_name_to_id(c_p, name); >> >> if (is_not_atom(res)) { >> *out = enif_make_copy(env, res); >> return 1; >> } >> } >> return 0; >> } >> >> This works fine for pids and ports, and one of my tests calls it from lots of parallel processes without any issues (both with and without lock checking enabled). >> >> It's got docs and tests, and I'm preparing the EEP Lite commit comment to be pushed shortly. >> >> So, anyone want to throw some darts at it? >> >> ? Ted >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sun Apr 2 00:38:28 2017 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 1 Apr 2017 18:38:28 -0400 Subject: [erlang-questions] Pipe, maybe, and other silly parse transforms Message-ID: <20170401223827.GI641@ferdmbp.local> Hi everyone! On this April 1st, I decided to do a few foolish things, and implement an Erlang equivalent to Elixir's Pipe along with a 'maybe' construct. The first ideas for this I had are almost 2 years old now and are archived at http://erlang.org/pipermail/erlang-questions/2015-July/085109.html Well since today's a silly day, I spent a bit of time and made an actual version of it that can be toyed with, and put it in a library called 'fancyflow' (https://github.com/ferd/fancyflow) You can now do fancy control flow refactorings to move from code like: String = "a b c d e f", string:join( lists:map(fun string:to_upper/1, string:tokens(String, " ")), "," ). into: [pipe]("a b c d e f", string:tokens(_, " "), lists:map(fun string:to_upper/1, _), string:join(_, ",")). and change conditionals like: case file:get_cwd() of {ok, Dir} -> case file:read_file(filename:join([Dir, "demo", "data.txt"])) of {ok, Bin} -> {ok, {byte_size(Bin), Bin}}; {error, Reason} -> {error, Reason} end; {error, Reason} -> {error, Reason} end. into: [maybe](undefined, file:get_cwd(), file:read_file(filename:join([_, "demo", "data.txt"])), {ok, {byte_size(_), _}}). More details in the README. I can't actually advise or recommend using the library, but I felt like sharing it anyway :) Regards, Fred. From aschultz@REDACTED Sun Apr 2 12:55:19 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Sun, 2 Apr 2017 12:55:19 +0200 (CEST) Subject: [erlang-questions] Pipe, maybe, and other silly parse transforms In-Reply-To: <20170401223827.GI641@ferdmbp.local> References: <20170401223827.GI641@ferdmbp.local> Message-ID: <791321751.711294.1491130519259.JavaMail.zimbra@tpip.net> Hi Fred, ----- On Apr 2, 2017, at 12:38 AM, Fred Hebert mononcqc@REDACTED wrote: > Hi everyone! > > On this April 1st, I decided to do a few foolish things, and implement > an Erlang equivalent to Elixir's Pipe along with a 'maybe' construct. Out of curiosity, how does this differ from erlando's (https://github.com/rabbitmq/erlando) `state` monad ? The syntax does seem to a bit more pleasing, but as far as I can tell the result is about the same. Thanks, Andreas > The first ideas for this I had are almost 2 years old now and are > archived at > http://erlang.org/pipermail/erlang-questions/2015-July/085109.html > > Well since today's a silly day, I spent a bit of time and made an actual > version of it that can be toyed with, and put it in a library called > 'fancyflow' (https://github.com/ferd/fancyflow) > > You can now do fancy control flow refactorings to move from code like: > > String = "a b c d e f", > string:join( > lists:map(fun string:to_upper/1, string:tokens(String, " ")), > "," > ). > > into: > > [pipe]("a b c d e f", > string:tokens(_, " "), > lists:map(fun string:to_upper/1, _), > string:join(_, ",")). > > and change conditionals like: > > case file:get_cwd() of > {ok, Dir} -> > case file:read_file(filename:join([Dir, "demo", "data.txt"])) of > {ok, Bin} -> > {ok, {byte_size(Bin), Bin}}; > {error, Reason} -> > {error, Reason} > end; > {error, Reason} -> > {error, Reason} > end. > > into: > > [maybe](undefined, > file:get_cwd(), > file:read_file(filename:join([_, "demo", "data.txt"])), > {ok, {byte_size(_), _}}). > > More details in the README. > > I can't actually advise or recommend using the library, but I felt like > sharing it anyway :) > > Regards, > Fred. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mononcqc@REDACTED Sun Apr 2 14:23:15 2017 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 2 Apr 2017 08:23:15 -0400 Subject: [erlang-questions] Pipe, maybe, and other silly parse transforms In-Reply-To: <791321751.711294.1491130519259.JavaMail.zimbra@tpip.net> References: <20170401223827.GI641@ferdmbp.local> <791321751.711294.1491130519259.JavaMail.zimbra@tpip.net> Message-ID: <20170402122314.GK641@ferdmbp.local> On 04/02, Andreas Schultz wrote: >Hi Fred, > >Out of curiosity, how does this differ from erlando's >(https://github.com/rabbitmq/erlando) `state` monad ? > >The syntax does seem to a bit more pleasing, but as far as I can tell the >result is about the same. > >Thanks, >Andreas > There's a few differences: - by tripping onto [term](...) as accepted syntax for an obviously invalid function call, we can *add* semantics to an expression that had none rather than overloading those of an exisiting construct - by virtue of not being a monad fancyflow is also far less generic and doesn't allow overloading of the >>= nor return operators, which gives Erlando a lot of flexibility that fancyflow does not have - fancyflow does not use monad terminology, and hence should be immune to burrito-related tutorials - we avoid all of that pesky imperative programming by banishing the word 'do' and instead using the descriptive words 'pipe' and 'maybe', which totally express what is going to happen - the monadic approach can be composed and nested, fancyflow isn't that fancy; this probably brings it to the same place as 'fancy feast' cat food, which I am told isn't that fancy of a feast* - the 'cut' syntax is bound in context a bit more, which lets me work around this Erlando issue I had opened in 2011 https://github.com/rabbitmq/erlando/issues/2 (props to them for never closing it in anger) - Erlando was possibly intended to actually be used by its authors, whereas fancyflow was just me trying to toy with that idea I had posted in the mailing list a few years ago Again, I can't really advise using either of these libraries in a code base. One of the big strengths of Erlang is its simplicity and ease of jumping into code, any code, and easily understand what it does. The use of parse transforms to overload or add new semantic meaning to existing constructs is a thing that should be done with care. There was certainly not that much care involved in fancyflow's case, since the RFC process involved me writing on a mailing list and then saying "screw this it's april 1st let's try it" about 2 years after. Regards, Fred. * as a general rule, if a thing needs to assert what it is in the name, it likely does not bear that property. To wit: - 'fancy feast' does not make for fancy feasts, as mentioned earlier - the S in SOAP stands for 'simple' which by now should have us all laughing; see also a bunch of amazon services - restaurants with 'gourmet' in their names (or brightly flashing neon signs) tend not to inspire much confidence - calling yourself a classy person likely means you are not that classy to begin with, though if you refer to marxist ideals then it may be an easier point to debate - I guess if you have an object oriented language you could also describe it as classy and it would be technically correct - flame decals and/or stripes on a car do not make the car go faster, no matter what my former 4 years old self would like to assert. Dodge vipers ain't that cool, 4 years old Fred! You'll drive a humble grey versa for at least 10 years anyway when you're older so don't pretend to be that hot From ok@REDACTED Mon Apr 3 03:16:54 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 3 Apr 2017 13:16:54 +1200 Subject: [erlang-questions] Pipe, maybe, and other silly parse transforms In-Reply-To: <20170402122314.GK641@ferdmbp.local> References: <20170401223827.GI641@ferdmbp.local> <791321751.711294.1491130519259.JavaMail.zimbra@tpip.net> <20170402122314.GK641@ferdmbp.local> Message-ID: <6DF925CB-54D2-4B0E-8471-CB7D84C9E7C2@cs.otago.ac.nz> > On 3/04/2017, at 12:23 AM, Fred Hebert wrote: > - the 'cut' syntax is bound in context a bit more, which lets me work around this Erlando issue I had opened in 2011 https://github.com/rabbitmq/erlando/issues/2 (props to them for never closing it in anger) I have a resurrection of Pop-2 that I've been working on in my spare time. (Nostalgia isn't what it used to be. Back then it was lime-flavoured.) Think of a cross between Lisp data, Algol syntax, and a Forth/Postscript- like visible stack designed in the 1960s. As part of spiffing that up, I've been looking through the SRFIs for ideas to steal. I can across 'cut' and 'cute' and thought, hmmm. It turns out that there is one lambda-expression every 11 SLOC Only about 5% of those could use 'cut' or 'cute'. I came to the conclusion that it is MUCH better not to have 'cut'/'cute'. From aschultz@REDACTED Mon Apr 3 12:46:19 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Mon, 3 Apr 2017 12:46:19 +0200 (CEST) Subject: [erlang-questions] ssl: TLS-1.2 signature extension Message-ID: <738269119.718970.1491216379968.JavaMail.zimbra@tpip.net> Hi, I've come across a curious TLS 1.2 connection problem. Erlang 19.x will not connect to https://validator5.addressdoctor.com/ in the default configuration or when more than TLS 1.2 is offered. After some digging I found out that the server (IIS-8.0) would attempt to use TLS 1.2, but chokes on the *missing* Signature Algorithms extension. This is clearly a bug in the server implementation, as it should assume a default value for the supported signature algorithms. Nevertheless, every other TLS client works normally. Erlang SSL behavior is 100% compliant with RFC-5246: > Note: this extension is not meaningful for TLS versions prior to 1.2. > Clients MUST NOT offer it if they are offering prior versions. Tests with OpenSSL and GNU-TLS show that they ignore the "MUST NOT" from the above clause and include the Signature Algorithms extension whenever they offer TLS 1.2 regardless of a possible fallback to a lower version. I think Erlang's SSL should do the same. There is also an alternative way to understand the clause. It's meaning depends on "offering prior version". The version offered is the value from ClientHello.client_version, so as soon as this value is {3, 3}, a Signature Algorithms extension should be sent. Regards Andreas From lukas@REDACTED Mon Apr 3 15:33:28 2017 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 3 Apr 2017 15:33:28 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator Message-ID: Hello! At the moment it is possible to build three different production versions of the Erlang VM; the smp emulator, the threaded non-smp emulator and the non-threaded non-smp emulator. The non-smp versions have been kept around since the introduction of the smp emulator as they can run on OS:s without thread support and also for certain applications they provide a small performance boost. However we now feel that they have outlived their usefulness and plan to remove them. The main reason why we want to remove non-smp is because we do not want to spend time on implementing the dirty-scheduler feature for threaded non-smp, which is something we would have to do for the threaded non-smp emulator to remain production quality. The different variants also increase our test-scope by quite a lot, so reducing that is a very beneficial side effect. The current plan is for the non-smp emulators to be deprecated and not built by default in OTP-20, and then removed completely in OTP-21. If you have a very good reason for wanting to use the non-smp emulator, please do let us know as soon as possible so that we can consider whether we want to keep the non-smp version anyway. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Mon Apr 3 15:50:31 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Mon, 3 Apr 2017 15:50:31 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: One point that I've heard a couple of times, is that single CPU emulator (not sure which one of them) is noticeably (like up to 20%) faster than SMP due to all the locks it doesn't have to maintain. So for some applications running say 8 of these will be better than a single SMP 8 core emulator. This is a casual opinion from the couch and I don't have the numbers to prove the point :) 2017-04-03 15:33 GMT+02:00 Lukas Larsson : > Hello! > > At the moment it is possible to build three different production versions > of the Erlang VM; the smp emulator, the threaded non-smp emulator and the > non-threaded non-smp emulator. > > The non-smp versions have been kept around since the introduction of the > smp emulator as they can run on OS:s without thread support and also for > certain applications they provide a small performance boost. However we now > feel that they have outlived their usefulness and plan to remove them. > > The main reason why we want to remove non-smp is because we do not want to > spend time on implementing the dirty-scheduler feature for threaded > non-smp, which is something we would have to do for the threaded non-smp > emulator to remain production quality. The different variants also increase > our test-scope by quite a lot, so reducing that is a very beneficial side > effect. > > The current plan is for the non-smp emulators to be deprecated and not > built by default in OTP-20, and then removed completely in OTP-21. > > If you have a very good reason for wanting to use the non-smp emulator, > please do let us know as soon as possible so that we can consider whether > we want to keep the non-smp version anyway. > > Lukas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ameretat.reith@REDACTED Mon Apr 3 16:49:23 2017 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Mon, 3 Apr 2017 19:19:23 +0430 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: We used to run containerized non-SMP VMs and we were under that impression it scales better, but I don't have numbers too. Since we don't use dirty schedulers and software is more Erlang than Erlang bounded to C, SMP looked unnecessary overkill for us, but If It's planned that dirty scheduler spread to core applications -say crypto module depend on it- I prefer non-SMP support dropped rather than my software stuck in a random function. From lukas@REDACTED Mon Apr 3 16:56:15 2017 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 3 Apr 2017 16:56:15 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: On Mon, Apr 3, 2017 at 4:49 PM, Ameretat Reith wrote: > If It's planned that dirty scheduler spread to core applications -say > crypto > module depend on it- I prefer non-SMP support dropped rather than my > software stuck in a random function. > The first and largest thing we are planning to do is to implement file I/O as nifs using dirty schedulers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Mon Apr 3 18:34:51 2017 From: t@REDACTED (Tristan Sloughter) Date: Mon, 03 Apr 2017 09:34:51 -0700 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right? This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. Because of tasks like common test, dialyzer and in the future if we add back parallel compile (for the most common case, only building a few files, it is much faster to just build sequentially) we can't use non- smp by default with rebar3. But I figured I'd throw this out there in case anyone was looking to speed up the boot time :) -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Apr 4 09:55:38 2017 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 4 Apr 2017 09:55:38 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> References: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> Message-ID: On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter wrote: > First use case I thought of when I heard this news was 'erlc' itself :). > It disables smp by default, right? > erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean" So, it uses the smp emulator if available. > This gives a few milliseconds speedup in startup time when booting the VM > which is mostly a non-issue but nice for cli tools. > If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ingela.Anderton.Andin@REDACTED Tue Apr 4 16:21:21 2017 From: Ingela.Anderton.Andin@REDACTED (Ingela Anderton Andin) Date: Tue, 4 Apr 2017 16:21:21 +0200 Subject: [erlang-questions] Patch package OTP 19.3.1 released Message-ID: <268925b3-309f-3fb0-7973-086dab90cfe1@ericsson.com> Patch Package: OTP 19.3.1 Git Tag: OTP-19.3.1 Date: 2017-04-04 Trouble Report Id: OTP-14242, OTP-14290, OTP-14298, OTP-14300, OTP-14308, OTP-14313 Seq num: seq13294 System: OTP Release: 19 Application: crypto-3.7.4, erts-8.3.1, inets-6.3.7, ssh-4.4.2, ssl-8.1.2 Predecessor: OTP 19.3 Check out the git tag OTP-19.3.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- crypto-3.7.4 ---------------------------------------------------- --------------------------------------------------------------------- The crypto-3.7.4 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-14313 Application(s): crypto Related Id(s): PR-1393 Fix a bug with AES CFB 128 for 192 and 256 bit keys. Thanks to kellymclaughlin ! Full runtime dependencies of crypto-3.7.4: erts-6.0, kernel-3.0, stdlib-2.0 --------------------------------------------------------------------- --- erts-8.3.1 ------------------------------------------------------ --------------------------------------------------------------------- Note! The erts-8.3.1 application can *not* be applied independently of other applications on an arbitrary OTP 19 installation. On a full OTP 19 installation, also the following runtime dependency has to be satisfied: -- sasl-3.0.1 (first satisfied in OTP 19.1) --- Fixed Bugs and Malfunctions --- OTP-14308 Application(s): erts Related Id(s): ERL-383 Trying to open a directory with file:read_file/1 on Unix leaked a file descriptor. This bug has now been fixed. --- Known Bugs and Problems --- OTP-14290 Application(s): erts Invoking init:stop/0 via the SIGTERM signal, in a non-SMP BEAM, could cause BEAM to terminate with fatal error. This has now been fixed and the BEAM will terminate normally when SIGTERM is received. Full runtime dependencies of erts-8.3.1: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --- inets-6.3.7 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.3.7 application can be applied independently of other applications on a full OTP 19 installation. --- Fixed Bugs and Malfunctions --- OTP-14242 Application(s): inets Fixed a bug in ftp that made further operations after a recv_chunk operation impossible. Full runtime dependencies of inets-6.3.7: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- ssh-4.4.2 ------------------------------------------------------- --------------------------------------------------------------------- Note! The ssh-4.4.2 application can *not* be applied independently of other applications on an arbitrary OTP 19 installation. On a full OTP 19 installation, also the following runtime dependencies have to be satisfied: -- crypto-3.7.3 (first satisfied in OTP 19.3) -- public_key-1.4 (first satisfied in OTP 19.3) -- stdlib-3.3 (first satisfied in OTP 19.3) --- Fixed Bugs and Malfunctions --- OTP-14298 Application(s): ssh Related Id(s): seq13294 ssh:daemon_info/1 crashed if the listening IP was not 'any' Full runtime dependencies of ssh-4.4.2: crypto-3.7.3, erts-6.0, kernel-3.0, public_key-1.4, stdlib-3.3 --------------------------------------------------------------------- --- ssl-8.1.2 ------------------------------------------------------- --------------------------------------------------------------------- Note! The ssl-8.1.2 application can *not* be applied independently of other applications on an arbitrary OTP 19 installation. On a full OTP 19 installation, also the following runtime dependency has to be satisfied: -- stdlib-3.2 (first satisfied in OTP 19.2) --- Fixed Bugs and Malfunctions --- OTP-14300 Application(s): ssl Correct active once emulation, for TLS. Now all data received by the connection process will be delivered through active once, even when the active once arrives after that the gen_tcp socket is closed by the peer. Full runtime dependencies of ssl-8.1.2: crypto-3.3, erts-7.0, inets-5.10.7, kernel-3.0, public_key-1.2, stdlib-3.2 --------------------------------------------------------------------- --- Thanks to ------------------------------------------------------- --------------------------------------------------------------------- Bram Verburg, Kelly McLaughlin --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From krzysztof.jurewicz@REDACTED Tue Apr 4 17:29:27 2017 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Tue, 04 Apr 2017 17:29:27 +0200 Subject: [erlang-questions] =?utf-8?b?W0FOTl0gYWJjaV9zZXJ2ZXIg4oCUIGZvciBi?= =?utf-8?q?uilding_blockchain_apps?= Message-ID: <87efx88a3s.fsf@gmail.com> abci_server has been released. It allows writing Application Blockchain Interface servers. Basically a developer writes a state machine and abci_server along with Tendermint ( https://tendermint.com ) takes care of the blockchain consensus. The code is available at: https://github.com/KrzysiekJ/abci_server An example application is available at: https://github.com/KrzysiekJ/abci_counter Snapshot of the documentation for 0.1.0 release has been put on IPFS: https://ipfs.io/ipfs/Qmb4T6XcYJENWVxVMKaDkqTb3WYz6Z12s8HNhcBF4vGDko/ Patches, comments and other feedback are welcome. From abhishek@REDACTED Tue Apr 4 15:54:23 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Tue, 4 Apr 2017 19:24:23 +0530 (IST) Subject: [erlang-questions] running custom module in ejabberd by using hooks Message-ID: <239877629.58108.1491314064036@webmail.blacklightsw.com> I have developed an application in erlang and have place it inside the ebin folder of ejabberd. I had to decide a server to be able to host the game hence I chose ejabberd for it.I am using Pidgin as a client side messaging server.My strategy was to detect the message stanza sent by the client and extract the actual message from its body tag and use it as a function call to run my application. I have created a custom module as -module(customMod). -behaviour(gen_mod). -export([start/2,stop/1,depends/2,mod_opt_type/1,extractMessage/1]). -include("logger.hrl"). -include("xmpp.hrl"). -include("ejabberd.hrl"). start(_Host, _Opts) -> ejabberd_hooks:add(user_send_packet, _Host, ?MODULE, extractMessage, 95). stop(_Host) -> ejabberd_hooks:delete(user_send_packet, _Host, ?MODULE, extractMessage,95). depends(_Host, _Opts)->[{?MODULE,soft}]. mod_opt_type(_Option)-> ok. extractMessage(Args)-> A=10, ?INFO_MSG("hello buddy ~p",[A]), Args. Now to call this module I have written the following code snippet in ejabberd_router inside do_route: do_route(OrigPacket) -> ?DEBUG("route:~n~s", [xmpp:pp(OrigPacket)]), A=1, To1 = xmpp:get_to(OrigPacket), LDstDomain = To1#jid.lserver, case xmpp:get_type(OrigPacket) of get-> get; result-> result; error-> error; probe-> probe; available-> available; set-> set; chat-> ?INFO_MSG("Inside ejabberd router value of PacketType is ~p",[A]), ejabberd_hooks:run_fold(user_receive_packet, LDstDomain, OrigPacket, ["dharun"]) end, case ejabberd_hooks:run_fold(filter_packet, OrigPacket, []) of drop -> ok; Packet -> To = xmpp:get_to(Packet), LDstDomain = To#jid.lserver, Mod = get_backend(), case Mod:find_routes(LDstDomain) of [] -> ejabberd_s2s:route(Packet); [Route] -> do_route(Packet, Route); Routes -> >From = xmpp:get_from(Packet), balancing_route(From, To, Packet, Routes) end, ok end. So according to me my custom module should be called fro ejabberd_router but it is being called as I start my ejabberd and that to it is being called multiple times ejabberd_log is: (ejabberd@REDACTED)1> 18:44:06.134 [info] Application lager started on node ejabberd@REDACTED 18:44:06.137 [info] Application crypto started on node ejabberd@REDACTED 18:44:06.146 [info] Application sasl started on node ejabberd@REDACTED 18:44:06.156 [info] Application asn1 started on node ejabberd@REDACTED 18:44:06.156 [info] Application public_key started on node ejabberd@REDACTED 18:44:06.164 [info] Application ssl started on node ejabberd@REDACTED 18:44:06.176 [info] Application fast_yaml started on node ejabberd@REDACTED 18:44:06.191 [info] Application fast_tls started on node ejabberd@REDACTED 18:44:06.207 [info] Application fast_xml started on node ejabberd@REDACTED 18:44:06.212 [info] Application p1_utils started on node ejabberd@REDACTED 18:44:06.215 [info] Application stringprep started on node ejabberd@REDACTED 18:44:06.217 [info] Application xmpp started on node ejabberd@REDACTED 18:44:06.229 [info] Application cache_tab started on node ejabberd@REDACTED 18:44:06.252 [info] Application elixir started on node ejabberd@REDACTED 18:44:06.419 [info] Application mnesia started on node ejabberd@REDACTED 18:44:06.542 [warning] Module mod_last_odbc is deprecated, use mod_last with 'db_type: sql' instead 18:44:06.554 [warning] Module mod_offline_odbc is deprecated, use mod_offline with 'db_type: sql' instead 18:44:06.561 [warning] Module mod_privacy_odbc is deprecated, use mod_privacy with 'db_type: sql' instead 18:44:06.566 [warning] Module mod_private_odbc is deprecated, use mod_private with 'db_type: sql' instead 18:44:06.570 [warning] Module mod_pubsub_odbc is deprecated, use mod_pubsub with 'db_type: sql' instead 18:44:06.575 [warning] Module mod_roster_odbc is deprecated, use mod_roster with 'db_type: sql' instead 18:44:06.591 [warning] Module mod_vcard_odbc is deprecated, use mod_vcard with 'db_type: sql' instead 18:44:07.355 [info] Adding machine's DNS IPs to Erlang system: [] 18:44:07.367 [info] FQDN used to check DIGEST-MD5 SASL authentication: DESKTOP-RI8MDR6 18:44:07.905 [info] Application p1_mysql started on node ejabberd@REDACTED 18:44:08.088 [warning] cyclic dependency detected between modules: [customMod,customMod] 18:44:08.088 [warning] module 'mod_mam' is recommended for module 'mod_muc' but is not found in the config slot has started 18:44:08.100 [info] Inside customMod start function {ok,<0.444.0>} 18:44:08.187 [info] Application inets started on node ejabberd@REDACTED 18:44:08.202 [info] Start accepting TCP connections at 0.0.0.0:5222 for ejabberd_c2s 18:44:08.202 [info] ejabberd 17.03 is started in the node ejabberd@REDACTED in 2.38s 18:44:08.202 [info] Start accepting TCP connections at 0.0.0.0:5269 for ejabberd_s2s_in 18:44:08.202 [info] Application ejabberd started on node ejabberd@REDACTED 18:44:08.202 [info] Start accepting TCP connections at 0.0.0.0:5280 for ejabberd_http 18:44:15.864 [info] (<0.498.0>) Accepted connection 127.0.0.1:63043 -> 127.0.0.1:5222 18:44:16.046 [info] (tls|<0.498.0>) Accepted c2s PLAIN authentication for admin1@REDACTED by sql backend from 127.0.0.1 18:44:16.085 [info] (<0.500.0>) Accepted connection 127.0.0.1:63044 -> 127.0.0.1:5222 18:44:16.179 [info] (tls|<0.500.0>) Accepted c2s PLAIN authentication for admin2@REDACTED by sql backend from 127.0.0.1 18:44:16.204 [info] (tls|<0.498.0>) Opened c2s session for admin1@REDACTED/- 18:44:16.266 [info] hello buddy 10 18:44:16.266 [info] hello buddy 10 18:44:16.374 [info] (tls|<0.500.0>) Opened c2s session for admin2@REDACTED/- 18:44:16.436 [info] hello buddy 10 18:44:16.436 [info] hello buddy 10 18:44:16.468 [info] Outbound s2s connection started: casino.com -> proxy.eu.jabber.org 18:44:16.469 [info] hello buddy 10 18:44:16.469 [info] hello buddy 10 18:44:16.469 [info] hello buddy 10 18:44:16.469 [info] hello buddy 10 18:44:16.503 [info] hello buddy 10 18:44:16.503 [info] hello buddy 10 18:44:16.508 [info] Failed to establish outbound s2s connection casino.com -> proxy.eu.jabber.org: DNS lookup failed: non-existing domain; bouncing for 129 seconds 18:44:16.508 [info] hello buddy 10 18:44:16.639 [info] hello buddy 10 18:44:16.639 [info] hello buddy 10 18:44:16.640 [info] hello buddy 10 18:44:16.640 [info] hello buddy 10 18:44:16.641 [info] hello buddy 10 18:44:16.641 [info] hello buddy 10 18:44:16.642 [info] hello buddy 10 18:44:17.214 [info] hello buddy 10 18:44:17.309 [info] The value of mod inside sm is ejabberd_sm_mnesia 18:44:17.309 [info] hello buddy 10 18:44:17.379 [info] hello buddy 10 18:44:17.449 [info] The value of mod inside sm is ejabberd_sm_mnesia 18:44:17.449 [info] hello buddy 10 18:44:47.007 [info] hello buddy 10 18:44:47.007 [info] hello buddy 10 18:45:47.000 [info] hello buddy 10 18:45:47.000 [info] hello buddy 10 18:46:47.001 [info] hello buddy 10 18:46:47.002 [info] hello buddy 10 18:47:47.001 [info] hello buddy 10 18:47:47.001 [info] hello buddy 10 18:48:47.001 [info] hello buddy 10 18:48:47.001 [info] hello buddy 10 18:49:47.001 [info] hello buddy 10 18:49:47.002 [info] hello buddy 10 18:50:47.001 [info] hello buddy 10 18:50:47.002 [info] hello buddy 10 18:51:47.002 [info] hello buddy 10 18:51:47.004 [info] hello buddy 10 18:52:47.001 [info] hello buddy 10 18:52:47.002 [info] hello buddy 10 18:53:47.002 [info] hello buddy 10 18:53:47.003 [info] hello buddy 10 18:54:47.001 [info] hello buddy 10 18:54:47.002 [info] hello buddy 10 18:55:47.000 [info] hello buddy 10 18:55:47.001 [info] hello buddy 10 18:56:47.002 [info] hello buddy 10 18:56:47.002 [info] hello buddy 10 18:57:47.000 [info] hello buddy 10 18:57:47.001 [info] hello buddy 10 18:58:47.001 [info] hello buddy 10 18:58:47.001 [info] hello buddy 10 18:59:47.003 [info] hello buddy 10 18:59:47.004 [info] hello buddy 10 19:00:47.002 [info] hello buddy 10 19:00:47.004 [info] hello buddy 10 19:01:47.001 [info] hello buddy 10 19:01:47.001 [info] hello buddy 10 19:02:47.001 [info] hello buddy 10 19:02:47.001 [info] hello buddy 10 19:03:47.001 [info] hello buddy 10 19:03:47.002 [info] hello buddy 10 19:04:47.002 [info] hello buddy 10 19:04:47.003 [info] hello buddy 10 19:05:47.001 [info] hello buddy 10 19:05:47.002 [info] hello buddy 10 19:06:47.001 [info] hello buddy 10 19:06:47.002 [info] hello buddy 10 19:07:47.002 [info] hello buddy 10 19:07:47.003 [info] hello buddy 10 19:08:47.001 [info] hello buddy 10 19:08:47.002 [info] hello buddy 10 19:09:47.001 [info] hello buddy 10 19:09:47.001 [info] hello buddy 10 19:10:47.001 [info] hello buddy 10 19:10:47.001 [info] hello buddy 10 19:11:47.000 [info] hello buddy 10 19:11:47.001 [info] hello buddy 10 19:12:47.000 [info] hello buddy 10 19:12:47.000 [info] hello buddy 10 19:13:47.000 [info] hello buddy 10 19:13:47.000 [info] hello buddy 10 19:14:47.001 [info] hello buddy 10 19:14:47.001 [info] hello buddy 10 19:15:47.001 [info] hello buddy 10 19:15:47.001 [info] hello buddy 10 can anyone help? -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Tue Apr 4 20:08:05 2017 From: t@REDACTED (Tristan Sloughter) Date: Tue, 04 Apr 2017 11:08:05 -0700 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> Message-ID: <1491329285.1527567.934071784.46A13B49@webmail.messagingengine.com> Ah, ok, I thought the minimal mode meant no smp and there was a special flag for erlc to turn on smp if you wanted it. But either way, I agree improving startup time of smp enabled vm is preferable. -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Tue, Apr 4, 2017, at 12:55 AM, Lukas Larsson wrote: > On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter > wrote: >> __ >> First use case I thought of when I heard this news was 'erlc' itself >> :). It disables smp by default, right? > > erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode > minimal -boot start_clean" > > So, it uses the smp emulator if available. > >> >> This gives a few milliseconds speedup in startup time when booting >> the VM which is mostly a non-issue but nice for cli tools. > > If startup time is important, I think the time would be better spent > optimizing the startup time of the smp-emulator, then to maintain the > non-smp emulators just for it's startup speed. > > Lukas > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Wed Apr 5 06:17:23 2017 From: zzantozz@REDACTED (Ryan Stewart) Date: Wed, 05 Apr 2017 04:17:23 +0000 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: <1491329285.1527567.934071784.46A13B49@webmail.messagingengine.com> References: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> <1491329285.1527567.934071784.46A13B49@webmail.messagingengine.com> Message-ID: If a big selling point of Erlang is how easy it makes concurrency (that's how it seems to me), then spending a lot of effort on maintaining non-SMP support seems silly. If you're not running SMP, then you're not really doing what Erlang is excellent at, right? You could just as well be writing C, Java, Lisp, or name your own language that doesn't have a great, built-in concurrency model. I'm a relative noob, though, so what do I know? On Tue, Apr 4, 2017 at 1:08 PM Tristan Sloughter wrote: Ah, ok, I thought the minimal mode meant no smp and there was a special flag for erlc to turn on smp if you wanted it. But either way, I agree improving startup time of smp enabled vm is preferable. -- Tristan Sloughter "I am not a crackpot" - Abe Simpson t@REDACTED On Tue, Apr 4, 2017, at 12:55 AM, Lukas Larsson wrote: On Mon, Apr 3, 2017 at 6:34 PM, Tristan Sloughter wrote: First use case I thought of when I heard this news was 'erlc' itself :). It disables smp by default, right? erlc uses the following startup flags: "erl +A0 +sbtu -noinput -mode minimal -boot start_clean" So, it uses the smp emulator if available. This gives a few milliseconds speedup in startup time when booting the VM which is mostly a non-issue but nice for cli tools. If startup time is important, I think the time would be better spent optimizing the startup time of the smp-emulator, then to maintain the non-smp emulators just for it's startup speed. Lukas *_______________________________________________* erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Wed Apr 5 08:59:29 2017 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 5 Apr 2017 07:59:29 +0100 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: On 3 April 2017 at 14:33, Lukas Larsson wrote: > If you have a very good reason for wanting to use the non-smp emulator, > please do let us know as soon as possible so that we can consider whether > we want to keep the non-smp version anyway. > Question: the smp emulator will continue to be supported on single core machines, right? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Apr 5 09:17:54 2017 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 05 Apr 2017 09:17:54 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: <1491237291.3589393.932666672.0AF64A38@webmail.messagingengine.com> <1491329285.1527567.934071784.46A13B49@webmail.messagingengine.com> Message-ID: Hi, Erlang can start a process on another node. So Erlang can also be nice for a distributed single Cpu cluster of nodes. In a sens Erlang is always multi Cpu, then. Regards "Envoy? depuis mon mobile " Eric ---- Ryan Stewart a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Apr 5 09:33:52 2017 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 5 Apr 2017 09:33:52 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: On Wed, Apr 5, 2017 at 8:59 AM, Roger Lipscombe wrote: > On 3 April 2017 at 14:33, Lukas Larsson wrote: > >> If you have a very good reason for wanting to use the non-smp emulator, >> please do let us know as soon as possible so that we can consider whether >> we want to keep the non-smp version anyway. >> > > Question: the smp emulator will continue to be supported on single core > machines, right? > Yes, it will continue to be possible to run a 1 scheduler SMP emulator on a single core machine. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Apr 5 09:48:27 2017 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 5 Apr 2017 09:48:27 +0200 Subject: [erlang-questions] getting args with erlang:get_stacktrace or similar? In-Reply-To: <50040593.8328512.1490895264527@mail.yahoo.com> References: <50040593.8328512.1490895264527.ref@mail.yahoo.com> <50040593.8328512.1490895264527@mail.yahoo.com> Message-ID: Hello, On Thu, Mar 30, 2017 at 7:34 PM, Vans S wrote: > When we call erlang:get_stacktrace we can get the function + line where we > crashed. > > Is there a way to get the arguments the function we crashed in was called, > or a way to get the current line we crashed on, what the arguments > are currently? > You get that information if the error is a function clause, however if you crash for any other reason (badmatch, badarith, etc) within a function you cannot get that information. The reason for this is because the compiler may have found that the arguments to the function in which you crashed to be dead and optimized the values away. Lukas PS. Sorry for late reply, for some reason the gmail spam filter keeps putting all @yahoo.com e-mails in my spam folder :( -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Wed Apr 5 10:04:11 2017 From: donpedrothird@REDACTED (John Doe) Date: Wed, 5 Apr 2017 11:04:11 +0300 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: Not always. Only if kernel was built with smp support. Some cheap VPS providers build kernels of their one-cpu VPS without smp, for example 512M RAM GoDaddy VPS a year ago was built without smp. I don't think this would be valid reason to continue support for non-smp emulator, but just fyi. 2017-04-05 10:33 GMT+03:00 Lukas Larsson : > On Wed, Apr 5, 2017 at 8:59 AM, Roger Lipscombe > wrote: > >> On 3 April 2017 at 14:33, Lukas Larsson wrote: >> >>> If you have a very good reason for wanting to use the non-smp emulator, >>> please do let us know as soon as possible so that we can consider whether >>> we want to keep the non-smp version anyway. >>> >> >> Question: the smp emulator will continue to be supported on single core >> machines, right? >> > > Yes, it will continue to be possible to run a 1 scheduler SMP emulator on > a single core machine. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Wed Apr 5 14:18:07 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Wed, 5 Apr 2017 17:48:07 +0530 (IST) Subject: [erlang-questions] Hook user_send_packet is crashing when client tries to connect Message-ID: <912553019.53948.1491394687523@webmail.blacklightsw.com> I have created a custom module in ejabberd that should print hello buddy in logs only when there is a exchange of message stanzas between the client and server: Here is the code-> -module(newCust). -behaviour(gen_mod). -export([]). -export([start/2,stop/1,depends/2,mod_opt_type/1,myMessage/4]). -include("logger.hrl"). -include("xmpp.hrl"). -include("ejabberd.hrl"). start(_Host, _Opts) -> ?INFO_MSG("Inside customMod start function ",[]), ejabberd_hooks:add(user_send_packet, _Host, ?MODULE, myMessage, 105), ok. stop(_Host) -> ejabberd_hooks:delete(user_send_packet, _Host, ?MODULE, myMessage,105). depends(_Host, _Opts)->[{?MODULE,soft}]. mod_opt_type(_Option)-> ok. myMessage(#xmlel{name = <<"message">>} = Pkt, _C2SState, _From, _To)-> ?INFO_MSG("hello buddy ",[]), Pkt. The log values are : 2017-04-05 17:43:02.498 [info] <0.7.0> Application lager started on node ejabberd@REDACTED 2017-04-05 17:43:02.502 [info] <0.7.0> Application crypto started on node ejabberd@REDACTED 2017-04-05 17:43:02.511 [info] <0.7.0> Application sasl started on node ejabberd@REDACTED 2017-04-05 17:43:02.521 [info] <0.7.0> Application asn1 started on node ejabberd@REDACTED 2017-04-05 17:43:02.521 [info] <0.7.0> Application public_key started on node ejabberd@REDACTED 2017-04-05 17:43:02.529 [info] <0.7.0> Application ssl started on node ejabberd@REDACTED 2017-04-05 17:43:02.543 [info] <0.7.0> Application fast_yaml started on node ejabberd@REDACTED 2017-04-05 17:43:02.559 [info] <0.7.0> Application fast_tls started on node ejabberd@REDACTED 2017-04-05 17:43:02.575 [info] <0.7.0> Application fast_xml started on node ejabberd@REDACTED 2017-04-05 17:43:02.579 [info] <0.7.0> Application p1_utils started on node ejabberd@REDACTED 2017-04-05 17:43:02.581 [info] <0.7.0> Application stringprep started on node ejabberd@REDACTED 2017-04-05 17:43:02.582 [info] <0.7.0> Application xmpp started on node ejabberd@REDACTED 2017-04-05 17:43:02.595 [info] <0.7.0> Application cache_tab started on node ejabberd@REDACTED 2017-04-05 17:43:02.618 [info] <0.7.0> Application elixir started on node ejabberd@REDACTED 2017-04-05 17:43:02.786 [info] <0.7.0> Application mnesia started on node ejabberd@REDACTED 2017-04-05 17:43:02.900 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_last_odbc is deprecated, use mod_last with 'db_type: sql' instead 2017-04-05 17:43:02.911 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_offline_odbc is deprecated, use mod_offline with 'db_type: sql' instead 2017-04-05 17:43:02.918 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_privacy_odbc is deprecated, use mod_privacy with 'db_type: sql' instead 2017-04-05 17:43:02.922 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_private_odbc is deprecated, use mod_private with 'db_type: sql' instead 2017-04-05 17:43:02.926 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_pubsub_odbc is deprecated, use mod_pubsub with 'db_type: sql' instead 2017-04-05 17:43:02.930 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_roster_odbc is deprecated, use mod_roster with 'db_type: sql' instead 2017-04-05 17:43:02.944 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_vcard_odbc is deprecated, use mod_vcard with 'db_type: sql' instead 2017-04-05 17:43:03.643 [info] <0.37.0>@ejabberd_app:add_windows_nameservers:132 Adding machine's DNS IPs to Erlang system: [] 2017-04-05 17:43:03.654 [info] <0.279.0>@cyrsasl_digest:start:62 FQDN used to check DIGEST-MD5 SASL authentication: DESKTOP-RI8MDR6 2017-04-05 17:43:04.176 [info] <0.7.0> Application p1_mysql started on node ejabberd@REDACTED 2017-04-05 17:43:04.308 [warning] <0.277.0>@gen_mod:sort_modules:149 module 'mod_mam' is recommended for module 'mod_muc' but is not found in the config 2017-04-05 17:43:04.309 [warning] <0.277.0>@gen_mod:sort_modules:157 cyclic dependency detected between modules: [newCust,newCust] 2017-04-05 17:43:04.314 [info] <0.277.0>@newCust:start:17 Inside customMod start function 2017-04-05 17:43:04.443 [info] <0.7.0> Application inets started on node ejabberd@REDACTED 2017-04-05 17:43:04.458 [info] <0.325.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections at 0.0.0.0:5222 for ejabberd_c2s 2017-04-05 17:43:04.458 [info] <0.326.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections at 0.0.0.0:5269 for ejabberd_s2s_in 2017-04-05 17:43:04.458 [info] <0.37.0>@ejabberd_app:start:59 ejabberd 17.03 is started in the node ejabberd@REDACTED in 2.22s 2017-04-05 17:43:04.458 [info] <0.327.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections at 0.0.0.0:5280 for ejabberd_http 2017-04-05 17:43:04.458 [info] <0.7.0> Application ejabberd started on node ejabberd@REDACTED 2017-04-05 17:43:09.676 [info] <0.325.0>@ejabberd_listener:accept:311 (<0.491.0>) Accepted connection 127.0.0.1:63321 -> 127.0.0.1:5222 2017-04-05 17:43:09.807 [info] <0.492.0>@ejabberd_c2s:handle_auth_success:439 (tls|<0.491.0>) Accepted c2s PLAIN authentication for admin2@REDACTED by sql backend from 127.0.0.1 2017-04-05 17:43:09.981 [info] <0.492.0>@ejabberd_c2s:bind:410 (tls|<0.491.0>) Opened c2s session for admin2@REDACTED/- 2017-04-05 17:43:09.991 [error] <0.492.0>@ejabberd_hooks:safe_apply:383 Hook user_send_packet crashed when running newCust:myMessage/1: ** Reason = {error,undef,[{newCust,myMessage,[],[{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},{jid,<<"admin2">>,<<"casino.com">>,<<>>,<<"admin2">>,<<"casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout => 60000,privacy_list => {userlist,none,[],false},stream_timeout => infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},tls_options => [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required => false,mgmt_timeout => 300,socket_monitor => #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => 1000,stream_version => {1,0},resource => <<"-">>,jid => {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => c2s,stream_state => established,tls_verify => false,sockmod => ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated => true,conn => c2s_tls}}]},{ejabberd_hooks,safe_apply,[{file,"src/ejabberd_hooks.erl"},{line,380}],4},{ejabberd_hooks,run_fold1,[{file,"src/ejabberd_hooks.erl"},{line,364}],4},{ejabberd_c2s,...},...]} ** Arguments = [{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},{jid,<<"admin2">>,<<"casino.com">>,<<>>,<<"admin2">>,<<"casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout => 60000,privacy_list => {userlist,none,[],false},stream_timeout => infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},tls_options => [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required => false,mgmt_timeout => 300,socket_monitor => #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => 1000,stream_version => {1,0},resource => <<"-">>,jid => {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => c2s,stream_state => established,tls_verify => false,sockmod => ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated => true,conn => c2s_tls}}] And the same error and reason is getting printed again and again after some time as the client tries to connect with the server. Can anyone help please? Also i am new to erlang and ejabberd so can anyone tell me how to create custom hooks in ejabberd? Thank you in advance -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.strigler@REDACTED Wed Apr 5 18:11:29 2017 From: stefan.strigler@REDACTED (Stefan Strigler) Date: Wed, 05 Apr 2017 16:11:29 +0000 Subject: [erlang-questions] Hook user_send_packet is crashing when client tries to connect In-Reply-To: <912553019.53948.1491394687523@webmail.blacklightsw.com> References: <912553019.53948.1491394687523@webmail.blacklightsw.com> Message-ID: Hi, this is because you only have one clause, the one for packets of type "message" but none for all the others. Check the example I've sent to you, it had a second clause that acts as a catch all. On Wed, Apr 5, 2017 at 3:08 PM Abhishek Ranjan wrote: > *I have created a custom module in ejabberd that should print hello buddy > in logs only when there is a exchange of message stanzas between the client > and server: Here is the code->* > > -module(newCust). > -behaviour(gen_mod). > > -export([]). > > -export([start/2,stop/1,depends/2,mod_opt_type/1,myMessage/4]). > -include("logger.hrl"). > -include("xmpp.hrl"). > -include("ejabberd.hrl"). > > start(_Host, _Opts) -> > > ?INFO_MSG("Inside customMod start function ",[]), > ejabberd_hooks:add(user_send_packet, _Host, ?MODULE, > myMessage, 105), > ok. > stop(_Host) -> > ejabberd_hooks:delete(user_send_packet, _Host, ?MODULE, myMessage,105). > > depends(_Host, _Opts)->[{?MODULE,soft}]. > > mod_opt_type(_Option)-> > ok. > > myMessage(#xmlel{name = <<"message">>} = Pkt, _C2SState, _From, _To)-> > > ?INFO_MSG("hello buddy ",[]), > Pkt. > > > *The log values are :* > > 2017-04-05 17:43:02.498 [info] <0.7.0> Application lager started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.502 [info] <0.7.0> Application crypto started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.511 [info] <0.7.0> Application sasl started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.521 [info] <0.7.0> Application asn1 started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.521 [info] <0.7.0> Application public_key started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.529 [info] <0.7.0> Application ssl started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.543 [info] <0.7.0> Application fast_yaml started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.559 [info] <0.7.0> Application fast_tls started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.575 [info] <0.7.0> Application fast_xml started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.579 [info] <0.7.0> Application p1_utils started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.581 [info] <0.7.0> Application stringprep started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.582 [info] <0.7.0> Application xmpp started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.595 [info] <0.7.0> Application cache_tab started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.618 [info] <0.7.0> Application elixir started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.786 [info] <0.7.0> Application mnesia started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.900 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_last_odbc > is deprecated, use mod_last with 'db_type: sql' instead > 2017-04-05 17:43:02.911 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_offline_odbc is deprecated, use mod_offline with 'db_type: sql' instead > 2017-04-05 17:43:02.918 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_privacy_odbc is deprecated, use mod_privacy with 'db_type: sql' instead > 2017-04-05 17:43:02.922 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_private_odbc is deprecated, use mod_private with 'db_type: sql' instead > 2017-04-05 17:43:02.926 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_pubsub_odbc is deprecated, use mod_pubsub with 'db_type: sql' instead > 2017-04-05 17:43:02.930 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_roster_odbc is deprecated, use mod_roster with 'db_type: sql' instead > 2017-04-05 17:43:02.944 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_vcard_odbc is deprecated, use mod_vcard with 'db_type: sql' instead > 2017-04-05 17:43:03.643 [info] > <0.37.0>@ejabberd_app:add_windows_nameservers:132 Adding machine's DNS IPs > to Erlang system: > [] > 2017-04-05 17:43:03.654 [info] <0.279.0>@cyrsasl_digest:start:62 FQDN used > to check DIGEST-MD5 SASL authentication: DESKTOP-RI8MDR6 > 2017-04-05 17:43:04.176 [info] <0.7.0> Application p1_mysql started on > node ejabberd@REDACTED > 2017-04-05 17:43:04.308 [warning] <0.277.0>@gen_mod:sort_modules:149 > module 'mod_mam' is recommended for module 'mod_muc' but is not found in > the config > 2017-04-05 17:43:04.309 [warning] <0.277.0>@gen_mod:sort_modules:157 > cyclic dependency detected between modules: [newCust,newCust] > 2017-04-05 17:43:04.314 [info] <0.277.0>@newCust:start:17 Inside customMod > start function > 2017-04-05 17:43:04.443 [info] <0.7.0> Application inets started on node > ejabberd@REDACTED > 2017-04-05 17:43:04.458 [info] <0.325.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5222 for ejabberd_c2s > 2017-04-05 17:43:04.458 [info] <0.326.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5269 for ejabberd_s2s_in > 2017-04-05 17:43:04.458 [info] <0.37.0>@ejabberd_app:start:59 ejabberd > 17.03 is started in the node ejabberd@REDACTED in 2.22s > 2017-04-05 17:43:04.458 [info] <0.327.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5280 for ejabberd_http > 2017-04-05 17:43:04.458 [info] <0.7.0> Application ejabberd started on > node ejabberd@REDACTED > 2017-04-05 17:43:09.676 [info] <0.325.0>@ejabberd_listener:accept:311 > (<0.491.0>) Accepted connection 127.0.0.1:63321 -> 127.0.0.1:5222 > 2017-04-05 17:43:09.807 [info] > <0.492.0>@ejabberd_c2s:handle_auth_success:439 (tls|<0.491.0>) Accepted c2s > PLAIN authentication for admin2@REDACTED by sql backend from 127.0.0.1 > 2017-04-05 17:43:09.981 [info] <0.492.0>@ejabberd_c2s:bind:410 > (tls|<0.491.0>) *Opened c2s session for admin2@REDACTED/- > * > 2017-04-05 17:43:09.991 [error] <0.492.0>@*ejabberd_hooks:safe_apply:383 > Hook user_send_packet crashed when running newCust:myMessage/1:* > ** *Reason* = > {error,undef,[{newCust,myMessage,[],[{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<" > casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com > ">>,<<"-">>},{jid,<<"admin2">>,<<"casino.com">>,<<>>,<<"admin2">>,<<" > casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip > => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => > <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => > {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod > => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => > inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => > ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout > => 60000,privacy_list => {userlist,none,[],false},stream_timeout => > infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<" > casino.com">>,<<>>},nil,nil}},tls_options => > [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required > => false,mgmt_timeout => 300,socket_monitor => > #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => > false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib > => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => > 1000,stream_version => {1,0},resource => <<"-">>,jid => > {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue > => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => > c2s,stream_state => established,tls_verify => false,sockmod => > ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => > {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => > <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated > => true,conn => > c2s_tls}}]},{ejabberd_hooks,safe_apply,[{file,"src/ejabberd_hooks.erl"},{line,380}],4},{ejabberd_hooks,run_fold1,[{file,"src/ejabberd_hooks.erl"},{line,364}],4},{ejabberd_c2s,...},...]} > > > > ** *Arguments* = > [{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"casino.com > ">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},{jid,<<"admin2">>,<<" > casino.com">>,<<>>,<<"admin2">>,<<"casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip > => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => > <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => > {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod > => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => > inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => > ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout > => 60000,privacy_list => {userlist,none,[],false},stream_timeout => > infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<" > casino.com">>,<<>>},nil,nil}},tls_options => > [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required > => false,mgmt_timeout => 300,socket_monitor => > #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => > false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib > => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => > 1000,stream_version => {1,0},resource => <<"-">>,jid => > {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue > => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => > c2s,stream_state => established,tls_verify => false,sockmod => > ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => > {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => > <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated > => true,conn => c2s_tls}}] > > > *And the same error and reason is getting printed again and again after > some time as the client tries to connect with the server.* > > Can anyone help please? > > Also i am new to erlang and ejabberd so can anyone tell me how to create > custom hooks in ejabberd? > > Thank you in advance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.strigler@REDACTED Wed Apr 5 18:15:33 2017 From: stefan.strigler@REDACTED (Stefan Strigler) Date: Wed, 05 Apr 2017 16:15:33 +0000 Subject: [erlang-questions] Hook user_send_packet is crashing when client tries to connect In-Reply-To: References: <912553019.53948.1491394687523@webmail.blacklightsw.com> Message-ID: Yeah, and sorry, apparently the signature changed. I was looking at an outdated version of ejabberd. Now they arity indeed is just 1. Now it's 'user_send_packet({Packet, C2SState})' where Packet can be one of #iq, #presence or #message. On Wed, Apr 5, 2017 at 6:11 PM Stefan Strigler wrote: > Hi, > > this is because you only have one clause, the one for packets of type > "message" but none for all the others. Check the example I've sent to you, > it had a second clause that acts as a catch all. > > On Wed, Apr 5, 2017 at 3:08 PM Abhishek Ranjan > wrote: > > *I have created a custom module in ejabberd that should print hello buddy > in logs only when there is a exchange of message stanzas between the client > and server: Here is the code->* > > -module(newCust). > -behaviour(gen_mod). > > -export([]). > > -export([start/2,stop/1,depends/2,mod_opt_type/1,myMessage/4]). > -include("logger.hrl"). > -include("xmpp.hrl"). > -include("ejabberd.hrl"). > > start(_Host, _Opts) -> > > ?INFO_MSG("Inside customMod start function ",[]), > ejabberd_hooks:add(user_send_packet, _Host, ?MODULE, > myMessage, 105), > ok. > stop(_Host) -> > ejabberd_hooks:delete(user_send_packet, _Host, ?MODULE, myMessage,105). > > depends(_Host, _Opts)->[{?MODULE,soft}]. > > mod_opt_type(_Option)-> > ok. > > myMessage(#xmlel{name = <<"message">>} = Pkt, _C2SState, _From, _To)-> > > ?INFO_MSG("hello buddy ",[]), > Pkt. > > > *The log values are :* > > 2017-04-05 17:43:02.498 [info] <0.7.0> Application lager started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.502 [info] <0.7.0> Application crypto started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.511 [info] <0.7.0> Application sasl started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.521 [info] <0.7.0> Application asn1 started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.521 [info] <0.7.0> Application public_key started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.529 [info] <0.7.0> Application ssl started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.543 [info] <0.7.0> Application fast_yaml started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.559 [info] <0.7.0> Application fast_tls started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.575 [info] <0.7.0> Application fast_xml started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.579 [info] <0.7.0> Application p1_utils started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.581 [info] <0.7.0> Application stringprep started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.582 [info] <0.7.0> Application xmpp started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.595 [info] <0.7.0> Application cache_tab started on > node ejabberd@REDACTED > 2017-04-05 17:43:02.618 [info] <0.7.0> Application elixir started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.786 [info] <0.7.0> Application mnesia started on node > ejabberd@REDACTED > 2017-04-05 17:43:02.900 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_last_odbc > is deprecated, use mod_last with 'db_type: sql' instead > 2017-04-05 17:43:02.911 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_offline_odbc is deprecated, use mod_offline with 'db_type: sql' instead > 2017-04-05 17:43:02.918 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_privacy_odbc is deprecated, use mod_privacy with 'db_type: sql' instead > 2017-04-05 17:43:02.922 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_private_odbc is deprecated, use mod_private with 'db_type: sql' instead > 2017-04-05 17:43:02.926 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_pubsub_odbc is deprecated, use mod_pubsub with 'db_type: sql' instead > 2017-04-05 17:43:02.930 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_roster_odbc is deprecated, use mod_roster with 'db_type: sql' instead > 2017-04-05 17:43:02.944 [warning] > <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module > mod_vcard_odbc is deprecated, use mod_vcard with 'db_type: sql' instead > 2017-04-05 17:43:03.643 [info] > <0.37.0>@ejabberd_app:add_windows_nameservers:132 Adding machine's DNS IPs > to Erlang system: > [] > 2017-04-05 17:43:03.654 [info] <0.279.0>@cyrsasl_digest:start:62 FQDN used > to check DIGEST-MD5 SASL authentication: DESKTOP-RI8MDR6 > 2017-04-05 17:43:04.176 [info] <0.7.0> Application p1_mysql started on > node ejabberd@REDACTED > 2017-04-05 17:43:04.308 [warning] <0.277.0>@gen_mod:sort_modules:149 > module 'mod_mam' is recommended for module 'mod_muc' but is not found in > the config > 2017-04-05 17:43:04.309 [warning] <0.277.0>@gen_mod:sort_modules:157 > cyclic dependency detected between modules: [newCust,newCust] > 2017-04-05 17:43:04.314 [info] <0.277.0>@newCust:start:17 Inside customMod > start function > 2017-04-05 17:43:04.443 [info] <0.7.0> Application inets started on node > ejabberd@REDACTED > 2017-04-05 17:43:04.458 [info] <0.325.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5222 for ejabberd_c2s > 2017-04-05 17:43:04.458 [info] <0.326.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5269 for ejabberd_s2s_in > 2017-04-05 17:43:04.458 [info] <0.37.0>@ejabberd_app:start:59 ejabberd > 17.03 is started in the node ejabberd@REDACTED in 2.22s > 2017-04-05 17:43:04.458 [info] <0.327.0>@ejabberd_listener:init_tcp:149 > Start accepting TCP connections at 0.0.0.0:5280 for ejabberd_http > 2017-04-05 17:43:04.458 [info] <0.7.0> Application ejabberd started on > node ejabberd@REDACTED > 2017-04-05 17:43:09.676 [info] <0.325.0>@ejabberd_listener:accept:311 > (<0.491.0>) Accepted connection 127.0.0.1:63321 -> 127.0.0.1:5222 > 2017-04-05 17:43:09.807 [info] > <0.492.0>@ejabberd_c2s:handle_auth_success:439 (tls|<0.491.0>) Accepted c2s > PLAIN authentication for admin2@REDACTED by sql backend from 127.0.0.1 > 2017-04-05 17:43:09.981 [info] <0.492.0>@ejabberd_c2s:bind:410 > (tls|<0.491.0>) *Opened c2s session for admin2@REDACTED/- > * > 2017-04-05 17:43:09.991 [error] <0.492.0>@*ejabberd_hooks:safe_apply:383 > Hook user_send_packet crashed when running newCust:myMessage/1:* > ** *Reason* = > {error,undef,[{newCust,myMessage,[],[{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<" > casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com > ">>,<<"-">>},{jid,<<"admin2">>,<<"casino.com">>,<<>>,<<"admin2">>,<<" > casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip > => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => > <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => > {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod > => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => > inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => > ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout > => 60000,privacy_list => {userlist,none,[],false},stream_timeout => > infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<" > casino.com">>,<<>>},nil,nil}},tls_options => > [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required > => false,mgmt_timeout => 300,socket_monitor => > #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => > false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib > => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => > 1000,stream_version => {1,0},resource => <<"-">>,jid => > {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue > => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => > c2s,stream_state => established,tls_verify => false,sockmod => > ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => > {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => > <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated > => true,conn => > c2s_tls}}]},{ejabberd_hooks,safe_apply,[{file,"src/ejabberd_hooks.erl"},{line,380}],4},{ejabberd_hooks,run_fold1,[{file,"src/ejabberd_hooks.erl"},{line,364}],4},{ejabberd_c2s,...},...]} > > > > ** *Arguments* = > [{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"casino.com > ">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},{jid,<<"admin2">>,<<" > casino.com">>,<<>>,<<"admin2">>,<<"casino.com">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip > => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => > <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => > {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod > => ejabberd_c2s,server => <<"casino.com">>,mgmt_state => > inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => > ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout > => 60000,privacy_list => {userlist,none,[],false},stream_timeout => > infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<" > casino.com">>,<<>>},nil,nil}},tls_options => > [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required > => false,mgmt_timeout => 300,socket_monitor => > #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => > false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"casino.com">>,zlib > => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => > 1000,stream_version => {1,0},resource => <<"-">>,jid => > {jid,<<"admin2">>,<<"casino.com">>,<<"-">>,<<"admin2">>,<<"casino.com">>,<<"-">>},csi_queue > => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => > c2s,stream_state => established,tls_verify => false,sockmod => > ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => > {1,{{<<"admin2">>,<<"casino.com">>,<<>>},nil,nil}},owner => > <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated > => true,conn => c2s_tls}}] > > > *And the same error and reason is getting printed again and again after > some time as the client tries to connect with the server.* > > Can anyone help please? > > Also i am new to erlang and ejabberd so can anyone tell me how to create > custom hooks in ejabberd? > > Thank you in advance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Thu Apr 6 08:06:53 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Thu, 6 Apr 2017 11:36:53 +0530 (IST) Subject: [erlang-questions] Hook user_send_packet is crashing when client tries to connect In-Reply-To: References: <912553019.53948.1491394687523@webmail.blacklightsw.com> Message-ID: <1552930023.23355.1491458813464@webmail.blacklightsw.com> Hi stefan I did tried extractMessge({#xmlel{name = <<"message">>} = Pkt,C2SState}) but it still said that there was a function clause error in this line and the hooks crashes again. Is't there any guide where it is mentioned that what are the defined syntax for hooks. Also in my ejabberd_c2s the run statement which calls this 'user_send_packet' looks like this {Pkt2, State2} = ejabberd_hooks:run_fold( user_send_packet, LServer, {Pkt1, State1}, []), case Pkt2 of drop -> State2; #iq{type = set, sub_els = [_]} -> case xmpp:get_subtag(Pkt2, #xmpp_session{}) of #xmpp_session{} -> send(State2, xmpp:make_iq_result(Pkt2)); _ -> check_privacy_then_route(State2, Pkt2) end; #presence{to = #jid{luser = LUser, lserver = LServer, lresource = <<"">>}} -> process_self_presence(State2, Pkt2); #presence{} -> process_presence_out(State2, Pkt2); _ -> check_privacy_then_route(State2, Pkt2) end. Here I can see that what happens if the packet is dropped,or if it is a 'iq' packet or a 'presence' one but nothing is mentioned about the message stanzas? I am using ejabberd 17.03. Cheers!! Abhishek Checkout my awesome apps on Google Play Store https://play.google.com/store/apps/dev?id=7448460448507458941 and iOS http://bit.ly/bsw_iOS > On April 5, 2017 at 9:45 PM Stefan Strigler wrote: > > Yeah, and sorry, apparently the signature changed. I was looking at an outdated version of ejabberd. Now they arity indeed is just 1. > > Now it's 'user_send_packet({Packet, C2SState})' where Packet can be one of #iq, #presence or #message. > > On Wed, Apr 5, 2017 at 6:11 PM Stefan Strigler wrote: > > > > Hi, > > > > this is because you only have one clause, the one for packets of type "message" but none for all the others. Check the example I've sent to you, it had a second clause that acts as a catch all. > > > > On Wed, Apr 5, 2017 at 3:08 PM Abhishek Ranjan wrote: > > > > > > > > > > I have created a custom module in ejabberd that should print hello buddy in logs only when there is a exchange of message stanzas between the client and server: Here is the code-> > > > > > > -module(newCust). > > > -behaviour(gen_mod). > > > > > > -export([]). > > > > > > -export([start/2,stop/1,depends/2,mod_opt_type/1,myMessage/4]). > > > -include("logger.hrl"). > > > -include("xmpp.hrl"). > > > -include("ejabberd.hrl"). > > > > > > start(_Host, _Opts) -> > > > > > > ?INFO_MSG("Inside customMod start function ",[]), > > > ejabberd_hooks:add(user_send_packet, _Host, ?MODULE, > > > myMessage, 105), > > > ok. > > > stop(_Host) -> > > > ejabberd_hooks:delete(user_send_packet, _Host, ?MODULE, myMessage,105). > > > > > > depends(_Host, _Opts)->[{?MODULE,soft}]. > > > > > > mod_opt_type(_Option)-> > > > ok. > > > > > > myMessage(#xmlel{name = <<"message">>} = Pkt, _C2SState, _From, _To)-> > > > > > > ?INFO_MSG("hello buddy ",[]), > > > Pkt. > > > > > > > > > The log values are : > > > > > > 2017-04-05 17:43:02.498 [info] <0.7.0> Application lager started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.502 [info] <0.7.0> Application crypto started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.511 [info] <0.7.0> Application sasl started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.521 [info] <0.7.0> Application asn1 started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.521 [info] <0.7.0> Application public_key started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.529 [info] <0.7.0> Application ssl started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.543 [info] <0.7.0> Application fast_yaml started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.559 [info] <0.7.0> Application fast_tls started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.575 [info] <0.7.0> Application fast_xml started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.579 [info] <0.7.0> Application p1_utils started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.581 [info] <0.7.0> Application stringprep started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.582 [info] <0.7.0> Application xmpp started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.595 [info] <0.7.0> Application cache_tab started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.618 [info] <0.7.0> Application elixir started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.786 [info] <0.7.0> Application mnesia started on node ejabberd@REDACTED > > > 2017-04-05 17:43:02.900 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_last_odbc is deprecated, use mod_last with 'db_type: sql' instead > > > 2017-04-05 17:43:02.911 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_offline_odbc is deprecated, use mod_offline with 'db_type: sql' instead > > > 2017-04-05 17:43:02.918 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_privacy_odbc is deprecated, use mod_privacy with 'db_type: sql' instead > > > 2017-04-05 17:43:02.922 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_private_odbc is deprecated, use mod_private with 'db_type: sql' instead > > > 2017-04-05 17:43:02.926 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_pubsub_odbc is deprecated, use mod_pubsub with 'db_type: sql' instead > > > 2017-04-05 17:43:02.930 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_roster_odbc is deprecated, use mod_roster with 'db_type: sql' instead > > > 2017-04-05 17:43:02.944 [warning] <0.37.0>@ejabberd_config:emit_deprecation_warning:1427 Module mod_vcard_odbc is deprecated, use mod_vcard with 'db_type: sql' instead > > > 2017-04-05 17:43:03.643 [info] <0.37.0>@ejabberd_app:add_windows_nameservers:132 Adding machine's DNS IPs to Erlang system: > > > [] > > > 2017-04-05 17:43:03.654 [info] <0.279.0>@cyrsasl_digest:start:62 FQDN used to check DIGEST-MD5 SASL authentication: DESKTOP-RI8MDR6 > > > 2017-04-05 17:43:04.176 [info] <0.7.0> Application p1_mysql started on node ejabberd@REDACTED > > > 2017-04-05 17:43:04.308 [warning] <0.277.0>@gen_mod:sort_modules:149 module 'mod_mam' is recommended for module 'mod_muc' but is not found in the config > > > 2017-04-05 17:43:04.309 [warning] <0.277.0>@gen_mod:sort_modules:157 cyclic dependency detected between modules: [newCust,newCust] > > > 2017-04-05 17:43:04.314 [info] <0.277.0>@newCust:start:17 Inside customMod start function > > > 2017-04-05 17:43:04.443 [info] <0.7.0> Application inets started on node ejabberd@REDACTED > > > 2017-04-05 17:43:04.458 [info] <0.325.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections athttp://0.0.0.0:5222 for ejabberd_c2s > > > 2017-04-05 17:43:04.458 [info] <0.326.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections athttp://0.0.0.0:5269 for ejabberd_s2s_in > > > 2017-04-05 17:43:04.458 [info] <0.37.0>@ejabberd_app:start:59 ejabberd 17.03 is started in the node ejabberd@REDACTED in 2.22s > > > 2017-04-05 17:43:04.458 [info] <0.327.0>@ejabberd_listener:init_tcp:149 Start accepting TCP connections athttp://0.0.0.0:5280 for ejabberd_http > > > 2017-04-05 17:43:04.458 [info] <0.7.0> Application ejabberd started on node ejabberd@REDACTED > > > 2017-04-05 17:43:09.676 [info] <0.325.0>@ejabberd_listener:accept:311 (<0.491.0>) Accepted connectionhttp://127.0.0.1:63321 ->http://127.0.0.1:5222 > > > 2017-04-05 17:43:09.807 [info] <0.492.0>@ejabberd_c2s:handle_auth_success:439 (tls|<0.491.0>) Accepted c2s PLAIN authentication for admin2@REDACTED mailto:admin2@REDACTED by sql backend from 127.0.0.1 > > > 2017-04-05 17:43:09.981 [info] <0.492.0>@ejabberd_c2s:bind:410 (tls|<0.491.0>) Opened c2s session forhttp://admin2@REDACTED/- > > > 2017-04-05 17:43:09.991 [error] <0.492.0>@ejabberd_hooks:safe_apply:383 Hook user_send_packet crashed when running newCust:myMessage/1: > > > ** Reason = {error,undef,[{newCust,myMessage,[],[{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"http://casino.com ">>,<<"-">>,<<"admin2">>,<<"http://casino.com ">>,<<"-">>},{jid,<<"admin2">>,<<"http://casino.com ">>,<<>>,<<"admin2">>,<<"http://casino.com ">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod => ejabberd_c2s,server => <<"http://casino.com ">>,mgmt_state => inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout => 60000,privacy_list => {userlist,none,[],false},stream_timeout => infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<"http://casino.com ">>,<<>>},nil,nil}},tls_options => [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required => false,mgmt_timeout => 300,socket_monitor => #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"http://casino.com ">>,zlib => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => 1000,stream_version => {1,0},resource => <<"-">>,jid => {jid,<<"admin2">>,<<"http://casino.com ">>,<<"-">>,<<"admin2">>,<<"http://casino.com ">>,<<"-">>},csi_queue => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => c2s,stream_state => established,tls_verify => false,sockmod => ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => {1,{{<<"admin2">>,<<"http://casino.com ">>,<<>>},nil,nil}},owner => <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated => true,conn => c2s_tls}}]},{ejabberd_hooks,safe_apply,[{file,"src/ejabberd_hooks.erl"},{line,380}],4},{ejabberd_hooks,run_fold1,[{file,"src/ejabberd_hooks.erl"},{line,364}],4},{ejabberd_c2s,...},...]} > > > > > > > > > > > > ** Arguments = [{{iq,<<"purple701889db">>,set,<<"en">>,{jid,<<"admin2">>,<<"http://casino.com ">>,<<"-">>,<<"admin2">>,<<"http://casino.com ">>,<<"-">>},{jid,<<"admin2">>,<<"http://casino.com ">>,<<>>,<<"admin2">>,<<"http://casino.com ">>,<<>>},[{xmlel,<<"session">>,[{<<"xmlns">>,<<"urn:ietf:params:xml:ns:xmpp-session">>}],[]}],#{ip => {127,0,0,1}}},#{mgmt_stanzas_in => 0,xmlns => <<"jabber:client">>,stream_encrypted => true,tls_enabled => true,socket => {socket_state,fast_tls,{tlssock,#Port<0.23143>,#Port<0.23190>},<0.491.0>},mod => ejabberd_c2s,server => <<"http://casino.com ">>,mgmt_state => inactive,mgmt_max_timeout => 300,stream_restarted => true,auth_module => ejabberd_auth_sql,stream_id => <<"15132958958157338735">>,mgmt_ack_timeout => 60000,privacy_list => {userlist,none,[],false},stream_timeout => infinity,sid => {{1491,394389,1},<0.492.0>},pres_f => {1,{{<<"admin2">>,<<"http://casino.com ">>,<<>>},nil,nil}},tls_options => [compression_none,{protocol_options,<<"no_sslv3">>},{certfile,<<"C:\\ProgramData\\ejabberd\\conf\\server.pem">>}],tls_required => false,mgmt_timeout => 300,socket_monitor => #Ref<0.0.2.141>,stream_compressed => false,mgmt_resend => false,mgmt_stanzas_out => 0,csi_state => active,lserver => <<"http://casino.com ">>,zlib => false,lang => <<"en">>,ip => {{127,0,0,1},63321},mgmt_max_queue => 1000,stream_version => {1,0},resource => <<"-">>,jid => {jid,<<"admin2">>,<<"http://casino.com ">>,<<"-">>,<<"admin2">>,<<"http://casino.com ">>,<<"-">>},csi_queue => {0,0,#{}},shaper => c2s_shaper,user => <<"admin2">>,access => c2s,stream_state => established,tls_verify => false,sockmod => ejabberd_socket,stream_direction => in,mgmt_stanzas_req => 0,pres_t => {1,{{<<"admin2">>,<<"http://casino.com ">>,<<>>},nil,nil}},owner => <0.492.0>,stream_header_sent => true,pres_a => {0,nil},stream_authenticated => true,conn => c2s_tls}}] > > > > > > > > > And the same error and reason is getting printed again and again after some time as the client tries to connect with the server. > > > > > > Can anyone help please? > > > > > > Also i am new to erlang and ejabberd so can anyone tell me how to create custom hooks in ejabberd? > > > > > > Thank you in advance > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED mailto:erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Thu Apr 6 13:52:52 2017 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Thu, 06 Apr 2017 11:52:52 +0000 Subject: [erlang-questions] How can I release beam process memory? In-Reply-To: References: Message-ID: Hi Jack, *tl;dr;* try starting Erlang with '+MEas bf' Long explanation: If the memory is used by the ets_alloc allocator, it can be either actively used by ETS tables (such as Mnesia disc_copies tables) or it can be lost due to memory fragmentation. I guess this is memory fragmentation in your case, but let's quickly rule out the other possibility first. (By the way, in either case, garbage collecting processes won't reclaim the memory. GC operates on eheap_alloc allocated memory.) So if erlang:memory(ets) is close to what recon reports as allocated, the memory is in use. Your system wrote a lot of data to ETS/Mnesia, and didn't remove it afterwards. Inspect the contents of the tables and figure out what was left there that shouldn't be there, which part of the application should have removed that data and why didn't it do its job properly. The other, more likely option is fragmentation. I also experienced that the default memory allocation strategy (aoffcbf = address order first fit carrier best fit) can perform poorly when you use a lot of memory. All address order first fit strategies will use heavily the multiblock carriers with the lowest memory addresses, and if you have many carriers, those placed higher in memory will have less and less chance to be used. In my particular case ets_alloc created a total of 150k multiblock carriers for storing ~1.2TB data in ETS tables. This resulted in ~100 GB unused memory being wasted in the high address carriers. You have a much smaller system, but depending on the usage patterns of your ETS data you can end up in a similar situation. You can check the number of carriers with erlang:system_info({allocator, ets_alloc}). It will print something like this: [{instance,0, [{versions,"0.9","3.0"}, {options,[...]}, {mbcs,[... {carriers,1,1,1}, %% <- number of multi block carriers = 1 ...]}, {sbcs,[... {carriers,0,0,0}, %% <- number of single block carriers = 0 ...]}, {calls,[...]}]}, {instance,1,... Check theaw numbera across all your allocator instances. You will typically have very few single block carriers (unless you store huge records in ETS). In my experience, fragmentation correlates well with the number of carriers an allocator handles, and can become quite significant above ~10 carriers. So, If you have the same problem I described, I have some bad news and some good news. The bad news is that I don't know a way of forcing the VM to defragment memory and get rid of the waste. The good news is that the bf (best fit) allocation strategy (which used to be the default up to R16) performs much better when you have many carriers. You need to pass the '+MEas bf' command line argument to the VM to switch ets_alloc to bf strategy. Hope it helps, Daniel On Sat, 1 Apr 2017 at 05:36 Jack Tang wrote: > After setting up erlang memory visualization, we find etc allocator does > not release the memory during some historical datum are remove from mnesia > tables. Can I release the memory on the fly rather than restart the mnesia > application? Thanks! > > > > > BR > > On Sun, Jan 15, 2017 at 4:47 AM, D?niel Szoboszlay > wrote: > > Hi Jack, > > I guess the 9 GB is lost due to memory fragmentation. Erlang allocates > memory in large chunks called carriers from the OS, then places the blocks > your program actually needs on these carriers. A carrier can only be > returned to the OS once all the blocks on it have been freed (and even > then, the memory allocator may decide to keep it around for a while in case > more memory is needed). > > You can check with recon_alloc > how much unused memory is > lost due to fragmentation in the various allocators. > > The bad news is that you cannot defragment the carriers, and if the > selected memory allocator strategy doesn't work well for your application, > you cannot change it either without restarting the emulator. > > However, if the memory is wasted in the eheap_alloc, you may try to force > a GC on all processes a couple of times. As the GC copies the memory, it > will allocate new blocks and free up the old heap blocks. So there's a > chance the allocators can compact the blocks together on fewer segments. > But that's just a guess, it may or may not work at all. > > Cheers, > Daniel > > On Sat, 14 Jan 2017 at 08:04 Jack Tang wrote: > > Hello list, > > I run one Erlang application on Debian server and today I find the beam > process consumes around 35G memory by `top` command. > > ``` > KiB Mem: 99194912 total, 61682656 used, 37512252 free, 397380 buffers > KiB Swap: 0 total, 0 used, 0 free. 18684864 cached Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 11858 usr1 20 0 36.850g 0.032t 6220 S 73.5 34.4 8038:49 beam.smp > ``` > > I connect to the Erlang application using remote shell and find the > mem-leaked supervisor tree and run gc on the whole tree. Code looks like > blow: > > ``` > lists:foreach(fun(E) -> PId = element(2, E), erlang:garbage_collect(PId) > end, supervisor:which_children(some_thing_sup)). > ``` > > and erlang:memory() decreases from 32G to 23G. > ``` > [{total,22982011544}, > {processes,12884182336}, > {processes_used,12884170336}, > {system,10097829208}, > {atom,13828705}, > {atom_used,13796692}, > {binary,170530288}, > {code,16450626}, > {ets,9637717576}] > ``` > > However, when I input `top` command, the beam process still takes 35G > memory. What can I do to release the 9G memory? Thanks > > BR > -Jack > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Jack Tang > > > http://www.linkedin.com/in/jacktang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Thu Apr 6 14:17:56 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Thu, 6 Apr 2017 15:17:56 +0300 Subject: [erlang-questions] enif_inspect_binary behaviour Message-ID: Hello, I'm using enif_inspect_binary to get the content of a binary. My question is if ErlNifBinary will be a copy of the erlang binary or the data member will point to the same memory location? >From my tests seems points to the same buffer as time If I change something when I return to erlang I can observe the change Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From ted.ml.erlang@REDACTED Thu Apr 6 14:46:42 2017 From: ted.ml.erlang@REDACTED (Ted Burghart) Date: Thu, 6 Apr 2017 08:46:42 -0400 Subject: [erlang-questions] enif_inspect_binary behaviour In-Reply-To: References: Message-ID: <6CA1F271-B803-49F5-A6B4-FE7593CD4CF9@tedb.net> enid_inspect_binary populates the ErlNifBinary with read-only data that you should not modify. See the section Functionality => Binaries at the top of the erl_nif manual page for how to make the data mutable. ? Ted > On 6-Apr 2017, at 8:17 AM, Caragea Silviu wrote: > > Hello, > > I'm using enif_inspect_binary to get the content of a binary. > > My question is if ErlNifBinary will be a copy of the erlang binary or the data member will point to the same memory location? > > From my tests seems points to the same buffer as time If I change something when I return to erlang I can observe the change > > Silviu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bjorn@REDACTED Thu Apr 6 15:26:36 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 6 Apr 2017 15:26:36 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module Message-ID: I have written a QuickCheck module to test Dan's new string module (https://github.com/erlang/otp/pull/1330). It found some bugs. Here is the gist for anyone interested: https://gist.github.com/bjorng/03a869392d5a969ebf2c40044b664190 Comments are welcome. This is my first major use of QuickCheck. I am interested in how I could improve the QC specifications and generators. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From alex0player@REDACTED Thu Apr 6 15:30:54 2017 From: alex0player@REDACTED (Alex S.) Date: Thu, 6 Apr 2017 16:30:54 +0300 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: Message-ID: <4CB5E5FD-8064-4A77-80D6-FE7CCFDF8C5B@gmail.com> Hey Bj?rn, for those of us who don?t have QC handy, could you provide the sample run output, please? > 6 ???. 2017 ?., ? 16:26, Bj?rn Gustavsson ???????(?): > > I have written a QuickCheck module to test Dan's new string module > (https://github.com/erlang/otp/pull/1330). It found some bugs. > > Here is the gist for anyone interested: > > https://gist.github.com/bjorng/03a869392d5a969ebf2c40044b664190 > > Comments are welcome. This is my first major use of QuickCheck. I am > interested in how I could improve the QC specifications and > generators. > > /Bj?rn > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bjorn@REDACTED Thu Apr 6 15:43:41 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 6 Apr 2017 15:43:41 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: <4CB5E5FD-8064-4A77-80D6-FE7CCFDF8C5B@gmail.com> References: <4CB5E5FD-8064-4A77-80D6-FE7CCFDF8C5B@gmail.com> Message-ID: Running on the latest string.erl is boring since all tests pass. But I found this this output in one of my xterm windows: 24> c(string_eqc), eqc:quickcheck(eqc:numtests(10000, string_eqc:prop_chomp())). ....................................................................................................(x10)..............................................................................Failed! After 881 tests. {{[1060,10,13,13,10,10,13,13],[1060,10,13,13,10,10,13,13]}, {list,{split,2,{{split,1,{list,binary}},list}}}} flat: passed binary: passed mixed: failed string:chomp([[[[1060]|<<"\n\r\r">>],10,10,13,13]]) Shrinking .xxxxxxxxx..xxxxxxx.(4 times) {{"\r\r\n\n\r","\r\r\n\n\r"},{list,{split,2,{{split,1,{list,binary}},list}}}} mixed: failed string:chomp([[[[]|<<"\r\r">>],10,10,13]]) false 25> string:chomp([[[[]|<<"\r\r">>],10,10,13]]). [13,<<"\r">>,10,13] (Dan has now fixed this bug.) /Bjorn On Thu, Apr 6, 2017 at 3:30 PM, Alex S. wrote: > Hey Bj?rn, > > for those of us who don?t have QC handy, could you provide the sample run output, please? >> 6 ???. 2017 ?., ? 16:26, Bj?rn Gustavsson ???????(?): >> >> I have written a QuickCheck module to test Dan's new string module >> (https://github.com/erlang/otp/pull/1330). It found some bugs. >> >> Here is the gist for anyone interested: >> >> https://gist.github.com/bjorng/03a869392d5a969ebf2c40044b664190 >> >> Comments are welcome. This is my first major use of QuickCheck. I am >> interested in how I could improve the QC specifications and >> generators. >> >> /Bj?rn >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From kostis@REDACTED Thu Apr 6 15:46:31 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 6 Apr 2017 15:46:31 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: <4CB5E5FD-8064-4A77-80D6-FE7CCFDF8C5B@gmail.com> Message-ID: On 04/06/2017 03:43 PM, Bj?rn Gustavsson wrote: > Running on the latest string.erl is boring since all tests pass. Is the *original* (or at least the one you started from) string.erl available anywhere? Can it be posted here? Kostis From bjorn@REDACTED Thu Apr 6 16:40:41 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 6 Apr 2017 16:40:41 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: <4CB5E5FD-8064-4A77-80D6-FE7CCFDF8C5B@gmail.com> Message-ID: On Thu, Apr 6, 2017 at 3:46 PM, Kostis Sagonas wrote: > On 04/06/2017 03:43 PM, Bj?rn Gustavsson wrote: >> >> Running on the latest string.erl is boring since all tests pass. > > > Is the *original* (or at least the one you started from) string.erl > available anywhere? Can it be posted here? You will also need a few other modules (unicode, unicode_util). >From the Git's reflog, I have recovered the branch as it looked when I started working on string_eqc. I have created a new branch and pushed it to github: https://github.com/bjorng/otp/tree/buggy-new-string-module /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From himars@REDACTED Thu Apr 6 16:41:42 2017 From: himars@REDACTED (Jack Tang) Date: Thu, 6 Apr 2017 22:41:42 +0800 Subject: [erlang-questions] How can I release beam process memory? In-Reply-To: References: Message-ID: Hi Daniel Thank you for the detailed explanation and sharing the experiences. I focused on the performance of GC heavily before and ignored the allocation strategy. One more question, as you mentioned memory on eheap_alloc can be released by calling GC on every process. Right now I setup periodic gc process to free eheap. However can I set some process flag to make GC automatically? Thanks again! BR -Jack On Thu, Apr 6, 2017 at 7:52 PM, D?niel Szoboszlay wrote: > Hi Jack, > > *tl;dr;* try starting Erlang with '+MEas bf' > > Long explanation: > > If the memory is used by the ets_alloc allocator, it can be either > actively used by ETS tables (such as Mnesia disc_copies tables) or it can > be lost due to memory fragmentation. I guess this is memory fragmentation > in your case, but let's quickly rule out the other possibility first. (By > the way, in either case, garbage collecting processes won't reclaim the > memory. GC operates on eheap_alloc allocated memory.) > > So if erlang:memory(ets) is close to what recon reports as allocated, the > memory is in use. Your system wrote a lot of data to ETS/Mnesia, and didn't > remove it afterwards. Inspect the contents of the tables and figure out > what was left there that shouldn't be there, which part of the application > should have removed that data and why didn't it do its job properly. > > The other, more likely option is fragmentation. I also experienced that > the default memory allocation strategy (aoffcbf = address order first fit > carrier best fit) can perform poorly when you use a lot of memory. All > address order first fit strategies will use heavily the multiblock carriers > with the lowest memory addresses, and if you have many carriers, those > placed higher in memory will have less and less chance to be used. In my > particular case ets_alloc created a total of 150k multiblock carriers for > storing ~1.2TB data in ETS tables. This resulted in ~100 GB unused memory > being wasted in the high address carriers. You have a much smaller system, > but depending on the usage patterns of your ETS data you can end up in a > similar situation. > > You can check the number of carriers with erlang:system_info({allocator, > ets_alloc}). It will print something like this: > [{instance,0, > [{versions,"0.9","3.0"}, > {options,[...]}, > {mbcs,[... > {carriers,1,1,1}, %% <- number of multi block > carriers = 1 > ...]}, > {sbcs,[... > {carriers,0,0,0}, %% <- number of single block > carriers = 0 > ...]}, > {calls,[...]}]}, > {instance,1,... > Check theaw numbera across all your allocator instances. You will > typically have very few single block carriers (unless you store huge > records in ETS). In my experience, fragmentation correlates well with the > number of carriers an allocator handles, and can become quite significant > above ~10 carriers. > > So, If you have the same problem I described, I have some bad news and > some good news. The bad news is that I don't know a way of forcing the VM > to defragment memory and get rid of the waste. The good news is that the bf > (best fit) allocation strategy (which used to be the default up to R16) > performs much better when you have many carriers. You need to pass the '+MEas > bf' command line argument to the VM to switch ets_alloc to bf strategy. > > Hope it helps, > Daniel > > On Sat, 1 Apr 2017 at 05:36 Jack Tang wrote: > >> After setting up erlang memory visualization, we find etc allocator does >> not release the memory during some historical datum are remove from mnesia >> tables. Can I release the memory on the fly rather than restart the mnesia >> application? Thanks! >> >> >> >> >> BR >> >> On Sun, Jan 15, 2017 at 4:47 AM, D?niel Szoboszlay > > wrote: >> >> Hi Jack, >> >> I guess the 9 GB is lost due to memory fragmentation. Erlang allocates >> memory in large chunks called carriers from the OS, then places the blocks >> your program actually needs on these carriers. A carrier can only be >> returned to the OS once all the blocks on it have been freed (and even >> then, the memory allocator may decide to keep it around for a while in case >> more memory is needed). >> >> You can check with recon_alloc >> how much unused memory >> is lost due to fragmentation in the various allocators. >> >> The bad news is that you cannot defragment the carriers, and if the >> selected memory allocator strategy doesn't work well for your application, >> you cannot change it either without restarting the emulator. >> >> However, if the memory is wasted in the eheap_alloc, you may try to >> force a GC on all processes a couple of times. As the GC copies the memory, >> it will allocate new blocks and free up the old heap blocks. So there's a >> chance the allocators can compact the blocks together on fewer segments. >> But that's just a guess, it may or may not work at all. >> >> Cheers, >> Daniel >> >> On Sat, 14 Jan 2017 at 08:04 Jack Tang wrote: >> >> Hello list, >> >> I run one Erlang application on Debian server and today I find the beam >> process consumes around 35G memory by `top` command. >> >> ``` >> KiB Mem: 99194912 total, 61682656 used, 37512252 free, 397380 buffers >> KiB Swap: 0 total, 0 used, 0 free. 18684864 cached >> Mem >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ >> COMMAND >> 11858 usr1 20 0 36.850g 0.032t 6220 S 73.5 34.4 8038:49 beam.smp >> ``` >> >> I connect to the Erlang application using remote shell and find the >> mem-leaked supervisor tree and run gc on the whole tree. Code looks like >> blow: >> >> ``` >> lists:foreach(fun(E) -> PId = element(2, E), erlang:garbage_collect(PId) >> end, supervisor:which_children(some_thing_sup)). >> ``` >> >> and erlang:memory() decreases from 32G to 23G. >> ``` >> [{total,22982011544}, >> {processes,12884182336}, >> {processes_used,12884170336}, >> {system,10097829208}, >> {atom,13828705}, >> {atom_used,13796692}, >> {binary,170530288}, >> {code,16450626}, >> {ets,9637717576}] >> ``` >> >> However, when I input `top` command, the beam process still takes 35G >> memory. What can I do to release the 9G memory? Thanks >> >> BR >> -Jack >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Jack Tang >> >> >> http://www.linkedin.com/in/jacktang >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -- Jack Tang http://www.linkedin.com/in/jacktang -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Thu Apr 6 17:16:37 2017 From: vans_163@REDACTED (Vans S) Date: Thu, 6 Apr 2017 15:16:37 +0000 (UTC) Subject: [erlang-questions] How can I release beam process memory? In-Reply-To: References: Message-ID: <655241064.1785545.1491491797943@mail.yahoo.com> Just use the instrument module to track the actual terms held then reference the PID that created the term, set some hooks in beams allocator to track the pid references to the term (since instrument does not do this). ?Read beams memory to check what the actual term is. All these statistics and reporting we currently have for erlang memory issues is simply poking at random things in the dark. On Thursday, April 6, 2017 10:41 AM, Jack Tang wrote: Hi Daniel Thank you for the detailed explanation and sharing the experiences. I focused on the performance of GC heavily before and ignored the allocation strategy.? One more question, as you mentioned memory on eheap_alloc can be released by calling GC on every process. Right now I setup periodic gc process ?to free eheap. However can I set some process flag to make GC automatically? Thanks again! BR-Jack On Thu, Apr 6, 2017 at 7:52 PM, D?niel Szoboszlay wrote: Hi Jack, tl;dr; try starting Erlang with '+MEas?bf' Long explanation: If the memory is used by the ets_alloc allocator, it can be either actively used by ETS tables (such as Mnesia disc_copies tables) or it can be lost due to memory fragmentation. I guess this is memory fragmentation in your case, but let's quickly rule out the other possibility first. (By the way, in either case, garbage collecting processes won't reclaim the memory. GC operates on eheap_alloc allocated memory.) So if erlang:memory(ets) is close to what recon reports as allocated, the memory is in use. Your system wrote a lot of data to ETS/Mnesia, and didn't remove it afterwards. Inspect the contents of the tables and figure out what was left there that shouldn't be there, which part of the application should have removed that data and why didn't it do its job properly. The other, more likely option is fragmentation. I also experienced that the default memory allocation strategy (aoffcbf = address order first fit carrier best fit) can perform poorly when you use a lot of memory. All address order first fit strategies will use heavily the multiblock carriers with the lowest memory addresses, and if you have many carriers, those placed higher in memory will have less and less chance to be used. In my particular case ets_alloc created a total of 150k multiblock carriers for storing ~1.2TB data in ETS tables. This resulted in ~100 GB unused memory being wasted in the high address carriers. You have a much smaller system, but depending on the usage patterns of your ETS data you can end up in a similar situation. You can check the number of carriers with erlang:system_info({allocator, ets_alloc}). It will print something like this:[{instance,0,? ? ? ? ? ?[{versions,"0.9","3.0"},? ? ? ? ? ? {options,[...]},? ? ? ? ? ? {mbcs,[...? ? ? ? ? ? ? ? ? ?{carriers,1,1,1}, ?%% <- number of multi block carriers = 1? ? ? ? ? ? ? ? ? ?...]},? ? ? ? ? ? {sbcs,[...? ? ? ? ? ? ? ? ? ?{carriers,0,0,0},??%% <- number of single block carriers = 0? ? ? ? ? ? ? ? ? ?...]},? ? ? ? ? ? {calls,[...]}]},?{instance,1,...Check theaw numbera across all your allocator instances. You will typically have very few single block carriers (unless you store huge records in ETS). In my experience, fragmentation correlates well with the number of carriers an allocator handles, and can become quite significant above ~10 carriers. So, If you have the same problem I described, I have some bad news and some good news. The bad news is that I don't know a way of forcing the VM to defragment memory and get rid of the waste. The good news is that the bf (best fit) allocation strategy (which used to be the default up to R16) performs much better when you have many carriers. You need to pass the?'+MEas bf' command line argument to the VM to switch ets_alloc to bf strategy. Hope it helps,Daniel On Sat, 1 Apr 2017 at 05:36 Jack Tang wrote: After setting up erlang memory visualization, we find etc allocator does not release the memory during some historical datum are remove from mnesia tables. Can I release the memory on the fly rather than restart the mnesia application? Thanks! BR On Sun, Jan 15, 2017 at 4:47 AM, D?niel Szoboszlay wrote: Hi Jack, I guess the 9 GB is lost due to memory fragmentation. Erlang allocates memory in large chunks called carriers from the OS, then places the blocks your program actually needs on these carriers. A carrier can only be returned to the OS once all the blocks on it have been freed (and even then, the memory allocator may decide to keep it around for a while in case more memory is needed). You can check with?recon_alloc?how much unused memory is lost due to fragmentation in the various allocators. The bad news is that you cannot defragment the carriers, and if the selected memory allocator strategy doesn't work well for your application, you cannot change it either without restarting the emulator. However, if the memory is wasted in the eheap_alloc, you may try to force a GC on all processes a couple of times. As the GC copies the memory, it will allocate new blocks and free up the old heap blocks. So there's a chance the allocators can compact the blocks together on fewer segments. But that's just a guess, it may or may not work at all. Cheers,Daniel On Sat, 14 Jan 2017 at 08:04 Jack Tang wrote: Hello list, I run one Erlang application on Debian server and today I find the beam process consumes around 35G memory by `top` command. ```KiB Mem: ?99194912 total, 61682656 used, 37512252 free, ? 397380 buffersKiB Swap: ? ? ? ?0 total, ? ? ? ?0 used, ? ? ? ?0 free. 18684864 cached Mem ? PID USER ? ? ?PR ?NI ? ?VIRT ? ?RES ? ?SHR S ?%CPU %MEM ? ? TIME+ COMMAND11858 usr1 ? 20 ? 0 36.850g 0.032t ? 6220 S ?73.5 34.4 ? 8038:49 beam.smp``` I connect to the Erlang application using remote shell and find the mem-leaked supervisor tree and run gc on the whole tree. Code looks like blow: ```lists:foreach(fun(E) -> PId = element(2, E), erlang:garbage_collect(PId) end, supervisor:which_children( some_thing_sup)). ``` and erlang:memory() decreases from 32G to 23G.```[{total,22982011544},?{processes,12884182336},?{processes_used,12884170336},?{system,10097829208},?{atom,13828705},?{atom_used,13796692},?{binary,170530288},?{code,16450626},?{ets,9637717576}]``` However, when I input `top` command, the beam process still takes 35G memory. What can I do to release the 9G memory? Thanks BR-Jack______________________________ _________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/ listinfo/erlang-questions -- Jack Tang http://www.linkedin.com/in/ jacktang ______________________________ _________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/ listinfo/erlang-questions -- Jack Tang http://www.linkedin.com/in/jacktang _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Apr 6 19:33:25 2017 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 6 Apr 2017 10:33:25 -0700 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: Message-ID: <58E67BE5.5040104@gmail.com> It would help if you switched to using PropEr instead of QuickCheck, since QuickCheck for Erlang requires a license. Using PropEr instead would be more similar to using QuickCheck in Haskell due to its availability. That should also allow tests like these to be included easily in Erlang/OTP source and reused when testing the Erlang/OTP source code. Best Regards, Michael On 04/06/2017 06:26 AM, Bj?rn Gustavsson wrote: > I have written a QuickCheck module to test Dan's new string module > (https://github.com/erlang/otp/pull/1330). It found some bugs. > > Here is the gist for anyone interested: > > https://gist.github.com/bjorng/03a869392d5a969ebf2c40044b664190 > > Comments are welcome. This is my first major use of QuickCheck. I am > interested in how I could improve the QC specifications and > generators. > > /Bj?rn From mjtruog@REDACTED Thu Apr 6 20:11:25 2017 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 6 Apr 2017 11:11:25 -0700 Subject: [erlang-questions] How can I release beam process memory? In-Reply-To: References: Message-ID: <58E684CD.6040504@gmail.com> Hi Jack, The simplest way to make an Erlang process automatically GC is to use a temporary Erlang process, since the death of the Erlang process will cause GC to occur soon after. It is common to avoid long-lived Erlang processes that create lots of temporary data quickly due to excessive memory consumption, and to refactor the long-lived process a temporary process is used to create the temporary data within (this matters at high request rates, though you may not see the problem with low request rates). You may also tune an Erlang process with fullsweep_after to cause GC to occur more frequently, but I see that as only performance tuning, not fixing a problem that should be fixed with a temporary Erlang process. There also is the erlang:garbage_collect function, though you shouldn't use that in production source code (source code that needs to be dependable as its expectations change). Best Regards, Michael On 04/06/2017 07:41 AM, Jack Tang wrote: > Hi Daniel > > Thank you for the detailed explanation and sharing the experiences. I focused on the performance of GC heavily before and ignored the allocation strategy. > > One more question, as you mentioned memory on eheap_alloc can be released by calling GC on every process. Right now I setup periodic gc process to free eheap. However can I set some process flag to make GC automatically? > > Thanks again! > > BR > -Jack > > On Thu, Apr 6, 2017 at 7:52 PM, D?niel Szoboszlay > wrote: > > Hi Jack, > > *tl;dr;* try starting Erlang with '+MEasbf' > > Long explanation: > > If the memory is used by the ets_alloc allocator, it can be either actively used by ETS tables (such as Mnesia disc_copies tables) or it can be lost due to memory fragmentation. I guess this is memory fragmentation in your case, but let's quickly rule out the other possibility first. (By the way, in either case, garbage collecting processes won't reclaim the memory. GC operates on eheap_alloc allocated memory.) > > So if erlang:memory(ets) is close to what recon reports as allocated, the memory is in use. Your system wrote a lot of data to ETS/Mnesia, and didn't remove it afterwards. Inspect the contents of the tables and figure out what was left there that shouldn't be there, which part of the application should have removed that data and why didn't it do its job properly. > > The other, more likely option is fragmentation. I also experienced that the default memory allocation strategy (aoffcbf = address order first fit carrier best fit) can perform poorly when you use a lot of memory. All address order first fit strategies will use heavily the multiblock carriers with the lowest memory addresses, and if you have many carriers, those placed higher in memory will have less and less chance to be used. In my particular case ets_alloc created a total of 150k multiblock carriers for storing ~1.2TB data in ETS tables. This resulted in ~100 GB unused memory being wasted in the high address carriers. You have a much smaller system, but depending on the usage patterns of your ETS data you can end up in a similar situation. > > You can check the number of carriers with erlang:system_info({allocator, ets_alloc}). It will print something like this: > [{instance,0, > [{versions,"0.9","3.0"}, > {options,[...]}, > {mbcs,[... > {carriers,1,1,1}, %% <- number of multi block carriers = 1 > ...]}, > {sbcs,[... > {carriers,0,0,0}, %% <- number of single block carriers = 0 > ...]}, > {calls,[...]}]}, > {instance,1,... > Check theaw numbera across all your allocator instances. You will typically have very few single block carriers (unless you store huge records in ETS). In my experience, fragmentation correlates well with the number of carriers an allocator handles, and can become quite significant above ~10 carriers. > > So, If you have the same problem I described, I have some bad news and some good news. The bad news is that I don't know a way of forcing the VM to defragment memory and get rid of the waste. The good news is that the bf (best fit) allocation strategy (which used to be the default up to R16) performs much better when you have many carriers. You need to pass the '+MEas bf' command line argument to the VM to switch ets_alloc to bf strategy. > > Hope it helps, > Daniel > > On Sat, 1 Apr 2017 at 05:36 Jack Tang > wrote: > > After setting up erlang memory visualization, we find etc allocator does not release the memory during some historical datum are remove from mnesia tables. Can I release the memory on the fly rather than restart the mnesia application? Thanks! > > > > > BR > > On Sun, Jan 15, 2017 at 4:47 AM, D?niel Szoboszlay > wrote: > > Hi Jack, > > I guess the 9 GB is lost due to memory fragmentation. Erlang allocates memory in large chunks called carriers from the OS, then places the blocks your program actually needs on these carriers. A carrier can only be returned to the OS once all the blocks on it have been freed (and even then, the memory allocator may decide to keep it around for a while in case more memory is needed). > > You can check with recon_alloc how much unused memory is lost due to fragmentation in the various allocators. > > The bad news is that you cannot defragment the carriers, and if the selected memory allocator strategy doesn't work well for your application, you cannot change it either without restarting the emulator. > > However, if the memory is wasted in the eheap_alloc, you may try to force a GC on all processes a couple of times. As the GC copies the memory, it will allocate new blocks and free up the old heap blocks. So there's a chance the allocators can compact the blocks together on fewer segments. But that's just a guess, it may or may not work at all. > > Cheers, > Daniel > > On Sat, 14 Jan 2017 at 08:04 Jack Tang > wrote: > > Hello list, > > I run one Erlang application on Debian server and today I find the beam process consumes around 35G memory by `top` command. > > ``` > KiB Mem: 99194912 total, 61682656 used, 37512252 free, 397380 buffers > KiB Swap: 0 total, 0 used, 0 free. 18684864 cached Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 11858 usr1 20 0 36.850g 0.032t 6220 S 73.5 34.4 8038:49 beam.smp > ``` > > I connect to the Erlang application using remote shell and find the mem-leaked supervisor tree and run gc on the whole tree. Code looks like blow: > > ``` > lists:foreach(fun(E) -> PId = element(2, E), erlang:garbage_collect(PId) end, supervisor:which_children(some_thing_sup)). > ``` > > and erlang:memory() decreases from 32G to 23G. > ``` > [{total,22982011544}, > {processes,12884182336}, > {processes_used,12884170336}, > {system,10097829208}, > {atom,13828705}, > {atom_used,13796692}, > {binary,170530288}, > {code,16450626}, > {ets,9637717576}] > ``` > > However, when I input `top` command, the beam process still takes 35G memory. What can I do to release the 9G memory? Thanks > > BR > -Jack > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Jack Tang > > > http://www.linkedin.com/in/jacktang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Jack Tang > > > http://www.linkedin.com/in/jacktang > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From krzysztof.jurewicz@REDACTED Thu Apr 6 20:34:58 2017 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Thu, 06 Apr 2017 20:34:58 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: <58E67BE5.5040104@gmail.com> References: <58E67BE5.5040104@gmail.com> Message-ID: <87r315xu3x.fsf@gmail.com> Michael Truog writes: > It would help if you switched to using PropEr instead of QuickCheck, since QuickCheck for Erlang requires a license. Using PropEr instead would be more similar to using QuickCheck in Haskell due to its availability. That should also allow tests like these to be included easily in Erlang/OTP source and reused when testing the Erlang/OTP source code. Unfortunately, PropEr is licensed under GNU GPL 3.0, which is not compatible with Apache License 2.0. Probably this will not change anytime soon. See this issue for further reference: https://github.com/manopapad/proper/issues/29 triq may be a better solution, as it is licensed under Apache License 2.0. Here is the most active fork: https://github.com/triqng/triq From heinz@REDACTED Fri Apr 7 02:12:34 2017 From: heinz@REDACTED (Heinz N. Gies) Date: Fri, 7 Apr 2017 02:12:34 +0200 Subject: [erlang-questions] [ANN] otters: a community focused cleanup of the otter open tracing lib Message-ID: <96E3E094-4229-4FC5-9A24-945DF164080E@licenser.net> Because many otters build a damn a lot faster then a singe otter alone! The fork merges all (I hope I?ve not forgotten anything?) the community contributes, cleans up the API, dialyse, adds docs and is published as a hex package. Code: https://github.com/project-fifo/otters Package: https://hex.pm/packages/otters -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Fri Apr 7 14:06:08 2017 From: mbike2000ru@REDACTED (Dmitry) Date: Fri, 7 Apr 2017 12:06:08 +0000 (UTC) Subject: [erlang-questions] How to force init/1 to start at application start References: <295444666.2184116.1491566768068.ref@mail.yahoo.com> Message-ID: <295444666.2184116.1491566768068@mail.yahoo.com> Hello I have an init/1 function which does not start at application startup. init([Ref, Socket, Transport, Opts]) -> ? ? MyTable = ets:new(myphone, [set]), ? ? MyTable2 = ets:new(socket1, [set]),? ? lager:start(), ? Timer = erlang:send_after(1210000, self(), check), ? ? {ok, #state{ref=Ref, socket=Socket, transport=Transport, opts= Opts, dict = MyTable, dict2 = MyTable2, timer = Timer}, 0}. How to force it to start at an application startup and only once.? -------------- next part -------------- An HTML attachment was scrubbed... URL: From manojy@REDACTED Fri Apr 7 12:23:21 2017 From: manojy@REDACTED (Manoj) Date: Fri, 7 Apr 2017 15:53:21 +0530 Subject: [erlang-questions] Memory corruption due to erl_interface libraries that are implemented in C Message-ID: In my application I've a erlang node which periodically communicates with the C-node every 1 second to gather periodic data such as alarm and performance. The C-node is implemented in such a way that, it consists of two threads, one thread receives the requests from the erlang node and the other thread serves the queries and replies to erlang node. To reply to the erlang node all the data that is collected in the cnode need to converted to erlang format (in my case to a list of tuples) using the function erl_format. The problem observed here is, after running for approximately 45 minutes, I'm incurring a glibc error which complains about a memory corruption. What could be the probable cause for this? I'm using the 3.9 version of the erl_interface libraries which are compiled with thread safe options(such as _REENTRANT) Please find below the log that I got where glibc complains about a possible memory corruption When I did a addr2line on the address 0x101bb12c it pointed to the erl_format() *** glibc detected *** /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode: malloc(): memory corruption (fast): 0x1021fb08 *** ======= Backtrace: ========= /lib/libc.so.6[0xfd84610] /lib/libc.so.6[0xfd864fc] /lib/libc.so.6(__libc_malloc+0xb4)[0xfd887b8] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode(erl_eterm_alloc+0xac)[0x101ba1fc] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode(erl_alloc_eterm+0x2c)[0x101bb848] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode(erl_mk_tuple+0x94)[0x101b88c0] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode[0x101baf00] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode[0x101bb1bc] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode[0x101baf58] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode[0x101bb300] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode[0x101baee8] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode(erl_format+0x7c)[0x101bb12c] /root/rel-1.0.0/galaxy/lib/galaxy-1.6.0/priv/hyphy_cnode(query_handler+0x4264)[0x100235fc] /lib/libpthread.so.0[0xff967f4] /lib/libc.so.6(clone+0x8c)[0xfde226c] ======= Memory map: ======== 00100000-00103000 r-xp 00100000 00:00 0 [vdso] 0fc31000-0fc41000 r-xp 00000000 fd:01 3213 /lib/libresolv-2.5.so 0fc41000-0fc50000 ---p 00010000 fd:01 3213 /lib/libresolv-2.5.so 0fc50000-0fc51000 r--p 0000f000 fd:01 3213 /lib/libresolv-2.5.so 0fc51000-0fc52000 rwxp 00010000 fd:01 3213 /lib/libresolv-2.5.so 0fc52000-0fc54000 rwxp 0fc52000 00:00 0 0fc64000-0fc68000 r-xp 00000000 fd:01 3214 /lib/libnss_dns-2.5.so 0fc68000-0fc77000 ---p 00004000 fd:01 3214 /lib/libnss_dns-2.5.so 0fc77000-0fc78000 r--p 00003000 fd:01 3214 /lib/libnss_dns-2.5.so 0fc78000-0fc79000 rwxp 00004000 fd:01 3214 /lib/libnss_dns-2.5.so 0fc89000-0fc93000 r-xp 00000000 fd:01 3223 /lib/libnss_nis-2.5.so 0fc93000-0fca2000 ---p 0000a000 fd:01 3223 /lib/libnss_nis-2.5.so 0fca2000-0fca3000 r--p 00009000 fd:01 3223 /lib/libnss_nis-2.5.so 0fca3000-0fca4000 rwxp 0000a000 fd:01 3223 /lib/libnss_nis-2.5.so 0fcb4000-0fcc0000 r-xp 00000000 fd:01 3243 /lib/libnss_nisplus-2.5.so 0fcc0000-0fccf000 ---p 0000c000 fd:01 3243 /lib/libnss_nisplus-2.5.so 0fccf000-0fcd0000 r--p 0000b000 fd:01 3243 /lib/libnss_nisplus-2.5.so 0fcd0000-0fcd1000 rwxp 0000c000 fd:01 3243 /lib/libnss_nisplus-2.5.so 0fce1000-0fceb000 r-xp 00000000 fd:01 3240 /lib/libnss_files-2.5.so 0fceb000-0fcfa000 ---p 0000a000 fd:01 3240 /lib/libnss_files-2.5.so 0fcfa000-0fcfb000 r--p 00009000 fd:01 3240 /lib/libnss_files-2.5.so 0fcfb000-0fcfc000 rwxp 0000a000 fd:01 3240 /lib/libnss_files-2.5.so 0fd0c000-0fe49000 r-xp 00000000 fd:01 3215 /lib/libc-2.5.so 0fe49000-0fe59000 ---p 0013d000 fd:01 3215 /lib/libc-2.5.so 0fe59000-0fe5b000 r--p 0013d000 fd:01 3215 /lib/libc-2.5.so 0fe5b000-0fe5e000 rwxp 0013f000 fd:01 3215 /lib/libc-2.5.so 0fe5e000-0fe61000 rwxp 0fe5e000 00:00 0 0fe71000-0fe7a000 r-xp 00000000 fd:01 3272 /lib/librt-2.5.so 0fe7a000-0fe89000 ---p 00009000 fd:01 3272 /lib/librt-2.5.so 0fe89000-0fe8a000 r--p 00008000 fd:01 3272 /lib/librt-2.5.so 0fe8a000-0fe8b000 rwxp 00009000 fd:01 3272 /lib/librt-2.5.so 0fe8b000-0fe96000 rwxp 0fe8b000 00:00 0 0fea6000-0ff49000 r-xp 00000000 fd:01 3211 /lib/libm-2.5.so 0ff49000-0ff58000 ---p 000a3000 fd:01 3211 /lib/libm-2.5.so 0ff58000-0ff59000 r--p 000a2000 fd:01 3211 /lib/libm-2.5.so 0ff59000-0ff5d000 rwxp 000a3000 fd:01 3211 /lib/libm-2.5.so 0ff6d000-0ff70000 r-xp 00000000 fd:01 3202 /lib/libdl-2.5.so 0ff70000-0ff7f000 ---p 00003000 fd:01 3202 /lib/libdl-2.5.so 0ff7f000-0ff80000 r--p 00002000 fd:01 3202 /lib/libdl-2.5.so 0ff80000-0ff81000 rwxp 00003000 fd:01 3202 /lib/libdl-2.5.so 0ff91000-0ffa6000 r-xp 00000000 fd:01 3246 /lib/libpthread-2.5.so 0ffa6000-0ffb5000 ---p 00015000 fd:01 3246 /lib/libpthread-2.5.so 0ffb5000-0ffb6000 r--p 00014000 fd:01 3246 /lib/libpthread-2.5.so 0ffb6000-0ffb7000 rwxp 00015000 fd:01 3246 /lib/libpthread-2.5.so 0ffb7000-0ffb9000 rwxp 0ffb7000 00:00 0 0ffc9000- -- Regards, Manoj -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Apr 7 17:10:59 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 07 Apr 2017 15:10:59 +0000 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: Message-ID: On Thu, Apr 6, 2017 at 3:26 PM Bj?rn Gustavsson wrote: > > Comments are welcome. This is my first major use of QuickCheck. I am > interested in how I could improve the QC specifications and > generators. > > I think it's looking rather good. If you have the commercial variant of EQC, here are some things you may want to do: I'm very fond of using eqc:module({testing_budget, 30}, Mod) because it gives you 30 seconds of tests equally distributed over the properties in the file. Time bounds are usually nicer than number of tests. You can also weight your properties so those which most often fail are tested a bit more. I tend to pick 15 seconds when developing, 2-5 minutes for coffee runs, 30 minutes for lunch and 12 hours for when you leave the office or go to sleep. You can use the in_parallel transform on your file to execute your test cases on all cores. The speedup is more or less linear in the number of cores. Start using classification in your test cases. You want to classify on the structure of your generated strings, so you can see if you actually cover a realistic set of strings or if you are looking at rather small strings only. Collect information about the length of your strings. I have some tooling in https://github.com/jlouis/eqc_lib/blob/master/eqc_lib.erl for summarizing data in the form of what R does on a data set (and stem+leaf plots). Again, the goal is to verify that your generator is generating a realistic input set. Since we are trying to handle unicode, I would lace the input with a frequency generator which deliberately creates strings which are known to be naughty[0][1]. In principle we should hit them randomly after a while, but it is often simpler to just generate all the nasty strings more often in the code. Normal tests and use are likely to quickly hit the common faults. So go straight for the jugular: hit all the corner cases early and often. you want to hit errors in less than 100 test cases if possible. The goal here is to crash the code base. In general, look up what people in the security world are using as fuzzing inputs. Another point, which you may already cover, is that of negative testing: * Positive: Valid inputs must succeed with the right value * Negative: Invalid inputs should return the right error or throw an error In my maps_eqc tests, which are available at [2], we have the following lines: https://github.com/jlouis/maps_eqc/blob/3ab960018684785415e7265245889caf083e330c/src/maps_eqc.erl#L320-L379 which verifies the property of the maps module if you input values which are not valid maps or inputs. We can, in each case, predict what the error should be, especially in the situation of {badkey, K} errors. This in turn ensures that the error cases are hit in all cases. Typical strategy here is either to use the fault/2 generator and then use a parameter to alter the fault injection rate. Or to have separate properties which always generate faulty input. Lace the generator with a 10% fault injection rate at each part of your tree, say, so the chances of generating a fault is fairly high when multiple such are taken together. Then guard it with a ?SUCHTHAT on acutally having a fault. But beware having to search too much in the ?SUCHTHAT as that slows down test case generation. Classification of the types of faults become paramount here. You can find some of these strategies used in my enacl test cases[3] Feel free to question me with stuff if needed! [0] http://www.lookout.net/2011/06/special-unicode-characters-for-error.html [1] https://github.com/minimaxir/big-list-of-naughty-strings [2] https://github.com/jlouis/maps_eqc [3] https://github.com/jlouis/enacl/blob/master/eqc_test/enacl_eqc.erl -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Apr 7 19:14:49 2017 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 7 Apr 2017 10:14:49 -0700 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: <87r315xu3x.fsf@gmail.com> References: <87r315xu3x.fsf@gmail.com> Message-ID: what is the issue in using a tool using the gpl3 license? it's not like you will include in your code. Anyway how does triq compare? does it provides the same level of features. It would be indeed a lot better to provides tests in a form that anyone can launch without having to buy a really expensive license... - benoit On 6 April 2017 at 20:34:58, Krzysztof Jurewicz ( krzysztof.jurewicz@REDACTED) wrote: > Michael Truog writes: > > It would help if you switched to using PropEr instead of QuickCheck, since > QuickCheck for Erlang requires a license. Using PropEr instead would be > more similar to using QuickCheck in Haskell due to its availability. That > should also allow tests like these to be included easily in Erlang/OTP > source and reused when testing the Erlang/OTP source code. > > > Unfortunately, PropEr is licensed under GNU GPL 3.0, which is not > compatible with Apache License 2.0. Probably this will not change anytime > soon. See this issue for further reference: > https://github.com/manopapad/proper/issues/29 > > triq may be a better solution, as it is licensed under Apache License 2.0. > Here is the most active fork: https://github.com/triqng/triq > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bp@REDACTED Sat Apr 8 07:48:54 2017 From: bp@REDACTED (Bjarne Wichmann Bagge Petersen) Date: Sat, 8 Apr 2017 07:48:54 +0200 Subject: [erlang-questions] CT: outputting lager debug messages Message-ID: I have a little issue where I would like to output lager debug-messages when running common tests. But I can't get it to work. In `init_pr_suite` I have application:load(lager), application:set_env(lager, handlers, [{lager_console_backend, debug}, {lager_common_test_backend, debug}]), And in the test-code I have this little snippet: ct:log("ENV ~p", [application:get_all_env(lager)]), lager:warning("Now you see me"), lager:debug("Now you don't"), Which outputs: *** User 2017-04-08 07:36:42.089 *** ENV [{crash_log_count,5}, {crash_log_date,"$D0"}, {colored,false}, {crash_log_msg_size,65536}, {colors,[{debug,"\e[0;38m"}, {info,"\e[1;37m"}, {notice,"\e[1;36m"}, {warning,"\e[1;33m"}, {error,"\e[1;31m"}, {critical,"\e[1;35m"}, {alert,"\e[1;44m"}, {emergency,"\e[1;41m"}]}, {included_applications,[]}, {async_threshold,20}, {crash_log_size,10485760}, {error_logger_whitelist,[cth_readable_failonly]}, {error_logger_redirect,true}, {async_threshold_window,5}, {crash_log,"log/crash.log"}, {error_logger_hwm,50}, {handlers,[{lager_console_backend,debug}, {lager_common_test_backend,debug}]}] *** User 2017-04-08 07:36:42.090 *** 07:36:42.089 [warning] Now you see me Missing the debug-line. So what am I missing? Is `lager_common_test_backend` the wrong flag? Or is CT muting debug-messages anyway? Best regards Bjarne Wichmann Bagge Petersen Backend Developer, ShopGun E: bp@REDACTED L: dk.linkedin.com/in/bjarnewp W: shopgun.com A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark -- ShopGun, your helping hand in shopping, is available at shopgun.com and as the ShopGun app . This email is intended solely for the addressee and may contain confidential information and/or privileged confidential advice. If you receive this email by mistake, please do not read, print, transmit, store, or in any other way use the email or its attachments. Instead, please notify the addresser immediately and then permanently delete it. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Sun Apr 9 08:48:51 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 9 Apr 2017 08:48:51 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: <87r315xu3x.fsf@gmail.com> Message-ID: Note that string_eqc.erl is in Gist in my Github user and not in the otp repository. The tests that we "provide" are in a common_test test suite: https://github.com/dgud/otp/blob/1ac2425d7c4791fbce55cb662154e3646c5f9af0/lib/stdlib/test/string_SUITE.erl Those tests don't require QuickCheck or any similar tool. Tests for all bugs that we found using QuickCheck have been incorporated into string_SUITE.erl. We have no plans to incorporate QuickCheck into our test suites or daily builds. Using QuickCheck was just a way to find more bugs in a (more or less) new module before we released it. Instead of doing a traditional code review, we thought that we would find more bugs if we would write QuickCheck properties based on the documentation. And, yes, we could have used PropEr. I was not personally aware of triq. I choose QuickCheck because Ericsson has a license and because I have previously used eqc_erlang_program to generate random Erlang programs to test the compiler. /Bj?rn On Fri, Apr 7, 2017 at 7:14 PM, Benoit Chesneau wrote: > > what is the issue in using a tool using the gpl3 license? it's not like you > will include in your code. > > Anyway how does triq compare? does it provides the same level of features. > It would be indeed a lot better to provides tests in a form that anyone can > launch without having to buy a really expensive license... > > - benoit > > On 6 April 2017 at 20:34:58, Krzysztof Jurewicz > (krzysztof.jurewicz@REDACTED) wrote: >> >> Michael Truog writes: >> >> It would help if you switched to using PropEr instead of QuickCheck, since >> QuickCheck for Erlang requires a license. Using PropEr instead would be more >> similar to using QuickCheck in Haskell due to its availability. That should >> also allow tests like these to be included easily in Erlang/OTP source and >> reused when testing the Erlang/OTP source code. >> >> >> Unfortunately, PropEr is licensed under GNU GPL 3.0, which is not >> compatible with Apache License 2.0. Probably this will not change anytime >> soon. See this issue for further reference: >> https://github.com/manopapad/proper/issues/29 >> >> triq may be a better solution, as it is licensed under Apache License 2.0. >> Here is the most active fork: https://github.com/triqng/triq >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Sun Apr 9 09:36:42 2017 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 9 Apr 2017 09:36:42 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: Message-ID: Thanks! I will certainly save your email to use as a reference. I have not done any negative tests, mostly because I ran out of time. It is also somewhat tricky, because many functions don't always traverse all of their input string (e.g. string:equal/2 return false as soon as two characters are not equal; the rest of the input strings are not validated). /Bj?rn On Fri, Apr 7, 2017 at 5:10 PM, Jesper Louis Andersen wrote: > On Thu, Apr 6, 2017 at 3:26 PM Bj?rn Gustavsson wrote: >> >> >> Comments are welcome. This is my first major use of QuickCheck. I am >> interested in how I could improve the QC specifications and >> generators. >> > > I think it's looking rather good. If you have the commercial variant of EQC, > here are some things you may want to do: > > I'm very fond of using eqc:module({testing_budget, 30}, Mod) because it > gives you 30 seconds of tests equally distributed over the properties in the > file. Time bounds are usually nicer than number of tests. You can also > weight your properties so those which most often fail are tested a bit more. > I tend to pick 15 seconds when developing, 2-5 minutes for coffee runs, 30 > minutes for lunch and 12 hours for when you leave the office or go to sleep. > > You can use the in_parallel transform on your file to execute your test > cases on all cores. The speedup is more or less linear in the number of > cores. > > Start using classification in your test cases. You want to classify on the > structure of your generated strings, so you can see if you actually cover a > realistic set of strings or if you are looking at rather small strings only. > > Collect information about the length of your strings. I have some tooling in > https://github.com/jlouis/eqc_lib/blob/master/eqc_lib.erl for summarizing > data in the form of what R does on a data set (and stem+leaf plots). Again, > the goal is to verify that your generator is generating a realistic input > set. > > Since we are trying to handle unicode, I would lace the input with a > frequency generator which deliberately creates strings which are known to be > naughty[0][1]. In principle we should hit them randomly after a while, but > it is often simpler to just generate all the nasty strings more often in the > code. Normal tests and use are likely to quickly hit the common faults. So > go straight for the jugular: hit all the corner cases early and often. you > want to hit errors in less than 100 test cases if possible. The goal here is > to crash the code base. In general, look up what people in the security > world are using as fuzzing inputs. > > Another point, which you may already cover, is that of negative testing: > > * Positive: Valid inputs must succeed with the right value > * Negative: Invalid inputs should return the right error or throw an error > > In my maps_eqc tests, which are available at [2], we have the following > lines: > > https://github.com/jlouis/maps_eqc/blob/3ab960018684785415e7265245889caf083e330c/src/maps_eqc.erl#L320-L379 > > which verifies the property of the maps module if you input values which are > not valid maps or inputs. We can, in each case, predict what the error > should be, especially in the situation of {badkey, K} errors. This in turn > ensures that the error cases are hit in all cases. > > Typical strategy here is either to use the fault/2 generator and then use a > parameter to alter the fault injection rate. Or to have separate properties > which always generate faulty input. Lace the generator with a 10% fault > injection rate at each part of your tree, say, so the chances of generating > a fault is fairly high when multiple such are taken together. Then guard it > with a ?SUCHTHAT on acutally having a fault. But beware having to search too > much in the ?SUCHTHAT as that slows down test case generation. > Classification of the types of faults become paramount here. You can find > some of these strategies used in my enacl test cases[3] > > Feel free to question me with stuff if needed! > > [0] http://www.lookout.net/2011/06/special-unicode-characters-for-error.html > [1] https://github.com/minimaxir/big-list-of-naughty-strings > [2] https://github.com/jlouis/maps_eqc > [3] https://github.com/jlouis/enacl/blob/master/eqc_test/enacl_eqc.erl -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From krzysztof.jurewicz@REDACTED Sun Apr 9 11:48:37 2017 From: krzysztof.jurewicz@REDACTED (Krzysztof Jurewicz) Date: Sun, 09 Apr 2017 11:48:37 +0200 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: References: <87r315xu3x.fsf@gmail.com> Message-ID: <87h91xykqy.fsf@gmail.com> Benoit Chesneau writes: > what is the issue in using a tool using the gpl3 license? it's not like you > will include in your code. You have to include PropEr header file which is licensed under GPL v3. (I?m not saying that there would be no licensing problems if this file was not included). > Anyway how does triq compare? does it provides the same level of features. I would guess that PropEr is technically superior to triq, but I have no systematic comparison to prove it. From duncan@REDACTED Sun Apr 9 21:31:48 2017 From: duncan@REDACTED (duncan@REDACTED) Date: Sun, 09 Apr 2017 12:31:48 -0700 Subject: [erlang-questions] 'Dynamic Calls' Message-ID: <20170409123148.7e43b23f706d1a78218bd3e1c66e57ee.a8a8581678.wbe@email23.godaddy.com> An HTML attachment was scrubbed... URL: From jakub.groncki@REDACTED Sun Apr 9 21:46:42 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Sun, 9 Apr 2017 21:46:42 +0200 Subject: [erlang-questions] Shell pid changes after an error occurred Message-ID: Hi, My first message here so it may be very n00b but well, I just started to pick up some Erlang a couple of days ago (I'm halfway through "Introducing Erlang" + I find this book quite enjoyable). So there's this one thing that's been bugging me ever since I learnt about processes, pids and the like - the shell pid seems to change after an error caused by malformed code or after a user exception is thrown. See this terminal session snippet: Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.3 (abort with ^G) 1> self(). <0.58.0> 2> error(sadface). ** exception error: sadface 3> self(). <0.61.0> So here you can see self() yields different values. Does anybody know why it happens the way it happens? PS Of course I'm not implying there's something wrong with this kind of behavior but still it's really interesting. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sun Apr 9 22:53:01 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sun, 9 Apr 2017 22:53:01 +0200 Subject: [erlang-questions] Shell pid changes after an error occurred In-Reply-To: References: Message-ID: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> Before running your error command, try: catch_exception(true). http://erlang.org/doc/man/shell.html#catch_exception-1 On 04/09/2017 09:46 PM, Jakub Groncki wrote: > Hi, > > My first message here so it may be very n00b but well, I just started to > pick up some Erlang a couple of days ago (I'm halfway through > "Introducing Erlang" + I find this book quite enjoyable). So there's > this one thing that's been bugging me ever since I learnt about > processes, pids and the like - the shell pid seems to change after an > error caused by malformed code or after a user exception is thrown. See > this terminal session snippet: > > Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V8.3 (abort with ^G) > 1> self(). > <0.58.0> > 2> error(sadface). > ** exception error: sadface > 3> self(). > <0.61.0> > > So here you can see self() yields different values. Does anybody know > why it happens the way it happens? > > PS Of course I'm not implying there's something wrong with this kind of > behavior but still it's really interesting. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From jesper.louis.andersen@REDACTED Sun Apr 9 23:05:17 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 09 Apr 2017 21:05:17 +0000 Subject: [erlang-questions] 'Dynamic Calls' In-Reply-To: <20170409123148.7e43b23f706d1a78218bd3e1c66e57ee.a8a8581678.wbe@email23.godaddy.com> References: <20170409123148.7e43b23f706d1a78218bd3e1c66e57ee.a8a8581678.wbe@email23.godaddy.com> Message-ID: Usually, it is a trade-off between readability, dialyzer/xref precision, and generality. The price of using many dynamic modules is that you can't use something like grep(1) anymore to find out who are callers. Likewise, tools usch as xref fails to see the call graph. This argues the notion should be used with care, although there are many good reasons for dynamically invoking a module as well. What to do depends on the problem. Sometimes code can just be rewritten to something simpler. Other times, you can create a callback system and use that. Or you may be able to let a target process vary. And of course there are situations where the best approach is to just use dynamic modules in the code base. A somewhat sound advice is to make sure there are few dynamically injected modules and that they are not subject to change as the system runs. Otherwise, the code may be hard to follow for a newcomer. On Sun, Apr 9, 2017 at 9:32 PM wrote: > I use the "standard" elvis rules for checking my code and I'm getting an > error I'd like to understand the reasoning behind the rule. The error I'm > receiving is: > invalid_dynamic_call - Remove the dynamic function call on line 47. Only > modules that define callbacks should make dynamic calls > > The line in question is > Svr:stop(), > so the module is variable which is bound to an atom from a list in a state > map (ie the servers to which a message should be sent, in this particular > case it's a stop message). > > My first question is what is the reasoning behind the elvis rule (ie > what's wrong with dynamic calls)? > > My followup question is what should I do instead? Note that making a > function head for every potential choice (33 at moment) is doable for this > case but seems tedious and what if the number was much larger (which it > will be in other parts of the simulator being built)? > > Duncan Sparrell > sFractal Consulting LLC > iPhone, iTypo, iApologize > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakub.groncki@REDACTED Sun Apr 9 22:56:49 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Sun, 9 Apr 2017 22:56:49 +0200 Subject: [erlang-questions] Shell pid changes after an error occurred In-Reply-To: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> References: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> Message-ID: Oh wow, this is neat, I would have never guessed, thank you :) 2017-04-09 22:53 GMT+02:00 Lo?c Hoguin : > Before running your error command, try: > > catch_exception(true). > > http://erlang.org/doc/man/shell.html#catch_exception-1 > > > On 04/09/2017 09:46 PM, Jakub Groncki wrote: > >> Hi, >> >> My first message here so it may be very n00b but well, I just started to >> pick up some Erlang a couple of days ago (I'm halfway through >> "Introducing Erlang" + I find this book quite enjoyable). So there's >> this one thing that's been bugging me ever since I learnt about >> processes, pids and the like - the shell pid seems to change after an >> error caused by malformed code or after a user exception is thrown. See >> this terminal session snippet: >> >> Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] >> [hipe] [kernel-poll:false] >> >> Eshell V8.3 (abort with ^G) >> 1> self(). >> <0.58.0> >> 2> error(sadface). >> ** exception error: sadface >> 3> self(). >> <0.61.0> >> >> So here you can see self() yields different values. Does anybody know >> why it happens the way it happens? >> >> PS Of course I'm not implying there's something wrong with this kind of >> behavior but still it's really interesting. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Apr 9 23:29:19 2017 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 9 Apr 2017 14:29:19 -0700 Subject: [erlang-questions] QuickCheck module for testing the new string module In-Reply-To: <87h91xykqy.fsf@gmail.com> References: <87r315xu3x.fsf@gmail.com> <87h91xykqy.fsf@gmail.com> Message-ID: <58EAA7AF.6020101@gmail.com> On 04/09/2017 02:48 AM, Krzysztof Jurewicz wrote: > Benoit Chesneau writes: > >> what is the issue in using a tool using the gpl3 license? it's not like you >> will include in your code. > You have to include PropEr header file which is licensed under GPL v3. (I?m not saying that there would be no licensing problems if this file was not included). > >> Anyway how does triq compare? does it provides the same level of features. > I would guess that PropEr is technically superior to triq, but I have no systematic comparison to prove it. > Using PropEr as a build/test dependency is similar to relying on autoconf (under GPL license), but it would be more ideal to have triq usage within the Erlang/OTP installation, to be used like QuickCheck is used in Haskell, i.e., for any testing without restriction. Having repeatable tests is an important part of science, so this is a very basic, fundamental concern. Best Regards, Michael From ok@REDACTED Mon Apr 10 06:48:41 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 10 Apr 2017 16:48:41 +1200 Subject: [erlang-questions] Shell pid changes after an error occurred In-Reply-To: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> References: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> Message-ID: <690C0559-3E53-4E85-BA00-3A79874B00A2@cs.otago.ac.nz> Reflecting on this, I know *I* was surprised the first time I realised that exceptions crashed and restarted the shell. One reason it was surprising was that 1> X = 1. 1 2> Y. * 1: variable 'Y' is unbound 3> error(goo). ** exception error: goo 4> X. 1 5> Y = self(). <0.37.0> 6> error(cthu). ** exception error: cthu 7> {Y, self()}. {<0.37.0>,<0.41.0>} so that "ports and ETS tables as well as processes linked to the evaluator process" don't "survive the exception" in contrast to variable bindings, which visibly DO survive. The case of Y is particularly confusing. Would it help if restarted shells printed a message "Shell restarted: associated ports, ETS tables, and links lost." ? From jakub.groncki@REDACTED Mon Apr 10 08:14:51 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Mon, 10 Apr 2017 08:14:51 +0200 Subject: [erlang-questions] Shell pid changes after an error occurred In-Reply-To: <690C0559-3E53-4E85-BA00-3A79874B00A2@cs.otago.ac.nz> References: <08c50e6e-b335-d394-fd6a-6a94a7385d5d@ninenines.eu> <690C0559-3E53-4E85-BA00-3A79874B00A2@cs.otago.ac.nz> Message-ID: I think it would be more than helpful but on the other hand some new users could say that the shell is buggy because it crashes and restarts for no good reason... 2017-04-10 6:48 GMT+02:00 Richard A. O'Keefe : > Reflecting on this, I know *I* was surprised the first time I > realised that exceptions crashed and restarted the shell. > One reason it was surprising was that > 1> X = 1. > 1 > 2> Y. > * 1: variable 'Y' is unbound > 3> error(goo). > ** exception error: goo > 4> X. > 1 > 5> Y = self(). > <0.37.0> > 6> error(cthu). > ** exception error: cthu > 7> {Y, self()}. > {<0.37.0>,<0.41.0>} > > so that "ports and ETS tables as well as processes linked to > the evaluator process" don't "survive the exception" in > contrast to variable bindings, which visibly DO survive. > > The case of Y is particularly confusing. > > Would it help if restarted shells printed a message > "Shell restarted: associated ports, ETS tables, and links lost." > ? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Mon Apr 10 14:45:49 2017 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Mon, 10 Apr 2017 09:45:49 -0300 Subject: [erlang-questions] 'Dynamic Calls' In-Reply-To: References: <20170409123148.7e43b23f706d1a78218bd3e1c66e57ee.a8a8581678.wbe@email23.godaddy.com> Message-ID: Hi Duncan, This is the reasoning behind the rule. As usual, Jesper explained it in a much much better way :) Now, my question would be: If you have ~33 modules with the same function, why not defining a behavior for them? Simply adding a proper -callback (and maybe -optional_callbacks(?)) attribute(s) to your module where you have the dynamic call and -behaviour(?) to the others will not only remove Elvis warning but also let the compiler check if you?re doing things right ;) Cheers! > On Apr 9, 2017, at 18:05, Jesper Louis Andersen wrote: > > Usually, it is a trade-off between readability, dialyzer/xref precision, and generality. > > The price of using many dynamic modules is that you can't use something like grep(1) anymore to find out who are callers. Likewise, tools usch as xref fails to see the call graph. This argues the notion should be used with care, although there are many good reasons for dynamically invoking a module as well. > > What to do depends on the problem. Sometimes code can just be rewritten to something simpler. Other times, you can create a callback system and use that. Or you may be able to let a target process vary. And of course there are situations where the best approach is to just use dynamic modules in the code base. > > A somewhat sound advice is to make sure there are few dynamically injected modules and that they are not subject to change as the system runs. Otherwise, the code may be hard to follow for a newcomer. > > On Sun, Apr 9, 2017 at 9:32 PM > wrote: > I use the "standard" elvis rules for checking my code and I'm getting an error I'd like to understand the reasoning behind the rule. The error I'm receiving is: > invalid_dynamic_call - Remove the dynamic function call on line 47. Only modules that define callbacks should make dynamic calls > The line in question is > > Svr:stop(), > so the module is variable which is bound to an atom from a list in a state map (ie the servers to which a message should be sent, in this particular case it's a stop message). > > My first question is what is the reasoning behind the elvis rule (ie what's wrong with dynamic calls)? > > My followup question is what should I do instead? Note that making a function head for every potential choice (33 at moment) is doable for this case but seems tedious and what if the number was much larger (which it will be in other parts of the simulator being built)? > > Duncan Sparrell > sFractal Consulting LLC > iPhone, iTypo, iApologize > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Mon Apr 10 16:25:06 2017 From: duncan@REDACTED (Duncan) Date: Mon, 10 Apr 2017 14:25:06 +0000 (UTC) Subject: [erlang-questions] 'Dynamic Calls' In-Reply-To: References: <20170409123148.7e43b23f706d1a78218bd3e1c66e57ee.a8a8581678.wbe@email23.godaddy.com> Message-ID: <70C9BBDBF9B46632.65341C3F-8AFC-4E4C-9489-A55D55E42C20@mail.outlook.com> Jesper, Brujo,Thank you.?Wrt behavior/callbacks - The main reason I'm not using callbacks is due to my ignorance. I sort of know how to use them (obviously not fully understanding them) when others design the behavior and I use it, but I haven't made any myself. The 33 modules are all gen_servers and I think the modules I'm calling are the public api (eg in this case calling the stop function which sends a stop message to a selected subset of them). I don't understand what the behavior would be and the callbacks in this case for the 'stop a selected subset of servers' function.Duncan iPhone, iTypo, iApologize Duncan Sparrell sFractal Consulting, LLC The closer you look, the more you see _____________________________ From: Brujo Benavides Sent: Monday, April 10, 2017 8:45 AM Subject: Re: [erlang-questions] 'Dynamic Calls' To: Jesper Louis Andersen Cc: , Hi Duncan, This?is the reasoning behind the rule. As usual, Jesper explained it in a much much better way :) Now, my question would be: If you have ~33 modules with the same function, why not defining a behavior for them? Simply adding a proper -callback (and maybe -optional_callbacks(?)) attribute(s) to your module where you have the dynamic call and -behaviour(?) to the others will not only remove Elvis warning but also let the compiler check if you?re doing things right ;) Cheers! On Apr 9, 2017, at 18:05, Jesper Louis Andersen wrote: Usually, it is a trade-off between readability, dialyzer/xref precision, and generality. The price of using many dynamic modules is that you can't use something like grep(1) anymore to find out who are callers. Likewise, tools usch as xref fails to see the call graph. This argues the notion should be used with care, although there are many good reasons for dynamically invoking a module as well. What to do depends on the problem. Sometimes code can just be rewritten to something simpler. Other times, you can create a callback system and use that. Or you may be able to let a target process vary. And of course there are situations where the best approach is to just use dynamic modules in the code base. A somewhat sound advice is to make sure there are few dynamically injected modules and that they are not subject to change as the system runs. Otherwise, the code may be hard to follow for a newcomer. On Sun, Apr 9, 2017 at 9:32 PM wrote: I use the "standard" elvis rules for checking my code and I'm getting an error I'd like to understand the reasoning behind the rule. The error I'm receiving is:invalid_dynamic_call - Remove the dynamic function call on line 47. Only modules that define callbacks should make dynamic calls The line in question is Svr:stop(), so the module is variable which is bound to an atom from a list in a state map (ie the servers to which a message should be sent, in this particular case it's a stop message). My first question is what is the reasoning behind the elvis rule (ie what's wrong with dynamic calls)? My followup question is what should I do instead? Note that making a function head for every potential choice (33 at moment) is doable for this case but seems tedious and what if the number was much larger (which it will be in other parts of the simulator being built)? Duncan SparrellsFractal Consulting LLCiPhone, iTypo, iApologize_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From christopher.meiklejohn@REDACTED Mon Apr 10 18:22:46 2017 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Mon, 10 Apr 2017 18:22:46 +0200 Subject: [erlang-questions] Curry On 2017 - Call for Presentations Message-ID: ------------------------------------------------------------------------------ Curry On Barcelona! (Curry On 2017) Barcelona, Spain, June 19-20, 2017 co-located with PLDI + ECOOP www.curry-on.org/2017 ------------------------------------------------------------------------------ Curry On has two central goals. Bringing industry and academia together to have an open conversation. Exploring the intersection of programming or programming languages with emerging challenges in industry, such as security, big data, or machine learning. Talks can cover topics as varied as an academic idea applied to a surprising problem domain in a real-world system, to something racy like ?programming language academia is irrelevant and here?s why?. We give no firm guidelines on topics (go wild!), however we particularly seek talks that can provoke thought and discussion, or even (more idealistically) can elicit or encourage new ideas or change. Areas of particular interest to the conference include: - distributed systems/big data - machine learning - security - concurrency - types - functional or logic programming - databases - compilers and virtual machines - PL approaches to front-end development - probabilistic programming - next generation tooling We sold out of tickets (last year), so please do consider joining our lineup of excellent speakers! Some of our excellent speakers so far: ====================================== KEYNOTEs - Martin Odersky, EPFL, (creator of Scala language) - Brian Goetz, Oracle, (Java) - Jos? Valim, Platformatec, (creator of Elixir language) - Roberto Ierusalimschy, PUC-Rio, (Lua) INVITED - Jean Yang, CMU (privacy & security + PL) - Niko Matsakis, Mozilla Research (Rust) - Julien Verlaguet, Facebook (Hack) - Dave Thomas, Kx Systems (Array programming) - Helena Edelson, (big data, distributed systems, Scala) - Martin Abadi, Google (security, PL, TensorFlow) - Matt Might, University of Utah (precision medicine, static analysis) - John Hughes, Chalmers University, (property-based testing, FP) Talk Formats ============ Accepted speakers can choose from: - Normal talk slot (40 minutes) = or = - Chesstimer talk slot (20 minutes speaker time, 20 minutes audience time) About Chess-Timer Talks ----------------------- One of Curry On's prime goals is to bring industry and academia together to have an open conversation. But one speaker addressing hundreds of people is hardly a conversation. So, we're changing that. Chess-timer talks aim to make tech conferences a more interactive, more fun, and better place for learning and discussions. Chess-timer talks primarily seek to get more audience members participating in the presentation. Speakers who choose to give a chess-timer talk are allowed 20 minutes of solo speaking time, and 20 minutes of discussion time. A Curry On representative operates a chess-timer during the presentation, switching between solo + discussion time budgets. When an audience member interrupts the talk to ask a question, for example, we switch the timer to deduct from discussion time. This style of giving talks has been demoed in small (20-30 person) academic workshops to great success. In those workshops, it has incentivized speakers to strategically insert fodder for discussion points into their presentations so as to try and elicit audience members to ask questions. Speakers with selected talks will have the opportunity to choose whether or not they would like to give a chess-timer talk. Curry On will feature a limited number of chess-timer sessions, meaning that a majority of talks will still be standard conference talks. Deadlines ========= - Call for presentations closes: -> Friday, April 14th, 2017 <- - Speakers notified/program announced: -> Friday, April 21st, 2017 <- From max.lapshin@REDACTED Mon Apr 10 21:10:48 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 10 Apr 2017 22:10:48 +0300 Subject: [erlang-questions] luerl Message-ID: Hi. We are using luerl in production and we have some issues with it like lack of repl or stacktraces. I wonder if there are any other people that are using it and ready to commit to public repo or people who are ready to implement features above. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Mon Apr 10 21:54:52 2017 From: andre@REDACTED (Andre Graf) Date: Mon, 10 Apr 2017 21:54:52 +0200 Subject: [erlang-questions] luerl In-Reply-To: References: Message-ID: <8855f383-f388-255c-4738-a25e97d6a3ea@dergraf.org> Hi Max, the VerneMQ broker is using Luerl. We would certainly be interested in better stacktraces for Luerl. Depending on the effort, we could contribute something. Cheers, Andre On 04/10/2017 09:10 PM, Max Lapshin wrote: > Hi. > > We are using luerl in production and we have some issues with it like > lack of repl or stacktraces. > > I wonder if there are any other people that are using it and ready to > commit to public repo or people who are ready to implement features above. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From frank.muller.erl@REDACTED Mon Apr 10 22:19:13 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 10 Apr 2017 20:19:13 +0000 Subject: [erlang-questions] Lager's "console_backend" not working Message-ID: Hi, I?m using Lager in a basic Erlang app on Windows 7 (and even 10). Whenever I activate the console backend, I get this warning message: > WARNING: Lager's console backend is incompatible with the 'old' shell, not enabling it Can someone explain me what does it mean and how I can fix it? /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakub.groncki@REDACTED Mon Apr 10 22:26:14 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Mon, 10 Apr 2017 22:26:14 +0200 Subject: [erlang-questions] Lager's "console_backend" not working In-Reply-To: References: Message-ID: Hi, It's always good to look at the code: is_new_style_console_available() -> %% Criteria: %% 1. If the user has specified '-noshell' on the command line, %% then we will pretend that the new-style console is available. %% If there is no shell at all, then we don't have to worry %% about log events being blocked by the old-style shell. %% 2. Windows doesn't support the new shell, so all windows users %% have is the oldshell. %% 3. If the user_drv process is registered, all is OK. %% 'user_drv' is a registered proc name used by the "new" %% console driver. [...] try {is_new_style_console_available(), lager_util:config_to_mask(Level)} of {false, _} -> Msg = "Lager's console backend is incompatible with the 'old' shell, not enabling it", 2017-04-10 22:19 GMT+02:00 Frank Muller : > Hi, > > I?m using Lager in a basic Erlang app on Windows 7 (and even 10). > > Whenever I activate the console backend, I get this warning message: > > > WARNING: Lager's console backend is incompatible with the 'old' shell, > not enabling it > > > Can someone explain me what does it mean and how I can fix it? > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Apr 11 05:27:06 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 11 Apr 2017 03:27:06 +0000 Subject: [erlang-questions] Lager's "console_backend" not working In-Reply-To: References: Message-ID: Hi Jakub So because of 2 (I'm on windows), only old-shell is available and thus I can't get messages in the console :-/ ??? How people debug things in the console then? /Frank Hi, > It's always good to look at the code: > > is_new_style_console_available() -> > %% Criteria: > %% 1. If the user has specified '-noshell' on the command line, > %% then we will pretend that the new-style console is available. > %% If there is no shell at all, then we don't have to worry > %% about log events being blocked by the old-style shell. > %% 2. Windows doesn't support the new shell, so all windows users > %% have is the oldshell. > %% 3. If the user_drv process is registered, all is OK. > %% 'user_drv' is a registered proc name used by the "new" > %% console driver. > > [...] > > try {is_new_style_console_available(), lager_util:config_to_mask(Level)} of > {false, _} -> > Msg = "Lager's console backend is incompatible with the 'old' > shell, not enabling it", > > 2017-04-10 22:19 GMT+02:00 Frank Muller : > > Hi, > > I?m using Lager in a basic Erlang app on Windows 7 (and even 10). > > Whenever I activate the console backend, I get this warning message: > > > WARNING: Lager's console backend is incompatible with the 'old' shell, > not enabling it > > > Can someone explain me what does it mean and how I can fix it? > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakub.groncki@REDACTED Tue Apr 11 08:28:37 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Tue, 11 Apr 2017 08:28:37 +0200 Subject: [erlang-questions] Lager's "console_backend" not working In-Reply-To: References: Message-ID: Hi, Looks like it. Have you tried to run it with the file backend? That might work. This brief section explains it: https://github.com/basho/lager/blob/master/README.md#configuration 2017-04-11 5:27 GMT+02:00 Frank Muller : > Hi Jakub > > So because of 2 (I'm on windows), only old-shell is available and thus I > can't get messages in the console :-/ ??? > > How people debug things in the console then? > > /Frank > > Hi, >> It's always good to look at the code: >> >> is_new_style_console_available() -> >> %% Criteria: >> %% 1. If the user has specified '-noshell' on the command line, >> %% then we will pretend that the new-style console is available. >> %% If there is no shell at all, then we don't have to worry >> %% about log events being blocked by the old-style shell. >> %% 2. Windows doesn't support the new shell, so all windows users >> %% have is the oldshell. >> %% 3. If the user_drv process is registered, all is OK. >> %% 'user_drv' is a registered proc name used by the "new" >> %% console driver. >> >> [...] >> >> try {is_new_style_console_available(), lager_util:config_to_mask(Level)} >> of >> {false, _} -> >> Msg = "Lager's console backend is incompatible with the 'old' >> shell, not enabling it", >> >> 2017-04-10 22:19 GMT+02:00 Frank Muller : >> >> Hi, >> >> I?m using Lager in a basic Erlang app on Windows 7 (and even 10). >> >> Whenever I activate the console backend, I get this warning message: >> >> > WARNING: Lager's console backend is incompatible with the 'old' shell, >> not enabling it >> >> >> Can someone explain me what does it mean and how I can fix it? >> >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Tue Apr 11 16:33:48 2017 From: comptekki@REDACTED (Wes James) Date: Tue, 11 Apr 2017 08:33:48 -0600 Subject: [erlang-questions] can't use ssl lib in f26 compile Message-ID: I tried to "make" erlang 19.3 from source. All went well except it doesn't like the version of openssl that is installed. I am testing fedora 26, which just came out in alpha. Is there something in the Make file I can change so it accepts the the openssl that is there?. Thanks, wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Tue Apr 11 17:00:32 2017 From: comptekki@REDACTED (Wes James) Date: Tue, 11 Apr 2017 09:00:32 -0600 Subject: [erlang-questions] can't use ssl lib in f26 compile In-Reply-To: References: Message-ID: actually, it's configure that complains... On Tue, Apr 11, 2017 at 8:33 AM, Wes James wrote: > I tried to "make" erlang 19.3 from source. All went well except it > doesn't like the version of openssl that is installed. I am testing fedora > 26, which just came out in alpha. Is there something in the Make file I > can change so it accepts the the openssl that is there?. > > Thanks, > > wes > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakub.groncki@REDACTED Tue Apr 11 17:14:00 2017 From: jakub.groncki@REDACTED (Jakub Groncki) Date: Tue, 11 Apr 2017 17:14:00 +0200 Subject: [erlang-questions] can't use ssl lib in f26 compile In-Reply-To: References: Message-ID: Hi, otp_src_19.3/HOWTO/INSTALL.md contains the possible configuration options you can use to customize your install. Amongst others: * `--{with,without}-ssl` - OpenSSL (without implies that the `crypto`, `ssh`, and `ssl` won't be built) * `--with-ssl=PATH` - Specify location of OpenSSL include and lib * `--with-ssl-incl=PATH` - Location of OpenSSL `include` directory, if different than specified by `--with-ssl=PATH` You can use them in combination with ./configure 2017-04-11 17:00 GMT+02:00 Wes James : > actually, it's configure that complains... > > On Tue, Apr 11, 2017 at 8:33 AM, Wes James wrote: > >> I tried to "make" erlang 19.3 from source. All went well except it >> doesn't like the version of openssl that is installed. I am testing fedora >> 26, which just came out in alpha. Is there something in the Make file I >> can change so it accepts the the openssl that is there?. >> >> Thanks, >> >> wes >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Apr 11 18:05:07 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 11 Apr 2017 19:05:07 +0300 Subject: [erlang-questions] Lager's "console_backend" not working In-Reply-To: References: Message-ID: I usually change this place in lager code and use output not reading these important warnings. It is windows, anything here can hang. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Apr 11 18:37:10 2017 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 11 Apr 2017 19:37:10 +0300 Subject: [erlang-questions] can't use ssl lib in f26 compile In-Reply-To: References: Message-ID: <19F76424-32F9-44F7-A498-F58339714D3A@gmail.com> Hello, and you have openssl devel package installed on your system, right? alterative option to build openssl from sources. - Dmitry > On Apr 11, 2017, at 6:00 PM, Wes James wrote: > > actually, it's configure that complains... > > On Tue, Apr 11, 2017 at 8:33 AM, Wes James wrote: > I tried to "make" erlang 19.3 from source. All went well except it doesn't like the version of openssl that is installed. I am testing fedora 26, which just came out in alpha. Is there something in the Make file I can change so it accepts the the openssl that is there?. > > Thanks, > > wes > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ferigis@REDACTED Tue Apr 11 21:48:12 2017 From: ferigis@REDACTED (Felipe Ripoll Gisbert) Date: Tue, 11 Apr 2017 13:48:12 -0600 Subject: [erlang-questions] error with xref and on_load functions Message-ID: Hi all, I have a problem running xref on modules with `on_load` functions, for example, giving this module: -module(xreftest). -on_load(init/0). init() -> ok. if we run xref: 1> c(xreftest, [debug_info]). {ok,xreftest} 2> xref:d("."). [{deprecated,[]}, {undefined,[]}, {unused,[{xreftest,init,0}]}] we get an `unused` warning for init/0 but it is actually used on load. Is this a known issue? Thanks -- *Felipe Ripoll Gisbert* www.feliperipoll.es felipe@REDACTED ferigis@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From uwe@REDACTED Wed Apr 12 08:15:11 2017 From: uwe@REDACTED (uwe) Date: Wed, 12 Apr 2017 09:15:11 +0300 Subject: [erlang-questions] =?utf-8?q?great_news?= Message-ID: <1160443342.20170412091511@dauernheim.net> Dear! I've got great news for you, you're going to be really surprised, just take a look http://senzabedankjes.nl/steer.php?7c7d uwe -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Wed Apr 12 13:34:16 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Wed, 12 Apr 2017 17:04:16 +0530 (IST) Subject: [erlang-questions] registration process in ejabberd Message-ID: <2002178531.49530.1491996856373@webmail.blacklightsw.com> Hey everyone, I am a noob in terms of erlang and ejabberd and I have built a basic application in erlang. Now I saw that ejabberd also allows user to be registered and I also found the table 'users' in which it keeps its users. I also know that there is a register_user hook called from ejabberd_auth_autonomous.erl and spans through mod_metrics,mod_last and mod_shared_roster. But I was only able to track it in logs in case of mod_last.erl. But for the other modules I am not able to figure out the fact that ho exactly the control flows in these modules as soon as a request to register user is made.Anyone any Idea over this? I just want to use this functionality of ejabberd of registering users instead of using my own module. Thanks in advance. Regards, Abhishek Ranjan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.rascao@REDACTED Wed Apr 12 17:32:34 2017 From: luis.rascao@REDACTED (=?UTF-8?B?THVpcyBSYXNjw6Nv?=) Date: Wed, 12 Apr 2017 16:32:34 +0100 Subject: [erlang-questions] CT: outputting lager debug messages In-Reply-To: References: Message-ID: Maybe you're missing `lager:set_loglevel(lager_console_backend, debug).` ? On Sat, Apr 8, 2017 at 6:48 AM, Bjarne Wichmann Bagge Petersen < bp@REDACTED> wrote: > I have a little issue where I would like to output lager debug-messages > when running common tests. But I can't get it to work. > > In `init_pr_suite` I have > > application:load(lager), > application:set_env(lager, handlers, [{lager_console_backend, debug}, > {lager_common_test_backend, debug}]), > > And in the test-code I have this little snippet: > > ct:log("ENV ~p", [application:get_all_env(lager)]), > lager:warning("Now you see me"), > lager:debug("Now you don't"), > > Which outputs: > > *** User 2017-04-08 07:36:42.089 *** > ENV [{crash_log_count,5}, > {crash_log_date,"$D0"}, > {colored,false}, > {crash_log_msg_size,65536}, > {colors,[{debug,"\e[0;38m"}, > {info,"\e[1;37m"}, > {notice,"\e[1;36m"}, > {warning,"\e[1;33m"}, > {error,"\e[1;31m"}, > {critical,"\e[1;35m"}, > {alert,"\e[1;44m"}, > {emergency,"\e[1;41m"}]}, > {included_applications,[]}, > {async_threshold,20}, > {crash_log_size,10485760}, > {error_logger_whitelist,[cth_readable_failonly]}, > {error_logger_redirect,true}, > {async_threshold_window,5}, > {crash_log,"log/crash.log"}, > {error_logger_hwm,50}, > {handlers,[{lager_console_backend,debug}, > {lager_common_test_backend,debug}]}] > > *** User 2017-04-08 07:36:42.090 *** > 07:36:42.089 [warning] Now you see me > > > Missing the debug-line. > > So what am I missing? Is `lager_common_test_backend` the wrong flag? Or is > CT muting debug-messages anyway? > > > Best regards > > > Bjarne Wichmann Bagge Petersen > > Backend Developer, ShopGun > > > E: bp@REDACTED > L: dk.linkedin.com/in/bjarnewp > W: shopgun.com > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark > > ShopGun, your helping hand in shopping, is available at shopgun.com and > as the ShopGun app . > > This email is intended solely for the addressee and may contain > confidential information and/or privileged confidential advice. If you > receive this email by mistake, please do not read, print, transmit, store, > or in any other way use the email or its attachments. Instead, please > notify the addresser immediately and then permanently delete it. Thank you. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex0player@REDACTED Wed Apr 12 17:38:09 2017 From: alex0player@REDACTED (Alex S.) Date: Wed, 12 Apr 2017 18:38:09 +0300 Subject: [erlang-questions] CT: outputting lager debug messages In-Reply-To: References: Message-ID: <24D4B780-9F06-4326-AA00-19784323C231@gmail.com> You?re, iirc, supposed to call lager_common_test_backend:bounce/1, otherwise I?m not sure how it works. > 8 ???. 2017 ?., ? 8:48, Bjarne Wichmann Bagge Petersen ???????(?): > > I have a little issue where I would like to output lager debug-messages when running common tests. But I can't get it to work. > > In `init_pr_suite` I have > > application:load(lager), > application:set_env(lager, handlers, [{lager_console_backend, debug}, {lager_common_test_backend, debug}]), > > And in the test-code I have this little snippet: > > ct:log("ENV ~p", [application:get_all_env(lager)]), > lager:warning("Now you see me"), > lager:debug("Now you don't"), > > Which outputs: > > *** User 2017-04-08 07:36:42.089 *** > ENV [{crash_log_count,5}, > {crash_log_date,"$D0"}, > {colored,false}, > {crash_log_msg_size,65536}, > {colors,[{debug,"\e[0;38m"}, > {info,"\e[1;37m"}, > {notice,"\e[1;36m"}, > {warning,"\e[1;33m"}, > {error,"\e[1;31m"}, > {critical,"\e[1;35m"}, > {alert,"\e[1;44m"}, > {emergency,"\e[1;41m"}]}, > {included_applications,[]}, > {async_threshold,20}, > {crash_log_size,10485760}, > {error_logger_whitelist,[cth_readable_failonly]}, > {error_logger_redirect,true}, > {async_threshold_window,5}, > {crash_log,"log/crash.log"}, > {error_logger_hwm,50}, > {handlers,[{lager_console_backend,debug}, > {lager_common_test_backend,debug}]}] > > *** User 2017-04-08 07:36:42.090 *** > 07:36:42.089 [warning] Now you see me > > > Missing the debug-line. > > So what am I missing? Is `lager_common_test_backend` the wrong flag? Or is CT muting debug-messages anyway? > > > Best regards > > > Bjarne Wichmann Bagge Petersen > > Backend Developer, ShopGun > > E: bp@REDACTED > L: dk.linkedin.com/in/bjarnewp > W: shopgun.com > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark > > ShopGun, your helping hand in shopping, is available at shopgun.com and as the ShopGun app . > > This email is intended solely for the addressee and may contain confidential information and/or privileged confidential advice. If you receive this email by mistake, please do not read, print, transmit, store, or in any other way use the email or its attachments. Instead, please notify the addresser immediately and then permanently delete it. Thank you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacob01@REDACTED Wed Apr 12 17:42:09 2017 From: jacob01@REDACTED (Jacob) Date: Wed, 12 Apr 2017 17:42:09 +0200 Subject: [erlang-questions] CT: outputting lager debug messages In-Reply-To: References: Message-ID: <7a8f77b2-6613-160a-4b83-11b579200b1e@gmx.net> I am using this code in my test suite modules to catch lager debug output: init_per_suite(Cfg) -> lager_common_test_backend:bounce(debug), Cfg. No other changes are required. The lager logging messages are then included in the regular test suite output files. Best regards Jacob On 04/12/2017 05:32 PM, Luis Rasc?o wrote: > Maybe you're missing `lager:set_loglevel(lager_console_backend, debug).` ? > > On Sat, Apr 8, 2017 at 6:48 AM, Bjarne Wichmann Bagge Petersen > > wrote: > > I have a little issue where I would like to output lager > debug-messages when running common tests. But I can't get it to work. > > In `init_pr_suite` I have > > application:load(lager), > application:set_env(lager, handlers, [{lager_console_backend, > debug}, {lager_common_test_backend, debug}]), > > And in the test-code I have this little snippet: > > ct:log("ENV ~p", [application:get_all_env(lager)]), > lager:warning("Now you see me"), > lager:debug("Now you don't"), > > Which outputs: > > *** User 2017-04-08 07:36:42.089 *** > ENV [{crash_log_count,5}, > {crash_log_date,"$D0"}, > {colored,false}, > {crash_log_msg_size,65536}, > {colors,[{debug,"\e[0;38m"}, > {info,"\e[1;37m"}, > {notice,"\e[1;36m"}, > {warning,"\e[1;33m"}, > {error,"\e[1;31m"}, > {critical,"\e[1;35m"}, > {alert,"\e[1;44m"}, > {emergency,"\e[1;41m"}]}, > {included_applications,[]}, > {async_threshold,20}, > {crash_log_size,10485760}, > {error_logger_whitelist,[cth_readable_failonly]}, > {error_logger_redirect,true}, > {async_threshold_window,5}, > {crash_log,"log/crash.log"}, > {error_logger_hwm,50}, > {handlers,[{lager_console_backend,debug}, > {lager_common_test_backend,debug}]}] > > *** User 2017-04-08 07:36:42.090 *** > 07:36:42.089 [warning] Now you see me > > > Missing the debug-line. > > So what am I missing? Is `lager_common_test_backend` the wrong flag? > Or is CT muting debug-messages anyway? > > > Best regards > > > Bjarne Wichmann Bagge Petersen > > Backend Developer, ShopGun > > > E: bp@REDACTED > L: dk.linkedin.com/in/bjarnewp > W: shopgun.com > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, > Denmark > > ShopGun, your helping hand in shopping, is available at shopgun.com > and as the ShopGun app > . > > This email is intended solely for the addressee and may contain > confidential information and/or privileged confidential advice. If > you receive this email by mistake, please do not read, print, > transmit, store, or in any other way use the email or its > attachments. Instead, please notify the addresser immediately and > then permanently delete it. Thank you. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lawrence.ansell@REDACTED Thu Apr 13 12:03:13 2017 From: lawrence.ansell@REDACTED (Lawrence Ansell) Date: Thu, 13 Apr 2017 11:03:13 +0100 Subject: [erlang-questions] AN EUC17 Speakers Confirmed Message-ID: The Speakers for the EUC17 have just been announced, find out who will be speaking this year and register for your Early Bird ticket here: http://www.erlang-factory.com/euc2017 -------------- next part -------------- An HTML attachment was scrubbed... URL: From Maxim.Minin@REDACTED Thu Apr 13 15:27:40 2017 From: Maxim.Minin@REDACTED (Minin Maxim) Date: Thu, 13 Apr 2017 13:27:40 +0000 Subject: [erlang-questions] Is it a compiler bug? Message-ID: Hello, i'm confused about a very simple module like this: ----------------------------------------------------------------------- -module(sample). -record (a, {field1}). -export([bug/0]). bug() -> [ #a{field1 = 1} %% COMMA IS MISSING #a{field1 = 2}, #a{field1 = 3} ]. ----------------------------------------------------------------------- Why can it be compiled? Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V8.3 (abort with ^G) 1> c("sample.erl"). {ok,sample} 2> sample:bug(). [{a,2},{a,3}] And eshell do it to: 3> rr(sample). [a] 4> [#a{field1 = 1} #a{field1 = 2}, #a{field1 = 3}]. [#a{field1 = 2},#a{field1 = 3}] 5> Do I understand something wrong or is it a compiler bug? Thanks Maxim -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5536 bytes Desc: not available URL: From erlang.org@REDACTED Thu Apr 13 15:39:26 2017 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Thu, 13 Apr 2017 15:39:26 +0200 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: References: Message-ID: <20170413133926.GA25754@jarowit.net> On Thu, Apr 13, 2017 at 01:27:40PM +0000, Minin Maxim wrote: > i'm confused about a very simple module like this: > ----------------------------------------------------------------------- > -module(sample). > > -record (a, {field1}). > > -export([bug/0]). > > bug() -> > [ > #a{field1 = 1} %% COMMA IS MISSING > #a{field1 = 2}, > #a{field1 = 3} > ]. > ----------------------------------------------------------------------- > > Why can it be compiled? [...] > 2> sample:bug(). > [{a,2},{a,3}] Apparently the compiler takes the first #a{} record as value X, and then replaces in X field1. Let's complicate your example a little: #v+ -module(sample). -record (a, {field1, field2}). -export([bug/0]). bug() -> [ #a{field1 = 1, field2 = "foof"} %% COMMA IS MISSING #a{field1 = 2}, #a{field1 = 3} ]. #v- Note that with the missing comma you'll get a list of {a,1,"foof"}, {a,2,undefined}, and {a,3,undefined}, and with missing comma you'll get list of {a,2,"foof"}, {a,3,undefined}. Or a different view on your code: "#a{field1 = 1} #a{field1 = 2}" can be read as "(#a{field1 = 1})#a{field1 = 2}", or in two steps: "X = #a{field1 = 1}, X#a{field1 = 2}". -- Stanislaw Klekot From rtrlists@REDACTED Thu Apr 13 15:42:39 2017 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 13 Apr 2017 15:42:39 +0200 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: References: Message-ID: This is because the first record constructor returns as its value the record and the seemingly second constructor then functions as a record modification. It is parsed as (*#a{field1 = 1})#a{field1 = 2}* Cheers, Robby On 13 Apr 2017 15:27, "Minin Maxim" wrote: Hello, i'm confused about a very simple module like this: ----------------------------------------------------------------------- *-module(sample).* *-record (a, {field1}).* *-export([bug/0]).* *bug() ->* * [* * #a{field1 = 1} %% COMMA IS MISSING* * #a{field1 = 2},* * #a{field1 = 3}* * ].* ----------------------------------------------------------------------- Why can it be compiled? *Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]* *Eshell V8.3 (abort with ^G)* *1> c("sample.erl").* *{ok,sample}* *2> sample:bug().* *[{a,2},{a,3}]* And eshell do it to: *3> rr(sample).* *[a]* *4> [#a{field1 = 1} #a{field1 = 2}, #a{field1 = 3}].* *[#a{field1 = 2},#a{field1 = 3}]* *5>* Do I understand something wrong or is it a compiler bug? Thanks Maxim _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Maxim.Minin@REDACTED Thu Apr 13 16:04:15 2017 From: Maxim.Minin@REDACTED (Minin Maxim) Date: Thu, 13 Apr 2017 14:04:15 +0000 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: References: Message-ID: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> I'm so stupid today ? Thanks guys (Robert & Stanislaw) Cheers Maxim -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5536 bytes Desc: not available URL: From jesper.louis.andersen@REDACTED Thu Apr 13 16:14:47 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 13 Apr 2017 14:14:47 +0000 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> References: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> Message-ID: No, it is not stupidity in any way. People hit this now and again. It makes sense to solve the problem this way. I remember we considered the alternative, which is to reject the notion, but this requires some special-handling in the compiler and isn't clear-cut either. In short, regarding this as an invalid expression is to a certain extent possible, and certainly desirable. But we run into subtle problems when we want to reject it too, which is what complicates matters. I think it is healthy to challenge certain design decisions in Erlang, especially because dynamically typed languages tend to have corner cases in their semantics which are hard to handle in general. On Thu, Apr 13, 2017 at 4:04 PM Minin Maxim wrote: > I'm so stupid today J > > Thanks guys (Robert & Stanislaw) > > Cheers > > Maxim > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Thu Apr 13 16:43:14 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Thu, 13 Apr 2017 16:43:14 +0200 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: References: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> Message-ID: I can see how enforcing parentheses in map()map() syntax can be useful. A compiler warning would be awesome here. Also this is a nice thing for Elvis style checker to do, but when it comes to running Elvis, your program should already be correct ? so that is too late. In variable()map() syntax there is no confusion so that is fine. 2017-04-13 16:14 GMT+02:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > No, it is not stupidity in any way. > > People hit this now and again. It makes sense to solve the problem this > way. I remember we considered the alternative, which is to reject the > notion, but this requires some special-handling in the compiler and isn't > clear-cut either. > > In short, regarding this as an invalid expression is to a certain extent > possible, and certainly desirable. But we run into subtle problems when we > want to reject it too, which is what complicates matters. > > I think it is healthy to challenge certain design decisions in Erlang, > especially because dynamically typed languages tend to have corner cases in > their semantics which are hard to handle in general. > > On Thu, Apr 13, 2017 at 4:04 PM Minin Maxim > wrote: > >> I'm so stupid today J >> >> Thanks guys (Robert & Stanislaw) >> >> Cheers >> >> Maxim >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri Apr 14 02:14:23 2017 From: rvirding@REDACTED (Robert Virding) Date: Fri, 14 Apr 2017 02:14:23 +0200 Subject: [erlang-questions] Is it a compiler bug? In-Reply-To: References: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> Message-ID: IIRC originally parentheses were required around the first was required which would have caught this case. The requirement was then removed, but I don't know why. Perhaps to save the extra 2 chars. :-) On 13 April 2017 at 16:43, Dmytro Lytovchenko wrote: > I can see how enforcing parentheses in map()map() syntax can be useful. > A compiler warning would be awesome here. Also this is a nice thing for > Elvis style checker to do, but when it comes to running Elvis, your program > should already be correct ? so that is too late. > > In variable()map() syntax there is no confusion so that is fine. > > 2017-04-13 16:14 GMT+02:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > >> No, it is not stupidity in any way. >> >> People hit this now and again. It makes sense to solve the problem this >> way. I remember we considered the alternative, which is to reject the >> notion, but this requires some special-handling in the compiler and isn't >> clear-cut either. >> >> In short, regarding this as an invalid expression is to a certain extent >> possible, and certainly desirable. But we run into subtle problems when we >> want to reject it too, which is what complicates matters. >> >> I think it is healthy to challenge certain design decisions in Erlang, >> especially because dynamically typed languages tend to have corner cases in >> their semantics which are hard to handle in general. >> >> On Thu, Apr 13, 2017 at 4:04 PM Minin Maxim >> wrote: >> >>> I'm so stupid today J >>> >>> Thanks guys (Robert & Stanislaw) >>> >>> Cheers >>> >>> Maxim >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri Apr 14 02:15:46 2017 From: rvirding@REDACTED (Robert Virding) Date: Fri, 14 Apr 2017 02:15:46 +0200 Subject: [erlang-questions] luerl In-Reply-To: <8855f383-f388-255c-4738-a25e97d6a3ea@dergraf.org> References: <8855f383-f388-255c-4738-a25e97d6a3ea@dergraf.org> Message-ID: Mail me about it so we can discuss it. rvirding@REDACTED On 10 April 2017 at 21:54, Andre Graf wrote: > Hi Max, > > the VerneMQ broker is using Luerl. We would certainly be interested in > better stacktraces for Luerl. Depending on the effort, we could > contribute something. > > Cheers, > Andre > > > On 04/10/2017 09:10 PM, Max Lapshin wrote: > > Hi. > > > > We are using luerl in production and we have some issues with it like > > lack of repl or stacktraces. > > > > I wonder if there are any other people that are using it and ready to > > commit to public repo or people who are ready to implement features > above. > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri Apr 14 02:17:07 2017 From: rvirding@REDACTED (Robert Virding) Date: Fri, 14 Apr 2017 02:17:07 +0200 Subject: [erlang-questions] How to force init/1 to start at application start In-Reply-To: <295444666.2184116.1491566768068@mail.yahoo.com> References: <295444666.2184116.1491566768068.ref@mail.yahoo.com> <295444666.2184116.1491566768068@mail.yahoo.com> Message-ID: Sorry missing something here, and init/1 function in what? On 7 April 2017 at 14:06, Dmitry wrote: > Hello > > I have an init/1 function which does not start at application startup. > > > > > init([Ref, Socket, Transport, Opts]) -> > > MyTable = ets:new(myphone, [set]), > > MyTable2 = ets:new(socket1, [set]), > lager:start(), > > Timer = erlang:send_after(1210000, self(), check), > > {ok, #state{ref=Ref, socket=Socket, transport=Transport, opts= Opts, > dict = MyTable, dict2 = MyTable2, timer = Timer}, 0}. > > > > How to force it to start at an application startup and only once.? > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From botanyzh@REDACTED Fri Apr 14 04:24:39 2017 From: botanyzh@REDACTED (=?gb2312?B?1dQguro=?=) Date: Fri, 14 Apr 2017 02:24:39 +0000 Subject: [erlang-questions] =?gb2312?b?tPC4tDogIElzIGl0IGEgY29tcGlsZXIg?= =?gb2312?b?YnVnPw==?= In-Reply-To: References: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> , Message-ID: I still don?t get the sense , What is the need for such weird appending use? ??? Windows 10 ????? ???: Robert Virding ????: 2017?4?14? 8:15 ???: Dmytro Lytovchenko ??: Erlang Questions; Stanislaw Klekot ??: Re: [erlang-questions] Is it a compiler bug? IIRC originally parentheses were required around the first was required which would have caught this case. The requirement was then removed, but I don't know why. Perhaps to save the extra 2 chars. :-) On 13 April 2017 at 16:43, Dmytro Lytovchenko > wrote: I can see how enforcing parentheses in map()map() syntax can be useful. A compiler warning would be awesome here. Also this is a nice thing for Elvis style checker to do, but when it comes to running Elvis, your program should already be correct ? so that is too late. In variable()map() syntax there is no confusion so that is fine. 2017-04-13 16:14 GMT+02:00 Jesper Louis Andersen >: No, it is not stupidity in any way. People hit this now and again. It makes sense to solve the problem this way. I remember we considered the alternative, which is to reject the notion, but this requires some special-handling in the compiler and isn't clear-cut either. In short, regarding this as an invalid expression is to a certain extent possible, and certainly desirable. But we run into subtle problems when we want to reject it too, which is what complicates matters. I think it is healthy to challenge certain design decisions in Erlang, especially because dynamically typed languages tend to have corner cases in their semantics which are hard to handle in general. On Thu, Apr 13, 2017 at 4:04 PM Minin Maxim > wrote: I'm so stupid today :) Thanks guys (Robert & Stanislaw) Cheers Maxim _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Fri Apr 14 14:12:54 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 14 Apr 2017 14:12:54 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang Message-ID: Hi everyone, I would like to propose to remove "tuple dispatches" from Erlang. The tuple dispatch is the ability to invoke a function or call erlang:apply/3 on a tuple as first argument: Eshell V9.0 (abort with ^G) 1> Var = dict:new(). {dict,0,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} 2> Var:size(). 0 This behaviour is considered by most in the community to be undesired and confusing, as it obfuscates the meaning of the code and adds indirection. I have also heard this behaviour made it harder to add some optimizations to the VM. I would love if someone more knowledgeable on the area could confirm or deny this. If true, it is also a strong argument to remove such behaviour. Another reason for removing it is that the behaviour can be implemented as needed by adding is_tuple/1 checks to the code or more programmatically by using a parse transforms (see note 1 at the bottom for a limitation though). Therefore those who need the behaviour can include it only when necessary and we don't impose it as a semantics to the whole language (and ecosystem). I can think of two strategies for removing the behaviour: 1. Clean-cut: the code responsible for tuple dispatching will be completely removed from the VM and a parse transform will be made available. The parse transform could be part of Erlang/OTP or a separate repository. This change is backwards incompatible at the BEAM level. Code that relies on tuple dispatch without the parse transform on OTP 19 will not work on OTP 20. However, the parse transform should work with any OTP version, so if the parse transform is used during compilation, the code is guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. 2. New byte codes: if we don't want to break backwards compatibility at the BEAM level, I believe our only option is to introduce new bytecodes and a new apply BIF. Usage of the old BIFs and bytecode could emit warnings while we phase them out. A compiler option or parse transform should still be made available if a developer relying on those features wants their code to run without warnings. Please let me know if there are other options available, I will be glad to send patches and implement the required parse-transforms if this is accepted by the OTP team. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D Note 1. A parse-transform would be unable to make the following code work in the same way as today: erlang:apply(erlang, apply, [dict:new(), size, []]) Although I would consider it highly unlikely to exist so it should not be a point of contention. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Apr 14 14:19:55 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 14 Apr 2017 14:19:55 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: Message-ID: <11f1fc03-17ba-7d16-0055-c83c486c25be@ninenines.eu> I'm all against using this feature personally, though removing is a bit harsh considering there's still a lot of code relying on it. Something is unclear to me though. Why do YOU want to remove it? Will it improve something for you? On 04/14/2017 02:12 PM, Jos? Valim wrote: > Hi everyone, > > I would like to propose to remove "tuple dispatches" from Erlang. > > The tuple dispatch is the ability to invoke a function or call > erlang:apply/3 on a tuple as first argument: > > Eshell V9.0 (abort with ^G) > 1> Var = dict:new(). > {dict,0,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} > 2> Var:size(). > 0 > > > This behaviour is considered by most in the community to be undesired > and confusing, as it obfuscates the meaning of the code and adds > indirection. > > I have also heard this behaviour made it harder to add some > optimizations to the VM. I would love if someone more knowledgeable on > the area could confirm or deny this. If true, it is also a strong > argument to remove such behaviour. > > Another reason for removing it is that the behaviour can be implemented > as needed by adding is_tuple/1 checks to the code or more > programmatically by using a parse transforms (see note 1 at the bottom > for a limitation though). Therefore those who need the behaviour can > include it only when necessary and we don't impose it as a semantics to > the whole language (and ecosystem). > > I can think of two strategies for removing the behaviour: > > 1. Clean-cut: the code responsible for tuple dispatching will be > completely removed from the VM and a parse transform will be made > available. The parse transform could be part of Erlang/OTP or a separate > repository. This change is backwards incompatible at the BEAM level. > Code that relies on tuple dispatch without the parse transform on OTP 19 > will not work on OTP 20. However, the parse transform should work with > any OTP version, so if the parse transform is used during compilation, > the code is guaranteed to work on OTP 19 and earlier as well as on OTP > 20 onwards. > > 2. New byte codes: if we don't want to break backwards compatibility at > the BEAM level, I believe our only option is to introduce new bytecodes > and a new apply BIF. Usage of the old BIFs and bytecode could emit > warnings while we phase them out. A compiler option or parse transform > should still be made available if a developer relying on those features > wants their code to run without warnings. > > Please let me know if there are other options available, > > I will be glad to send patches and implement the required > parse-transforms if this is accepted by the OTP team. > > * > Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > > Note 1. A parse-transform would be unable to make the following code > work in the same way as today: > > erlang:apply(erlang, apply, [dict:new(), size, []]) > > > Although I would consider it highly unlikely to exist so it should not > be a point of contention. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From tony@REDACTED Fri Apr 14 14:33:31 2017 From: tony@REDACTED (Tony Rogvall) Date: Fri, 14 Apr 2017 14:33:31 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: Message-ID: <650BD956-B3DD-4F96-9A5A-F8E8F47D4996@rogvall.se> > On 14 apr 2017, at 14:12, Jos? Valim wrote: > > Hi everyone, > > I would like to propose to remove "tuple dispatches" from Erlang. > > The tuple dispatch is the ability to invoke a function or call erlang:apply/3 on a tuple as first argument: > > Eshell V9.0 (abort with ^G) > 1> Var = dict:new(). > {dict,0,16,16,8,80,48, > {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} > 2> Var:size(). > 0 > I think it is calling erlang:apply/2, the above code is called like > apply({Var,size}, []). 0 This is, interestingly enough, equivalent to > erlang:apply(element(1,Var), size, [Var]). 0 But maybe that was what you meant? I think this once was part of the parametrized module stuff, this is now gone. Utterly confusing an strange. /Tony > This behaviour is considered by most in the community to be undesired and confusing, as it obfuscates the meaning of the code and adds indirection. > > I have also heard this behaviour made it harder to add some optimizations to the VM. I would love if someone more knowledgeable on the area could confirm or deny this. If true, it is also a strong argument to remove such behaviour. > > Another reason for removing it is that the behaviour can be implemented as needed by adding is_tuple/1 checks to the code or more programmatically by using a parse transforms (see note 1 at the bottom for a limitation though). Therefore those who need the behaviour can include it only when necessary and we don't impose it as a semantics to the whole language (and ecosystem). > > I can think of two strategies for removing the behaviour: > > 1. Clean-cut: the code responsible for tuple dispatching will be completely removed from the VM and a parse transform will be made available. The parse transform could be part of Erlang/OTP or a separate repository. This change is backwards incompatible at the BEAM level. Code that relies on tuple dispatch without the parse transform on OTP 19 will not work on OTP 20. However, the parse transform should work with any OTP version, so if the parse transform is used during compilation, the code is guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. > > 2. New byte codes: if we don't want to break backwards compatibility at the BEAM level, I believe our only option is to introduce new bytecodes and a new apply BIF. Usage of the old BIFs and bytecode could emit warnings while we phase them out. A compiler option or parse transform should still be made available if a developer relying on those features wants their code to run without warnings. > > Please let me know if there are other options available, > > I will be glad to send patches and implement the required parse-transforms if this is accepted by the OTP team. > > > Jos? Valim > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > > Note 1. A parse-transform would be unable to make the following code work in the same way as today: > > erlang:apply(erlang, apply, [dict:new(), size, []]) > > Although I would consider it highly unlikely to exist so it should not be a point of contention. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jose.valim@REDACTED Fri Apr 14 14:40:46 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 14 Apr 2017 12:40:46 +0000 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: <11f1fc03-17ba-7d16-0055-c83c486c25be@ninenines.eu> References: <11f1fc03-17ba-7d16-0055-c83c486c25be@ninenines.eu> Message-ID: Thanks for the input Lo?c. When you say removing it is harsh, do you mean removing it immediately in favor of a parse transform (implementation 1) or are you also against implementation 2 which would be backwards compatible at BEAM level and allow the feature to be slowly phased out through the years? Regarding my personal reasons, they have all been listed in the original email. It is a style that I believe should be discouraged, as it can be implemented manually when desired, especially if it is in the way of further improvements at the VM level (pending confirmation). Elixir and Erlang should be equally affected by the proposed changes (as Elixir targets the Erlang Abstract Format). -- *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Apr 14 15:00:17 2017 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 14 Apr 2017 09:00:17 -0400 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: Message-ID: <20170414130016.GB61497@ferdmbp.local> On 04/14, Jos? Valim wrote: >This behaviour is considered by most in the community to be undesired >and confusing, as it obfuscates the meaning of the code and adds >indirection. > >Please let me know if there are other options available, > >I will be glad to send patches and implement the required parse-transforms >if this is accepted by the OTP team. > One thing to consider is that this code is a remnant of parametrized modules, which have been removed back in 2012. See https://web-beta.archive.org/web/20121019180935/http://www.erlang.org/news/35 > To allow for source code compatibility, the board decides to only > remove the syntactic support for parameterized modules. This means: > > * Apply with tuple modules will be retained, supported and properly documented. > > [...] > > * The parser will accept the syntax for implementation of > parameterized modules [...] > > * A parse_transform handling source code implementing a parameterized > module will be published in the public domain, free to use and > include in any package needing it. It will however not be part of OTP. > As it uses supported and documented functionality, it will however > be expected to work for forthcoming releases as well, or at least > to be possible to adopt to forthcoming releases. This is to say the OTP team made a commitment to supporting tuple calls as a way to remove parametrized modules and keep the code working. Removing tuple calls must at the very least ensure a seamless transition to keep working with the parametrized module parse transforms, and moreover have the guarantee that it will "be expected to work for forthcoming releases" because it "uses supported and documented functionality". Regards, Fred. From v@REDACTED Fri Apr 14 15:04:50 2017 From: v@REDACTED (Valentin Micic) Date: Fri, 14 Apr 2017 15:04:50 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: <650BD956-B3DD-4F96-9A5A-F8E8F47D4996@rogvall.se> References: <650BD956-B3DD-4F96-9A5A-F8E8F47D4996@rogvall.se> Message-ID: <4CC5CEE0-C0CE-41D1-A3A7-58F78C474FB2@pharos-avantgard.com> I wonder if Fun implementation (e.g. F=fun(..) -> end) relies on "tuple dispatches". V/ On 14 Apr 2017, at 2:33 PM, Tony Rogvall wrote: > >> On 14 apr 2017, at 14:12, Jos? Valim wrote: >> >> Hi everyone, >> >> I would like to propose to remove "tuple dispatches" from Erlang. >> >> The tuple dispatch is the ability to invoke a function or call erlang:apply/3 on a tuple as first argument: >> >> Eshell V9.0 (abort with ^G) >> 1> Var = dict:new(). >> {dict,0,16,16,8,80,48, >> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} >> 2> Var:size(). >> 0 >> > > I think it is calling erlang:apply/2, the above code is called like > >> apply({Var,size}, []). > 0 > > This is, interestingly enough, equivalent to > >> erlang:apply(element(1,Var), size, [Var]). > 0 > > But maybe that was what you meant? > > I think this once was part of the parametrized module stuff, this is now gone. > > Utterly confusing an strange. > > /Tony > >> This behaviour is considered by most in the community to be undesired and confusing, as it obfuscates the meaning of the code and adds indirection. >> >> I have also heard this behaviour made it harder to add some optimizations to the VM. I would love if someone more knowledgeable on the area could confirm or deny this. If true, it is also a strong argument to remove such behaviour. >> >> Another reason for removing it is that the behaviour can be implemented as needed by adding is_tuple/1 checks to the code or more programmatically by using a parse transforms (see note 1 at the bottom for a limitation though). Therefore those who need the behaviour can include it only when necessary and we don't impose it as a semantics to the whole language (and ecosystem). >> >> I can think of two strategies for removing the behaviour: >> >> 1. Clean-cut: the code responsible for tuple dispatching will be completely removed from the VM and a parse transform will be made available. The parse transform could be part of Erlang/OTP or a separate repository. This change is backwards incompatible at the BEAM level. Code that relies on tuple dispatch without the parse transform on OTP 19 will not work on OTP 20. However, the parse transform should work with any OTP version, so if the parse transform is used during compilation, the code is guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. >> >> 2. New byte codes: if we don't want to break backwards compatibility at the BEAM level, I believe our only option is to introduce new bytecodes and a new apply BIF. Usage of the old BIFs and bytecode could emit warnings while we phase them out. A compiler option or parse transform should still be made available if a developer relying on those features wants their code to run without warnings. >> >> Please let me know if there are other options available, >> >> I will be glad to send patches and implement the required parse-transforms if this is accepted by the OTP team. >> >> >> Jos? Valim >> www.plataformatec.com.br >> Skype: jv.ptec >> Founder and Director of R&D >> >> >> Note 1. A parse-transform would be unable to make the following code work in the same way as today: >> >> erlang:apply(erlang, apply, [dict:new(), size, []]) >> >> Although I would consider it highly unlikely to exist so it should not be a point of contention. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Fri Apr 14 19:32:25 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 14 Apr 2017 19:32:25 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: <4CC5CEE0-C0CE-41D1-A3A7-58F78C474FB2@pharos-avantgard.com> References: <650BD956-B3DD-4F96-9A5A-F8E8F47D4996@rogvall.se> <4CC5CEE0-C0CE-41D1-A3A7-58F78C474FB2@pharos-avantgard.com> Message-ID: The original prototype implementation of funs did indeed use tuples as the representation. Later, funs became a proper built-in data type. The intention was of course for abstract modules to go the same path. /Richard 2017-04-14 15:04 GMT+02:00 Valentin Micic : > I wonder if Fun implementation (e.g. F=fun(..) -> end) relies on "tuple > dispatches". > > V/ > > On 14 Apr 2017, at 2:33 PM, Tony Rogvall wrote: > > > > >> On 14 apr 2017, at 14:12, Jos? Valim > wrote: > >> > >> Hi everyone, > >> > >> I would like to propose to remove "tuple dispatches" from Erlang. > >> > >> The tuple dispatch is the ability to invoke a function or call > erlang:apply/3 on a tuple as first argument: > >> > >> Eshell V9.0 (abort with ^G) > >> 1> Var = dict:new(). > >> {dict,0,16,16,8,80,48, > >> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, > >> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} > >> 2> Var:size(). > >> 0 > >> > > > > I think it is calling erlang:apply/2, the above code is called like > > > >> apply({Var,size}, []). > > 0 > > > > This is, interestingly enough, equivalent to > > > >> erlang:apply(element(1,Var), size, [Var]). > > 0 > > > > But maybe that was what you meant? > > > > I think this once was part of the parametrized module stuff, this is now > gone. > > > > Utterly confusing an strange. > > > > /Tony > > > >> This behaviour is considered by most in the community to be undesired > and confusing, as it obfuscates the meaning of the code and adds > indirection. > >> > >> I have also heard this behaviour made it harder to add some > optimizations to the VM. I would love if someone more knowledgeable on the > area could confirm or deny this. If true, it is also a strong argument to > remove such behaviour. > >> > >> Another reason for removing it is that the behaviour can be implemented > as needed by adding is_tuple/1 checks to the code or more programmatically > by using a parse transforms (see note 1 at the bottom for a limitation > though). Therefore those who need the behaviour can include it only when > necessary and we don't impose it as a semantics to the whole language (and > ecosystem). > >> > >> I can think of two strategies for removing the behaviour: > >> > >> 1. Clean-cut: the code responsible for tuple dispatching will be > completely removed from the VM and a parse transform will be made > available. The parse transform could be part of Erlang/OTP or a separate > repository. This change is backwards incompatible at the BEAM level. Code > that relies on tuple dispatch without the parse transform on OTP 19 will > not work on OTP 20. However, the parse transform should work with any OTP > version, so if the parse transform is used during compilation, the code is > guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. > >> > >> 2. New byte codes: if we don't want to break backwards compatibility at > the BEAM level, I believe our only option is to introduce new bytecodes and > a new apply BIF. Usage of the old BIFs and bytecode could emit warnings > while we phase them out. A compiler option or parse transform should still > be made available if a developer relying on those features wants their code > to run without warnings. > >> > >> Please let me know if there are other options available, > >> > >> I will be glad to send patches and implement the required > parse-transforms if this is accepted by the OTP team. > >> > >> > >> Jos? Valim > >> www.plataformatec.com.br > >> Skype: jv.ptec > >> Founder and Director of R&D > >> > >> > >> Note 1. A parse-transform would be unable to make the following code > work in the same way as today: > >> > >> erlang:apply(erlang, apply, [dict:new(), size, []]) > >> > >> Although I would consider it highly unlikely to exist so it should not > be a point of contention. > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sat Apr 15 20:13:22 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 15 Apr 2017 20:13:22 +0200 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching Message-ID: <22770.25282.691552.422821@gargle.gargle.HOWL> Suppose a frequently called function returns a positive integer (always a fixnum) and a boolean "success" or "failure" indicator (even in the failure case the return value is significant and will be used). Which is better from a performance perspective: 1. Tag the return value, {success, X} or {failure, Y}, and have callers pattern-match on that or, 2. Indicate failure by negating Y, and have callers match on X when X > 0 -> success case; MinusY -> failure case % MinusY =< 0 is implicit Option 2 avoids the consing overheads of option 1, but I'm worried that the X > 0 guard may be less optimized than a plain structural pattern-match. The last time I checked, term-comparisons would check inline for identity, and otherwise call the general cmp() function which would then do a lot of type-casing. In my pure Erlang code I'm using option 1, but I'm reimplementing the function as a NIF, and would like to avoid the consing overheads is that's indeed better. /Mikael From donpedrothird@REDACTED Sun Apr 16 01:53:55 2017 From: donpedrothird@REDACTED (John Doe) Date: Sun, 16 Apr 2017 02:53:55 +0300 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching In-Reply-To: <22770.25282.691552.422821@gargle.gargle.HOWL> References: <22770.25282.691552.422821@gargle.gargle.HOWL> Message-ID: Just've checked, the first one is approx 2 times faster on 19.1.6 1> L = lists:seq(1, 1000000). 2> N = -1. 3> T = {error, -1}. 4> F1 = fun() -> lists:foreach(fun(_) -> case T of {success, Z} -> Z; {error, Z} -> Z end end, L) end. 5> F2 = fun() -> lists:foreach(fun(_) -> case N of X when X > 0 -> X; _ -> N end end, L) end. 6> timer:tc(F1). {1089950,ok} 7> 7> timer:tc(F2). {1870456,ok} 2017-04-15 21:13 GMT+03:00 Mikael Pettersson : > Suppose a frequently called function returns a positive integer (always a > fixnum) > and a boolean "success" or "failure" indicator (even in the failure case > the return > value is significant and will be used). Which is better from a performance > perspective: > > 1. Tag the return value, {success, X} or {failure, Y}, and have callers > pattern-match > on that > > or, > > 2. Indicate failure by negating Y, and have callers match on > > X when X > 0 -> success case; > MinusY -> failure case % MinusY =< 0 is implicit > > Option 2 avoids the consing overheads of option 1, but I'm worried that > the X > 0 > guard may be less optimized than a plain structural pattern-match. The > last time > I checked, term-comparisons would check inline for identity, and otherwise > call > the general cmp() function which would then do a lot of type-casing. > > In my pure Erlang code I'm using option 1, but I'm reimplementing the > function > as a NIF, and would like to avoid the consing overheads is that's indeed > better. > > > /Mikael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Apr 16 11:50:51 2017 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 16 Apr 2017 11:50:51 +0200 Subject: [erlang-questions] OpenID github login don't work on bugs.erlang.org Message-ID: Hi, looks like I cannot log on bugs.erlang.org with Github account via OpenId anymore : "OpenID Failed There was a problem during authentication. Please refer to logs for an additional info." Can someone else confirm it is a bug ? Regards From jose.valim@REDACTED Sun Apr 16 12:58:53 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sun, 16 Apr 2017 12:58:53 +0200 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching In-Reply-To: References: <22770.25282.691552.422821@gargle.gargle.HOWL> Message-ID: John, did you measure in the shell? Code in the shell is evaluated and it won't give the same result as if the code would be compiled as part of a module. Ideally you want to define a module, compile it, and then execute a function that runs the benchmarking. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Sun, Apr 16, 2017 at 1:53 AM, John Doe wrote: > Just've checked, the first one is approx 2 times faster on 19.1.6 > 1> L = lists:seq(1, 1000000). > 2> N = -1. > 3> T = {error, -1}. > 4> F1 = fun() -> lists:foreach(fun(_) -> case T of {success, Z} -> Z; > {error, Z} -> Z end end, L) end. > 5> F2 = fun() -> lists:foreach(fun(_) -> case N of X when X > 0 -> X; _ -> > N end end, L) end. > 6> timer:tc(F1). > {1089950,ok} > 7> > 7> timer:tc(F2). > {1870456,ok} > > 2017-04-15 21:13 GMT+03:00 Mikael Pettersson : > >> Suppose a frequently called function returns a positive integer (always a >> fixnum) >> and a boolean "success" or "failure" indicator (even in the failure case >> the return >> value is significant and will be used). Which is better from a >> performance >> perspective: >> >> 1. Tag the return value, {success, X} or {failure, Y}, and have callers >> pattern-match >> on that >> >> or, >> >> 2. Indicate failure by negating Y, and have callers match on >> >> X when X > 0 -> success case; >> MinusY -> failure case % MinusY =< 0 is implicit >> >> Option 2 avoids the consing overheads of option 1, but I'm worried that >> the X > 0 >> guard may be less optimized than a plain structural pattern-match. The >> last time >> I checked, term-comparisons would check inline for identity, and >> otherwise call >> the general cmp() function which would then do a lot of type-casing. >> >> In my pure Erlang code I'm using option 1, but I'm reimplementing the >> function >> as a NIF, and would like to avoid the consing overheads is that's indeed >> better. >> >> >> /Mikael >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sun Apr 16 13:40:17 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sun, 16 Apr 2017 13:40:17 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: <650BD956-B3DD-4F96-9A5A-F8E8F47D4996@rogvall.se> <4CC5CEE0-C0CE-41D1-A3A7-58F78C474FB2@pharos-avantgard.com> Message-ID: Thanks everyone for the feedback. It is clear now that backwards compatibility must be considered. I will work on a new proposal that focus on those aspects. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Fri, Apr 14, 2017 at 7:32 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > The original prototype implementation of funs did indeed use tuples as the > representation. Later, funs became a proper built-in data type. The > intention was of course for abstract modules to go the same path. > > > /Richard > > 2017-04-14 15:04 GMT+02:00 Valentin Micic : > >> I wonder if Fun implementation (e.g. F=fun(..) -> end) relies on "tuple >> dispatches". >> >> V/ >> >> On 14 Apr 2017, at 2:33 PM, Tony Rogvall wrote: >> >> > >> >> On 14 apr 2017, at 14:12, Jos? Valim >> wrote: >> >> >> >> Hi everyone, >> >> >> >> I would like to propose to remove "tuple dispatches" from Erlang. >> >> >> >> The tuple dispatch is the ability to invoke a function or call >> erlang:apply/3 on a tuple as first argument: >> >> >> >> Eshell V9.0 (abort with ^G) >> >> 1> Var = dict:new(). >> >> {dict,0,16,16,8,80,48, >> >> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >> >> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} >> >> 2> Var:size(). >> >> 0 >> >> >> > >> > I think it is calling erlang:apply/2, the above code is called like >> > >> >> apply({Var,size}, []). >> > 0 >> > >> > This is, interestingly enough, equivalent to >> > >> >> erlang:apply(element(1,Var), size, [Var]). >> > 0 >> > >> > But maybe that was what you meant? >> > >> > I think this once was part of the parametrized module stuff, this is >> now gone. >> > >> > Utterly confusing an strange. >> > >> > /Tony >> > >> >> This behaviour is considered by most in the community to be undesired >> and confusing, as it obfuscates the meaning of the code and adds >> indirection. >> >> >> >> I have also heard this behaviour made it harder to add some >> optimizations to the VM. I would love if someone more knowledgeable on the >> area could confirm or deny this. If true, it is also a strong argument to >> remove such behaviour. >> >> >> >> Another reason for removing it is that the behaviour can be >> implemented as needed by adding is_tuple/1 checks to the code or more >> programmatically by using a parse transforms (see note 1 at the bottom for >> a limitation though). Therefore those who need the behaviour can include it >> only when necessary and we don't impose it as a semantics to the whole >> language (and ecosystem). >> >> >> >> I can think of two strategies for removing the behaviour: >> >> >> >> 1. Clean-cut: the code responsible for tuple dispatching will be >> completely removed from the VM and a parse transform will be made >> available. The parse transform could be part of Erlang/OTP or a separate >> repository. This change is backwards incompatible at the BEAM level. Code >> that relies on tuple dispatch without the parse transform on OTP 19 will >> not work on OTP 20. However, the parse transform should work with any OTP >> version, so if the parse transform is used during compilation, the code is >> guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. >> >> >> >> 2. New byte codes: if we don't want to break backwards compatibility >> at the BEAM level, I believe our only option is to introduce new bytecodes >> and a new apply BIF. Usage of the old BIFs and bytecode could emit warnings >> while we phase them out. A compiler option or parse transform should still >> be made available if a developer relying on those features wants their code >> to run without warnings. >> >> >> >> Please let me know if there are other options available, >> >> >> >> I will be glad to send patches and implement the required >> parse-transforms if this is accepted by the OTP team. >> >> >> >> >> >> Jos? Valim >> >> www.plataformatec.com.br >> >> Skype: jv.ptec >> >> Founder and Director of R&D >> >> >> >> >> >> Note 1. A parse-transform would be unable to make the following code >> work in the same way as today: >> >> >> >> erlang:apply(erlang, apply, [dict:new(), size, []]) >> >> >> >> Although I would consider it highly unlikely to exist so it should not >> be a point of contention. >> >> >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sun Apr 16 15:41:04 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 16 Apr 2017 15:41:04 +0200 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching In-Reply-To: <22770.25282.691552.422821@gargle.gargle.HOWL> References: <22770.25282.691552.422821@gargle.gargle.HOWL> Message-ID: <22771.29808.891451.257658@gargle.gargle.HOWL> Mikael Pettersson writes: > Suppose a frequently called function returns a positive integer (always a fixnum) > and a boolean "success" or "failure" indicator (even in the failure case the return > value is significant and will be used). Which is better from a performance > perspective: > > 1. Tag the return value, {success, X} or {failure, Y}, and have callers pattern-match > on that > > or, > > 2. Indicate failure by negating Y, and have callers match on > > X when X > 0 -> success case; > MinusY -> failure case % MinusY =< 0 is implicit > > Option 2 avoids the consing overheads of option 1, but I'm worried that the X > 0 > guard may be less optimized than a plain structural pattern-match. The last time > I checked, term-comparisons would check inline for identity, and otherwise call > the general cmp() function which would then do a lot of type-casing. > > In my pure Erlang code I'm using option 1, but I'm reimplementing the function > as a NIF, and would like to avoid the consing overheads is that's indeed better. I put together a micro-benchmark module (appended below), and have the following observations: - Option 2 above is about 15-25% faster than option 1 above. - Adding an is_integer/1 guard before the X > 0 in option 2 makes the code slower. - A major cost in option 1 is having to do case analysis on two atoms (because the runtime representation of an atom isn't known until the code is loaded into a running VM). However, replacing the atoms with the fixnums 0 and 1 makes the code much slower. - The best version I've found so far (function 'k' in the bechmark module) eliminates the tags, represents one case by wrapping the integer in a cons cell [X | []], and the other case as the plain integer. [X | []] is slightly better than {X} since the latter requires an additional load and test in the pattern-matching path. On my desktop box (OTP-19.3, Ivy Bridge @ 3.5 GHz), typical output from the benchmark is: > test:all(100000000). [{f,{2564769,2}}, {g,{1806006,2}}, {h,{1999038,2}}, {i,{3295790,2}}, {j,{1779385,2}}, {k,{1504119,2}}, {l,{1526827,2}}] f is option 1 above, g is option 2. h is g with added is_integer/1 check. i is f but with integer tags not atoms. j wraps the two cases as [X|[]] and {Y}. k is j but with Y left unwrapped. l is j but swaps the order and checks with is_integer/1 before matching on [Y|_]. /Mikael ==snip== -module(test). -export([all/1, f/1, g/1, h/1, i/1, j/1, k/1, l/1]). all(N) -> L = [ {f, fun f/1} , {g, fun g/1} , {h, fun h/1} , {i, fun i/1} , {j, fun j/1} , {k, fun k/1} , {l, fun l/1} ], [{A, F(N)} || {A, F} <- L]. f(N) -> timer:tc(fun() -> f2(N, [], []) end). g(N) -> timer:tc(fun() -> g2(N, [], []) end). h(N) -> timer:tc(fun() -> h2(N, [], []) end). i(N) -> timer:tc(fun() -> i2(N, [], []) end). j(N) -> timer:tc(fun() -> j2(N, [], []) end). k(N) -> timer:tc(fun() -> k2(N, [], []) end). l(N) -> timer:tc(fun() -> l2(N, [], []) end). %% f: tag with {success, X} or {failure, Y} f2(0, X, _Y) -> X; f2(N, X, Y) -> case f3(N) of {success, A} -> f2(N - 1, A, Y); {failure, B} -> f2(N - 1, X, B) end. f3(N) -> case N band 1 of 0 -> {success, N}; _ -> {failure, N} end. %% g: indicate success or failure via the sign g2(0, X, _Y) -> X; g2(N, X, Y) -> case g3(N) of A when A > 0 -> g2(N - 1, A, Y); MinusB -> g2(N - 1, X, -MinusB) end. g3(N) -> case N band 1 of 0 -> N; _ -> -N end. %% h: like g, but check is_integer before checking > 0 h2(0, X, _Y) -> X; h2(N, X, Y) -> case g3(N) of A when is_integer(A), A > 0 -> h2(N - 1, A, Y); MinusB -> h2(N - 1, X, -MinusB) end. %% i: like f, but tag with integers 0 and 1 instead i2(0, X, _Y) -> X; i2(N, X, Y) -> case i3(N) of {0, A} -> i2(N - 1, A, Y); {1, B} -> i2(N - 1, X, B) end. i3(N) -> case N band 1 of 0 -> {0, N}; _ -> {1, N} end. %% j: like f but drop the tags, instead tag as [X|[]] for success or {Y} for failure j2(0, X, _Y) -> X; j2(N, X, Y) -> case j3(N) of [A | _] -> j2(N - 1, A, Y); {B} -> j2(N - 1, X, B) end. j3(N) -> case N band 1 of 0 -> [N | []]; _ -> {N} end. %% k: like j, but only tag the success case not the failure case k2(0, X, _Y) -> X; k2(N, X, Y) -> case k3(N) of [A | _] -> k2(N - 1, A, Y); B -> k2(N - 1, X, B) end. k3(N) -> case N band 1 of 0 -> [N | []]; _ -> N end. %% l: like k but the other way around: tag the failure case [Y|[]], %% check for success with is_integer/1 l2(0, X, _Y) -> X; l2(N, X, Y) -> case l3(N) of A when is_integer(A) -> l2(N - 1, A, Y); [B | _] -> l2(N - 1, X, B) end. l3(N) -> case N band 1 of 0 -> N; _ -> [N | []] end. ==snip== From donpedrothird@REDACTED Sun Apr 16 16:24:02 2017 From: donpedrothird@REDACTED (John Doe) Date: Sun, 16 Apr 2017 17:24:02 +0300 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching In-Reply-To: References: <22770.25282.691552.422821@gargle.gargle.HOWL> Message-ID: You are right. In the compiled code comparison is slightly faster than matching. -module(test). -export([test/2]). test(N, T) -> L = lists:seq(1, 10000000), F1 = fun() -> lists:foreach(fun(_) -> case T of {success, Z} -> Z; {error, Z} -> Z end end, L) end, erlang:display(timer:tc(F1)), F2 = fun() -> lists:foreach(fun(_) -> case N of X when X > 0 -> X; _ -> N end end, L) end, erlang:display(timer:tc(F2)). 1> c(test). {ok,test} 2> test:test(-1, {error, -1}). {292472,ok} {250206,ok} 2017-04-16 13:58 GMT+03:00 Jos? Valim : > John, did you measure in the shell? > > Code in the shell is evaluated and it won't give the same result as if the > code would be compiled as part of a module. > > Ideally you want to define a module, compile it, and then execute a > function that runs the benchmarking. > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Sun, Apr 16, 2017 at 1:53 AM, John Doe wrote: > >> Just've checked, the first one is approx 2 times faster on 19.1.6 >> 1> L = lists:seq(1, 1000000). >> 2> N = -1. >> 3> T = {error, -1}. >> 4> F1 = fun() -> lists:foreach(fun(_) -> case T of {success, Z} -> Z; >> {error, Z} -> Z end end, L) end. >> 5> F2 = fun() -> lists:foreach(fun(_) -> case N of X when X > 0 -> X; _ >> -> N end end, L) end. >> 6> timer:tc(F1). >> {1089950,ok} >> 7> >> 7> timer:tc(F2). >> {1870456,ok} >> >> 2017-04-15 21:13 GMT+03:00 Mikael Pettersson : >> >>> Suppose a frequently called function returns a positive integer (always >>> a fixnum) >>> and a boolean "success" or "failure" indicator (even in the failure case >>> the return >>> value is significant and will be used). Which is better from a >>> performance >>> perspective: >>> >>> 1. Tag the return value, {success, X} or {failure, Y}, and have callers >>> pattern-match >>> on that >>> >>> or, >>> >>> 2. Indicate failure by negating Y, and have callers match on >>> >>> X when X > 0 -> success case; >>> MinusY -> failure case % MinusY =< 0 is implicit >>> >>> Option 2 avoids the consing overheads of option 1, but I'm worried that >>> the X > 0 >>> guard may be less optimized than a plain structural pattern-match. The >>> last time >>> I checked, term-comparisons would check inline for identity, and >>> otherwise call >>> the general cmp() function which would then do a lot of type-casing. >>> >>> In my pure Erlang code I'm using option 1, but I'm reimplementing the >>> function >>> as a NIF, and would like to avoid the consing overheads is that's indeed >>> better. >>> >>> >>> /Mikael >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vans_163@REDACTED Sun Apr 16 16:32:32 2017 From: vans_163@REDACTED (Vans S) Date: Sun, 16 Apr 2017 14:32:32 +0000 (UTC) Subject: [erlang-questions] OpenID github login don't work on bugs.erlang.org In-Reply-To: References: Message-ID: <2076311678.1607857.1492353152302@mail.yahoo.com> Same for me, its been like that for months already. On Sunday, April 16, 2017 5:51 AM, PAILLEAU Eric wrote: Hi, looks like I cannot log on bugs.erlang.org with Github account via OpenId anymore : "OpenID Failed There was a problem during authentication. Please refer to logs for an additional info." Can someone else confirm? it is a bug ? Regards _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Sun Apr 16 21:21:33 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Sun, 16 Apr 2017 21:21:33 +0200 Subject: [erlang-questions] parsing and preprocessing binary data Message-ID: hello i want to preproces and parse binary data as erlang module. Let's assume that we have read file containing erlang module source and have binary data form it but not the file itself (or got it by other means), now want something like epp:parse_file but we want avoid to write that data to a file first and than call epp_parse_file key thing here is: macros should be expanded before parsing. thanks Kuna Prime -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Apr 16 22:06:03 2017 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 16 Apr 2017 22:06:03 +0200 Subject: [erlang-questions] OpenID github login don't work on bugs.erlang.org In-Reply-To: <2076311678.1607857.1492353152302@mail.yahoo.com> Message-ID: <429414ec-20ff-44d8-8be4-849f0d0be38f@email.android.com> An HTML attachment was scrubbed... URL: From vans_163@REDACTED Sun Apr 16 22:41:05 2017 From: vans_163@REDACTED (Vans S) Date: Sun, 16 Apr 2017 20:41:05 +0000 (UTC) Subject: [erlang-questions] using anonymous functions when recompiling modules? References: <1519395992.1763811.1492375265866.ref@mail.yahoo.com> Message-ID: <1519395992.1763811.1492375265866@mail.yahoo.com> It seems if you define an anonymous function in a module then compile and hotload the module, once the full purge happens (after the second new version), the anonymous function ceases to exist. -module(test). -compile(export_all). t() -> Anon = fun()-> io:format("hi ~n", []) end, spawn(fun() -> (fun Loop()-> Anon(), timer:sleep(2000), Loop() end)() end). te() -> ok. Now if you change te function to say tee, recompile+hotload, then change it to teee, and recompile+hotload again. You stop seeing "hi" be printed, the process is dead, and there are no errors. Does this mean it is simply not recommended to use anonymous functions? Would saving the anonymous function to the process dictionary or something keep it alive? So many questions of why this extremely hard to debug behavior happens. From eric.pailleau@REDACTED Mon Apr 17 00:35:40 2017 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Mon, 17 Apr 2017 00:35:40 +0200 Subject: [erlang-questions] using anonymous functions when recompiling modules? In-Reply-To: <1519395992.1763811.1492375265866@mail.yahoo.com> Message-ID: An HTML attachment was scrubbed... URL: From fxn@REDACTED Mon Apr 17 10:43:17 2017 From: fxn@REDACTED (Xavier Noria) Date: Mon, 17 Apr 2017 10:43:17 +0200 Subject: [erlang-questions] AST from source? Message-ID: Is there a one-liner to get an AST from source code? I have seen you can get it from the .beam file if debug_info was enabled, and have also seen some multiline snippet that uses erl_scan and erl_parse, but nothing as encapsulated as what you have for generating core or kernel. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Mon Apr 17 11:01:42 2017 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 17 Apr 2017 11:01:42 +0200 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: Message-ID: On 04/14/2017 02:12 PM, Jos? Valim wrote: > Hi everyone, > > I would like to propose to remove "tuple dispatches" from Erlang. > > ... > > I can think of two strategies for removing the behaviour: > > 1. Clean-cut: the code responsible for tuple dispatching will be > completely removed from the VM and a parse transform will be made > available. The parse transform could be part of Erlang/OTP or a separate > repository. ... > > 2. New byte codes: FWIW, I strongly support option #1 above. I think it's about time that "features" that make Erlang weird get eliminated from the implementation. It took me several years to convince the OTP team that they should eliminate tuple-funs from the implementation. They finally did and, unsurprisingly, the world has not stopped turning, nor did anybody decide to stop using Erlang for what it is good at, simply because it had no "tuple-funs" any longer. On the contrary, I think we have a slightly better language since their removal. Similarly, I think it will become slightly better with the removal of "tuple dispatches". Anyway, that's my .02. Kostis From alex0player@REDACTED Mon Apr 17 12:04:28 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 17 Apr 2017 13:04:28 +0300 Subject: [erlang-questions] AST from source? In-Reply-To: References: Message-ID: epp:parse_file? > 17 ???. 2017 ?., ? 11:43, Xavier Noria ???????(?): > > Is there a one-liner to get an AST from source code? > > I have seen you can get it from the .beam file if debug_info was enabled, and have also seen some multiline snippet that uses erl_scan and erl_parse, but nothing as encapsulated as what you have for generating core or kernel. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vances@REDACTED Mon Apr 17 12:07:04 2017 From: vances@REDACTED (Vance Shipley) Date: Mon, 17 Apr 2017 15:37:04 +0530 Subject: [erlang-questions] [ANN] RADIUS & EAP plugins for Tsung In-Reply-To: References: Message-ID: SigScale has contributed support for RADIUS protocol (with EAP) to the Tsung load testing tool: https://github.com/sigscale/tsung We developed this to test our Online Charging Server (OCS) under massive load. SigScale also published that open source project recently: https://github.com/sigscale/OCS -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Mon Apr 17 12:18:04 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Mon, 17 Apr 2017 12:18:04 +0200 Subject: [erlang-questions] AST from source? In-Reply-To: References: Message-ID: case compile:file(Filename, [to_kernel, binary, report]) of {ok, ModuleName, Code} ->... change to_kernel to to_core, or to 'S' for you know what. Binary will get you the result as a part of return value Report will print errors 2017-04-17 10:43 GMT+02:00 Xavier Noria : > Is there a one-liner to get an AST from source code? > > I have seen you can get it from the .beam file if debug_info was enabled, > and have also seen some multiline snippet that uses erl_scan and erl_parse, > but nothing as encapsulated as what you have for generating core or kernel. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Mon Apr 17 12:45:48 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 17 Apr 2017 12:45:48 +0200 Subject: [erlang-questions] AST from source? In-Reply-To: References: Message-ID: It can also be done from the command line: erlc +to_pp path/to/file.erl Besides +to_pp, you can pass: - +to_pp - pretty printed Erlang format after parse transforms and file imports - +to_exp - pretty printed Erlang format after parse transforms, imports, records and others - +to_core - Core Erlang - +to_kernel - Kernel representation - +to_asm - ASM representation We may have others but from memory those are the main ones. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Mon Apr 17 13:49:09 2017 From: fxn@REDACTED (Xavier Noria) Date: Mon, 17 Apr 2017 13:49:09 +0200 Subject: [erlang-questions] AST from source? In-Reply-To: References: Message-ID: Awesome, I knew about +to_core, and +to_kernel but was not aware of the other ones. Also, it seems that +to_pp is the AST of the Erlang code c(File, 'P') yields, and similar for +to_exp and 'E'. I see 'P' and 'E' documented in the docs of compile:file/2, but none of the flags in that list... are those considered to be private maybe? Could volunteer a doc patch otherwise. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay.s.doane@REDACTED Mon Apr 17 22:43:06 2017 From: jay.s.doane@REDACTED (Jay Doane) Date: Mon, 17 Apr 2017 13:43:06 -0700 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information Message-ID: Our team is currently using the process dictionary to store meta information for long running processes, and would like to hear from folks familiar with beam internals whether this is a recommended practice. Essentially we add meta information about the current request in a process dictionary of every process related to handling of this request. This information could be: - current file - user name of initiator of the request - http query parameters - type of a process (indexer | compactor | http_handler) For example, when we detect performance problems, we run a function that returns the Pids of (say) top memory using processes. We use the meta info stored in process dictionaries to determine which documents are using excessive memory so we can take corrective action. In practice, we are unable to use normal OTP messaging because in many cases, the high memory using processes have also grown huge message queues, and so would be unresponsive to any additional messages. Given our existing constraints, is this a reasonable use of the process dictionary? Thanks, Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Mon Apr 17 23:07:18 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Mon, 17 Apr 2017 23:07:18 +0200 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information In-Reply-To: References: Message-ID: If you store some data that doesn't change much later ? I see no problem here, perfectly fine example of PD use. But on the other hand, if you store mutable data in PD ? this is considered an antipattern and it makes your code have side effects which are hard to track. Also when your process dies, all data in PD will be lost, which is OK if data is set once, but not OK if the data was important. 2017-04-17 22:43 GMT+02:00 Jay Doane : > Our team is currently using the process dictionary to store meta > information for long running processes, and would like to hear from folks > familiar with beam internals whether this is a recommended practice. > > Essentially we add meta information about the current request in a process > dictionary of every process related to handling of this request. This > information could be: > - current file > - user name of initiator of the request > - http query parameters > - type of a process (indexer | compactor | http_handler) > > For example, when we detect performance problems, we run a function that > returns the Pids of (say) top memory using processes. We use the meta info > stored in process dictionaries to determine which documents are using > excessive memory so we can take corrective action. > > In practice, we are unable to use normal OTP messaging because in many > cases, the high memory using processes have also grown huge message queues, > and so would be unresponsive to any additional messages. > > Given our existing constraints, is this a reasonable use of the process > dictionary? > > Thanks, > Jay > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Mon Apr 17 23:38:11 2017 From: igor.clark@REDACTED (Igor Clark) Date: Mon, 17 Apr 2017 22:38:11 +0100 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 Message-ID: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> Hi all, I'm having problems updating a NIF to work correctly with R19 and/or MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk to some hardware. It's been working just fine with R18 on El Capitan, but it fails in a manner I don't understand when I trying to run it on R19 or Sierra. I'm pretty baffled as to what's going on, and wonder if any of the following might ring bells or set off a spider-sense for anyone here? - The code continues to work 100% as expected on R18 / Mac OS El Capitan (10.11), as it has for the last year or so. - On R19 on El Capitan, 99% of the C code works as before, but one particular MacOS C API call fails unexpectedly. It doesn't crash or cause errors directly, but it returns null instead of a valid value as before. There are no build errors or warnings. (I'm using 'pc' plugin version 1.5.0 with rebar3). Other MacOS API calls talking to the same hardware work as expected, sending correct commands, getting the hardware to carry out operations as expected and returning expected values, without any problems. - On both R18 & R19 on Sierra, exactly the same problem occurs as above in R19 on El Capitan. - If I make the failing/null-returning OS API call in a plain, simple-as-possible C file compiled with gcc (Apple LLVM/clang), it works as expected, *on both OS versions*. - If I paste that same simple-as-possible C code verbatim into the NIF, as the first line in the load() function, and run it on R18/El Capitan, it continues to work as expected. - If I then try to run that identical pasted-verbatim NIF code with no changes on R18/Sierra or on R19 & either OS version, this one particular function call fails, as above. I realise it could be many things, and my initial thought was that it must be the OS change or even the hardware failing - but the hardware and C API call continue to work just fine on both OS versions when the NIF stuff is taken out of the picture, and on El Capitan, the previously-not-happening problem reliably (and only) starts happening when I switch from R18 to R19. Did anything change between R18 and R19 that might somehow make some, but not all, external C function calls fail on El Capitan/clang 8.0.0? And could that be something that also makes the same thing happen on both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps be something do with the port compiler & LD/CXX flags? Any ideas as to where else I could look to try to track this down, or what might be causing this? (BTW, I'm using homebrew erlang, and I've tried all the above with versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The gcc/clang version on El Capitan is from Xcode 8.1, "Apple LLVM version 8.0.0 (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple LLVM version 8.1.0 (clang-802.0.41)" - but as above, the plain/non-NIF C code works perfectly and identically on both versions.) Thanks very much, Igor From pfisher@REDACTED Tue Apr 18 00:52:20 2017 From: pfisher@REDACTED (Fisher, Paul) Date: Mon, 17 Apr 2017 22:52:20 +0000 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> Message-ID: Complete wild guess, but is there a difference in nif initialization running off the main thread in non-working version? This would be a problem for some low-level Mach related things. Not near laptop to look through the code to see if that changed. Get Outlook for iOS ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Igor Clark Sent: Monday, April 17, 2017 4:38:11 PM To: erlang-questions@REDACTED Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 Hi all, I'm having problems updating a NIF to work correctly with R19 and/or MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk to some hardware. It's been working just fine with R18 on El Capitan, but it fails in a manner I don't understand when I trying to run it on R19 or Sierra. I'm pretty baffled as to what's going on, and wonder if any of the following might ring bells or set off a spider-sense for anyone here? - The code continues to work 100% as expected on R18 / Mac OS El Capitan (10.11), as it has for the last year or so. - On R19 on El Capitan, 99% of the C code works as before, but one particular MacOS C API call fails unexpectedly. It doesn't crash or cause errors directly, but it returns null instead of a valid value as before. There are no build errors or warnings. (I'm using 'pc' plugin version 1.5.0 with rebar3). Other MacOS API calls talking to the same hardware work as expected, sending correct commands, getting the hardware to carry out operations as expected and returning expected values, without any problems. - On both R18 & R19 on Sierra, exactly the same problem occurs as above in R19 on El Capitan. - If I make the failing/null-returning OS API call in a plain, simple-as-possible C file compiled with gcc (Apple LLVM/clang), it works as expected, *on both OS versions*. - If I paste that same simple-as-possible C code verbatim into the NIF, as the first line in the load() function, and run it on R18/El Capitan, it continues to work as expected. - If I then try to run that identical pasted-verbatim NIF code with no changes on R18/Sierra or on R19 & either OS version, this one particular function call fails, as above. I realise it could be many things, and my initial thought was that it must be the OS change or even the hardware failing - but the hardware and C API call continue to work just fine on both OS versions when the NIF stuff is taken out of the picture, and on El Capitan, the previously-not-happening problem reliably (and only) starts happening when I switch from R18 to R19. Did anything change between R18 and R19 that might somehow make some, but not all, external C function calls fail on El Capitan/clang 8.0.0? And could that be something that also makes the same thing happen on both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps be something do with the port compiler & LD/CXX flags? Any ideas as to where else I could look to try to track this down, or what might be causing this? (BTW, I'm using homebrew erlang, and I've tried all the above with versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The gcc/clang version on El Capitan is from Xcode 8.1, "Apple LLVM version 8.0.0 (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple LLVM version 8.1.0 (clang-802.0.41)" - but as above, the plain/non-NIF C code works perfectly and identically on both versions.) Thanks very much, Igor _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZWIKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= Confidentiality Notice | This email and any included attachments may be privileged, confidential and/or otherwise protected from disclosure. Access to this email by anyone other than the intended recipient is unauthorized. If you believe you have received this email in error, please contact the sender immediately and delete all copies. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Tue Apr 18 04:47:11 2017 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 18 Apr 2017 02:47:11 +0000 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> Message-ID: Following Paul's idea, what happens in both the NIF and your minimal C program if you first launch a new thread and call the API from there? On Mon, Apr 17, 2017 at 5:52 PM Fisher, Paul wrote: > Complete wild guess, but is there a difference in nif initialization > running off the main thread in non-working version? This would be a problem > for some low-level Mach related things. Not near laptop to look through the > code to see if that changed. > > > Get Outlook for iOS > ------------------------------ > *From:* erlang-questions-bounces@REDACTED < > erlang-questions-bounces@REDACTED> on behalf of Igor Clark < > igor.clark@REDACTED> > *Sent:* Monday, April 17, 2017 4:38:11 PM > *To:* erlang-questions@REDACTED > *Subject:* [erlang-questions] NIF+device problem moving R18->R19 / Mac > 10.11->10.12 > > Hi all, > > I'm having problems updating a NIF to work correctly with R19 and/or > MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk to some > hardware. It's been working just fine with R18 on El Capitan, but it > fails in a manner I don't understand when I trying to run it on R19 or > Sierra. I'm pretty baffled as to what's going on, and wonder if any of > the following might ring bells or set off a spider-sense for anyone here? > > - The code continues to work 100% as expected on R18 / Mac OS El Capitan > (10.11), as it has for the last year or so. > > - On R19 on El Capitan, 99% of the C code works as before, but one > particular MacOS C API call fails unexpectedly. It doesn't crash or > cause errors directly, but it returns null instead of a valid value as > before. There are no build errors or warnings. (I'm using 'pc' plugin > version 1.5.0 with rebar3). Other MacOS API calls talking to the same > hardware work as expected, sending correct commands, getting the > hardware to carry out operations as expected and returning expected > values, without any problems. > > - On both R18 & R19 on Sierra, exactly the same problem occurs as above > in R19 on El Capitan. > > - If I make the failing/null-returning OS API call in a plain, > simple-as-possible C file compiled with gcc (Apple LLVM/clang), it works > as expected, *on both OS versions*. > > - If I paste that same simple-as-possible C code verbatim into the NIF, > as the first line in the load() function, and run it on R18/El Capitan, > it continues to work as expected. > > - If I then try to run that identical pasted-verbatim NIF code with no > changes on R18/Sierra or on R19 & either OS version, this one particular > function call fails, as above. > > I realise it could be many things, and my initial thought was that it > must be the OS change or even the hardware failing - but the hardware > and C API call continue to work just fine on both OS versions when the > NIF stuff is taken out of the picture, and on El Capitan, the > previously-not-happening problem reliably (and only) starts happening > when I switch from R18 to R19. > > Did anything change between R18 and R19 that might somehow make some, > but not all, external C function calls fail on El Capitan/clang 8.0.0? > And could that be something that also makes the same thing happen on > both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps be something > do with the port compiler & LD/CXX flags? > > Any ideas as to where else I could look to try to track this down, or > what might be causing this? > > (BTW, I'm using homebrew erlang, and I've tried all the above with > versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The gcc/clang > version on El Capitan is from Xcode 8.1, "Apple LLVM version 8.0.0 > (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple LLVM > version 8.1.0 (clang-802.0.41)" - but as above, the plain/non-NIF C code > works perfectly and identically on both versions.) > > Thanks very much, > Igor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZWIKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= > Confidentiality Notice | This email and any included attachments may be > privileged, confidential and/or otherwise protected from disclosure. Access > to this email by anyone other than the intended recipient is unauthorized. > If you believe you have received this email in error, please contact the > sender immediately and delete all copies. If you are not the intended > recipient, you are notified that disclosing, copying, distributing or > taking any action in reliance on the contents of this information is > strictly prohibited. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Apr 18 07:11:03 2017 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 18 Apr 2017 08:11:03 +0300 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information In-Reply-To: References: Message-ID: Yes, this is a good way. You can inspect process in such non-blocking way, but use this information as a hint, not as a proper and reliable data. -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Tue Apr 18 08:52:26 2017 From: igor.clark@REDACTED (Igor Clark) Date: Tue, 18 Apr 2017 07:52:26 +0100 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> Message-ID: <24009245-b22d-a245-fc71-c7c0cfcc8632@gmail.com> Thanks Paul and Daniel, Just tested this: the exact same thing happens with threads as without. The API call succeeds when run in a separate thread in the minimal C program using pthread_create(), on both El Capitan and Sierra; when run in a separate thread in the NIF using enif_thread_create(), the API call succeeds under R18 on El Capitan and fails under R19. (My NIF code hasn't been explicitly creating or using any threads at all until now, if that makes any difference.) Does that give any further clues? Cheers, Igor On 18/04/2017 03:47, Daniel Goertzen wrote: > Following Paul's idea, what happens in both the NIF and your minimal C > program if you first launch a new thread and call the API from there? > > On Mon, Apr 17, 2017 at 5:52 PM Fisher, Paul > wrote: > > Complete wild guess, but is there a difference in nif > initialization running off the main thread in non-working version? > This would be a problem for some low-level Mach related things. > Not near laptop to look through the code to see if that changed. > > > Get Outlook for iOS > ------------------------------------------------------------------------ > *From:* erlang-questions-bounces@REDACTED > > > on behalf of Igor > Clark > > *Sent:* Monday, April 17, 2017 4:38:11 PM > *To:* erlang-questions@REDACTED > *Subject:* [erlang-questions] NIF+device problem moving R18->R19 / > Mac 10.11->10.12 > Hi all, > > I'm having problems updating a NIF to work correctly with R19 and/or > MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk to some > hardware. It's been working just fine with R18 on El Capitan, but it > fails in a manner I don't understand when I trying to run it on > R19 or > Sierra. I'm pretty baffled as to what's going on, and wonder if > any of > the following might ring bells or set off a spider-sense for > anyone here? > > - The code continues to work 100% as expected on R18 / Mac OS El > Capitan > (10.11), as it has for the last year or so. > > - On R19 on El Capitan, 99% of the C code works as before, but one > particular MacOS C API call fails unexpectedly. It doesn't crash or > cause errors directly, but it returns null instead of a valid > value as > before. There are no build errors or warnings. (I'm using 'pc' plugin > version 1.5.0 with rebar3). Other MacOS API calls talking to the same > hardware work as expected, sending correct commands, getting the > hardware to carry out operations as expected and returning expected > values, without any problems. > > - On both R18 & R19 on Sierra, exactly the same problem occurs as > above > in R19 on El Capitan. > > - If I make the failing/null-returning OS API call in a plain, > simple-as-possible C file compiled with gcc (Apple LLVM/clang), it > works > as expected, *on both OS versions*. > > - If I paste that same simple-as-possible C code verbatim into the > NIF, > as the first line in the load() function, and run it on R18/El > Capitan, > it continues to work as expected. > > - If I then try to run that identical pasted-verbatim NIF code > with no > changes on R18/Sierra or on R19 & either OS version, this one > particular > function call fails, as above. > > I realise it could be many things, and my initial thought was that it > must be the OS change or even the hardware failing - but the hardware > and C API call continue to work just fine on both OS versions when > the > NIF stuff is taken out of the picture, and on El Capitan, the > previously-not-happening problem reliably (and only) starts happening > when I switch from R18 to R19. > > Did anything change between R18 and R19 that might somehow make some, > but not all, external C function calls fail on El Capitan/clang > 8.0.0? > And could that be something that also makes the same thing happen on > both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps be > something > do with the port compiler & LD/CXX flags? > > Any ideas as to where else I could look to try to track this down, or > what might be causing this? > > (BTW, I'm using homebrew erlang, and I've tried all the above with > versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The > gcc/clang > version on El Capitan is from Xcode 8.1, "Apple LLVM version 8.0.0 > (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple LLVM > version 8.1.0 (clang-802.0.41)" - but as above, the plain/non-NIF > C code > works perfectly and identically on both versions.) > > Thanks very much, > Igor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZWIKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= > > Confidentiality Notice | This email and any included attachments > may be privileged, confidential and/or otherwise protected from > disclosure. Access to this email by anyone other than the intended > recipient is unauthorized. If you believe you have received this > email in error, please contact the sender immediately and delete > all copies. If you are not the intended recipient, you are > notified that disclosing, copying, distributing or taking any > action in reliance on the contents of this information is strictly > prohibited. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Tue Apr 18 08:54:59 2017 From: igor.clark@REDACTED (Igor Clark) Date: Tue, 18 Apr 2017 07:54:59 +0100 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: <24009245-b22d-a245-fc71-c7c0cfcc8632@gmail.com> References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> <24009245-b22d-a245-fc71-c7c0cfcc8632@gmail.com> Message-ID: Sorry, to be clearer, that should have read "when run in a separate thread in the NIF using enif_thread_create(), the API call succeeds under R18 on El Capitan, fails under R19 on El Capitan, and fails under R18 and R19 on Sierra". On Tue, Apr 18, 2017 at 7:52 AM, Igor Clark wrote: > Thanks Paul and Daniel, > > Just tested this: the exact same thing happens with threads as without. > The API call succeeds when run in a separate thread in the minimal C > program using pthread_create(), on both El Capitan and Sierra; when run in > a separate thread in the NIF using enif_thread_create(), the API call > succeeds under R18 on El Capitan and fails under R19. > > (My NIF code hasn't been explicitly creating or using any threads at all > until now, if that makes any difference.) > > Does that give any further clues? > > Cheers, > Igor > > > On 18/04/2017 03:47, Daniel Goertzen wrote: > > Following Paul's idea, what happens in both the NIF and your minimal C > program if you first launch a new thread and call the API from there? > > On Mon, Apr 17, 2017 at 5:52 PM Fisher, Paul > wrote: > >> Complete wild guess, but is there a difference in nif initialization >> running off the main thread in non-working version? This would be a problem >> for some low-level Mach related things. Not near laptop to look through the >> code to see if that changed. >> >> >> Get Outlook for iOS >> ------------------------------ >> *From:* erlang-questions-bounces@REDACTED > erlang.org> on behalf of Igor Clark >> *Sent:* Monday, April 17, 2017 4:38:11 PM >> *To:* erlang-questions@REDACTED >> *Subject:* [erlang-questions] NIF+device problem moving R18->R19 / Mac >> 10.11->10.12 >> >> Hi all, >> >> I'm having problems updating a NIF to work correctly with R19 and/or >> MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk to some >> hardware. It's been working just fine with R18 on El Capitan, but it >> fails in a manner I don't understand when I trying to run it on R19 or >> Sierra. I'm pretty baffled as to what's going on, and wonder if any of >> the following might ring bells or set off a spider-sense for anyone here? >> >> - The code continues to work 100% as expected on R18 / Mac OS El Capitan >> (10.11), as it has for the last year or so. >> >> - On R19 on El Capitan, 99% of the C code works as before, but one >> particular MacOS C API call fails unexpectedly. It doesn't crash or >> cause errors directly, but it returns null instead of a valid value as >> before. There are no build errors or warnings. (I'm using 'pc' plugin >> version 1.5.0 with rebar3). Other MacOS API calls talking to the same >> hardware work as expected, sending correct commands, getting the >> hardware to carry out operations as expected and returning expected >> values, without any problems. >> >> - On both R18 & R19 on Sierra, exactly the same problem occurs as above >> in R19 on El Capitan. >> >> - If I make the failing/null-returning OS API call in a plain, >> simple-as-possible C file compiled with gcc (Apple LLVM/clang), it works >> as expected, *on both OS versions*. >> >> - If I paste that same simple-as-possible C code verbatim into the NIF, >> as the first line in the load() function, and run it on R18/El Capitan, >> it continues to work as expected. >> >> - If I then try to run that identical pasted-verbatim NIF code with no >> changes on R18/Sierra or on R19 & either OS version, this one particular >> function call fails, as above. >> >> I realise it could be many things, and my initial thought was that it >> must be the OS change or even the hardware failing - but the hardware >> and C API call continue to work just fine on both OS versions when the >> NIF stuff is taken out of the picture, and on El Capitan, the >> previously-not-happening problem reliably (and only) starts happening >> when I switch from R18 to R19. >> >> Did anything change between R18 and R19 that might somehow make some, >> but not all, external C function calls fail on El Capitan/clang 8.0.0? >> And could that be something that also makes the same thing happen on >> both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps be something >> do with the port compiler & LD/CXX flags? >> >> Any ideas as to where else I could look to try to track this down, or >> what might be causing this? >> >> (BTW, I'm using homebrew erlang, and I've tried all the above with >> versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The gcc/clang >> version on El Capitan is from Xcode 8.1, "Apple LLVM version 8.0.0 >> (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple LLVM >> version 8.1.0 (clang-802.0.41)" - but as above, the plain/non-NIF C code >> works perfectly and identically on both versions.) >> >> Thanks very much, >> Igor >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang. >> org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_ >> h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_ >> jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZW >> IKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= >> Confidentiality Notice | This email and any included attachments may be >> privileged, confidential and/or otherwise protected from disclosure. Access >> to this email by anyone other than the intended recipient is unauthorized. >> If you believe you have received this email in error, please contact the >> sender immediately and delete all copies. If you are not the intended >> recipient, you are notified that disclosing, copying, distributing or >> taking any action in reliance on the contents of this information is >> strictly prohibited. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Tue Apr 18 10:33:31 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 18 Apr 2017 10:33:31 +0200 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information In-Reply-To: References: Message-ID: <22773.53083.315629.304393@gargle.gargle.HOWL> Jay Doane writes: > Our team is currently using the process dictionary to store meta > information for long running processes, and would like to hear from folks > familiar with beam internals whether this is a recommended practice. > > Essentially we add meta information about the current request in a process > dictionary of every process related to handling of this request. This > information could be: > - current file > - user name of initiator of the request > - http query parameters > - type of a process (indexer | compactor | http_handler) > > For example, when we detect performance problems, we run a function that > returns the Pids of (say) top memory using processes. We use the meta info > stored in process dictionaries to determine which documents are using > excessive memory so we can take corrective action. > > In practice, we are unable to use normal OTP messaging because in many > cases, the high memory using processes have also grown huge message queues, > and so would be unresponsive to any additional messages. > > Given our existing constraints, is this a reasonable use of the process > dictionary? Yes, I think this is perfectly reasonable. From gianfranco.alongi@REDACTED Tue Apr 18 11:29:25 2017 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Tue, 18 Apr 2017 12:29:25 +0300 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information In-Reply-To: <22773.53083.315629.304393@gargle.gargle.HOWL> References: <22773.53083.315629.304393@gargle.gargle.HOWL> Message-ID: Hi! It kind of sounds similar to the idea behind gproc, which is a widely used library, so as everyone has said already - this is perfectly fine. https://github.com/uwiger/gproc Use case: System inspection Gproc was designed to work as a central index for "process metadata", i.e. properties that describe the role and characteristics of each process. Having a single registry that is flexible enough to hold important types of property makes it easier to (a) find processes of a certain type, and (b) query and browse key data in a running system. On Tue, Apr 18, 2017 at 11:33 AM, Mikael Pettersson wrote: > Jay Doane writes: > > Our team is currently using the process dictionary to store meta > > information for long running processes, and would like to hear from > folks > > familiar with beam internals whether this is a recommended practice. > > > > Essentially we add meta information about the current request in a > process > > dictionary of every process related to handling of this request. This > > information could be: > > - current file > > - user name of initiator of the request > > - http query parameters > > - type of a process (indexer | compactor | http_handler) > > > > For example, when we detect performance problems, we run a function that > > returns the Pids of (say) top memory using processes. We use the meta > info > > stored in process dictionaries to determine which documents are using > > excessive memory so we can take corrective action. > > > > In practice, we are unable to use normal OTP messaging because in many > > cases, the high memory using processes have also grown huge message > queues, > > and so would be unresponsive to any additional messages. > > > > Given our existing constraints, is this a reasonable use of the process > > dictionary? > > Yes, I think this is perfectly reasonable. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Apr 18 11:44:54 2017 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 18 Apr 2017 10:44:54 +0100 Subject: [erlang-questions] Request for opinions on using process dictionary for meta information In-Reply-To: References: <22773.53083.315629.304393@gargle.gargle.HOWL> Message-ID: On 18 April 2017 at 10:29, Gianfranco Alongi wrote: > Hi! > > It kind of sounds similar to the idea behind gproc, which is a widely used > library, > so as everyone has said already - this is perfectly fine. > > https://github.com/uwiger/gproc > Doesn't gproc store the process metadata in an ETS table? That's a valid approach, too, but you need a monitor process to clean up if the process dies. As another pro-example of storing information in the process dictionary; this is what lager does with metadata -- see lager:md. Both approaches are valid: we use all of the following: 1. Process dictionary (usually for lager, but sometimes separately). 2. Separate process registry using one or more ETS tables (we don't use gproc; we wanted slightly different semantics). 3. Process responds to gen_server:call(Pid, info) They're useful for different information, for different scenarios. I'd be reluctant to use the process dictionary for anything other than process-internal-use or debugging information, though. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.bolinder@REDACTED Tue Apr 18 13:07:02 2017 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Tue, 18 Apr 2017 11:07:02 +0000 Subject: [erlang-questions] error with xref and on_load functions In-Reply-To: References: Message-ID: Hi, I don't think the bug has been reported before. Thanks! I'll write a ticket. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson ________________________________ Fr?n: erlang-questions-bounces@REDACTED f?r Felipe Ripoll Gisbert Skickat: den 11 april 2017 21:48:12 Till: erlang-questions@REDACTED ?mne: [erlang-questions] error with xref and on_load functions Hi all, I have a problem running xref on modules with `on_load` functions, for example, giving this module: -module(xreftest). -on_load(init/0). init() -> ok. if we run xref: 1> c(xreftest, [debug_info]). {ok,xreftest} 2> xref:d("."). [{deprecated,[]}, {undefined,[]}, {unused,[{xreftest,init,0}]}] we get an `unused` warning for init/0 but it is actually used on load. Is this a known issue? Thanks -- Felipe Ripoll Gisbert www.feliperipoll.es felipe@REDACTED ferigis@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Tue Apr 18 15:45:56 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Tue, 18 Apr 2017 19:15:56 +0530 (IST) Subject: [erlang-questions] Building a chat application which allows users sitting on different PCs to chat in a chatroom Message-ID: <878101729.62290.1492523156802@webmail.blacklightsw.com> Hi, I am new to erlang and I wanted to build a chatting application using ejabberd as a server but I have been trying different things and have been able to do few things. Till now I have been able to learn how to run a custom module in ejabberd by making hooks successfully. I have also created a Multi User Chatroom using Client as Pidgin and server as ejabberd successfully. But now I have been searching the way to go one step further and make this MUC a chatroom that can be accessed from other Computers also.What I would really like to build is a MUC which can be accessed by people sitting over different PCs and in doing so I would like to use technologies like ejabberd, xmpp, Pidgin, erlang etc. But I can't figure out in which direction to head to do this. It would be very helpful if anyone can comment over this please. Regards, Abhishek Ranjan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bp@REDACTED Tue Apr 18 15:07:15 2017 From: bp@REDACTED (Bjarne Wichmann Bagge Petersen) Date: Tue, 18 Apr 2017 15:07:15 +0200 Subject: [erlang-questions] CT: outputting lager debug messages In-Reply-To: <7a8f77b2-6613-160a-4b83-11b579200b1e@gmx.net> References: <7a8f77b2-6613-160a-4b83-11b579200b1e@gmx.net> Message-ID: Great, thanks. init_per_suite(Cfg) -> lager_common_test_backend:bounce(debug), Cfg. did the trick, now I am getting debug messages in tests. :-) Best regards Bjarne Wichmann Bagge Petersen Backend Developer, ShopGun E: bp@REDACTED L: dk.linkedin.com/in/bjarnewp W: shopgun.com A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark 2017-04-12 17:42 GMT+02:00 Jacob : > I am using this code in my test suite modules to catch lager debug output: > > init_per_suite(Cfg) -> > lager_common_test_backend:bounce(debug), > Cfg. > > No other changes are required. The lager logging messages are then > included in the regular test suite output files. > > Best regards > > Jacob > > On 04/12/2017 05:32 PM, Luis Rasc?o wrote: > > Maybe you're missing `lager:set_loglevel(lager_console_backend, > debug).` ? > > > > On Sat, Apr 8, 2017 at 6:48 AM, Bjarne Wichmann Bagge Petersen > > > wrote: > > > > I have a little issue where I would like to output lager > > debug-messages when running common tests. But I can't get it to work. > > > > In `init_pr_suite` I have > > > > application:load(lager), > > application:set_env(lager, handlers, [{lager_console_backend, > > debug}, {lager_common_test_backend, debug}]), > > > > And in the test-code I have this little snippet: > > > > ct:log("ENV ~p", [application:get_all_env(lager)]), > > lager:warning("Now you see me"), > > lager:debug("Now you don't"), > > > > Which outputs: > > > > *** User 2017-04-08 07:36:42.089 *** > > ENV [{crash_log_count,5}, > > {crash_log_date,"$D0"}, > > {colored,false}, > > {crash_log_msg_size,65536}, > > {colors,[{debug,"\e[0;38m"}, > > {info,"\e[1;37m"}, > > {notice,"\e[1;36m"}, > > {warning,"\e[1;33m"}, > > {error,"\e[1;31m"}, > > {critical,"\e[1;35m"}, > > {alert,"\e[1;44m"}, > > {emergency,"\e[1;41m"}]}, > > {included_applications,[]}, > > {async_threshold,20}, > > {crash_log_size,10485760}, > > {error_logger_whitelist,[cth_readable_failonly]}, > > {error_logger_redirect,true}, > > {async_threshold_window,5}, > > {crash_log,"log/crash.log"}, > > {error_logger_hwm,50}, > > {handlers,[{lager_console_backend,debug}, > > {lager_common_test_backend,debug}]}] > > > > *** User 2017-04-08 07:36:42.090 *** > > 07:36:42.089 [warning] Now you see me > > > > > > Missing the debug-line. > > > > So what am I missing? Is `lager_common_test_backend` the wrong flag? > > Or is CT muting debug-messages anyway? > > > > > > Best regards > > > > > > Bjarne Wichmann Bagge Petersen > > > > Backend Developer, ShopGun > > > > > > E: bp@REDACTED > > L: dk.linkedin.com/in/bjarnewp > > W: shopgun.com > > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, > > Denmark > > > > ShopGun, your helping hand in shopping, is available at shopgun.com > > and as the ShopGun app > > . > > > > This email is intended solely for the addressee and may contain > > confidential information and/or privileged confidential advice. If > > you receive this email by mistake, please do not read, print, > > transmit, store, or in any other way use the email or its > > attachments. Instead, please notify the addresser immediately and > > then permanently delete it. Thank you. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > -- > > PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ShopGun, your helping hand in shopping, is available at shopgun.com and as the ShopGun app . This email is intended solely for the addressee and may contain confidential information and/or privileged confidential advice. If you receive this email by mistake, please do not read, print, transmit, store, or in any other way use the email or its attachments. Instead, please notify the addresser immediately and then permanently delete it. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bp@REDACTED Tue Apr 18 16:41:00 2017 From: bp@REDACTED (Bjarne Wichmann Bagge Petersen) Date: Tue, 18 Apr 2017 16:41:00 +0200 Subject: [erlang-questions] CT: outputting lager debug messages In-Reply-To: References: <7a8f77b2-6613-160a-4b83-11b579200b1e@gmx.net> Message-ID: I was too quick to celebrate. There is a side effect to lager_common_test_backend:bounce(debug) in that i mutes non-lager error messages from the CT logs (html), though I do see them in the console now. Best regards Bjarne Wichmann Bagge Petersen Backend Developer, ShopGun E: bp@REDACTED L: dk.linkedin.com/in/bjarnewp W: shopgun.com A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark 2017-04-18 15:07 GMT+02:00 Bjarne Wichmann Bagge Petersen : > Great, thanks. > > init_per_suite(Cfg) -> > lager_common_test_backend:bounce(debug), > Cfg. > > did the trick, now I am getting debug messages in tests. :-) > > > Best regards > > > Bjarne Wichmann Bagge Petersen > > Backend Developer, ShopGun > > > E: bp@REDACTED > L: dk.linkedin.com/in/bjarnewp > W: shopgun.com > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, Denmark > > 2017-04-12 17:42 GMT+02:00 Jacob : > >> I am using this code in my test suite modules to catch lager debug output: >> >> init_per_suite(Cfg) -> >> lager_common_test_backend:bounce(debug), >> Cfg. >> >> No other changes are required. The lager logging messages are then >> included in the regular test suite output files. >> >> Best regards >> >> Jacob >> >> On 04/12/2017 05:32 PM, Luis Rasc?o wrote: >> > Maybe you're missing `lager:set_loglevel(lager_console_backend, >> debug).` ? >> > >> > On Sat, Apr 8, 2017 at 6:48 AM, Bjarne Wichmann Bagge Petersen >> > > wrote: >> > >> > I have a little issue where I would like to output lager >> > debug-messages when running common tests. But I can't get it to >> work. >> > >> > In `init_pr_suite` I have >> > >> > application:load(lager), >> > application:set_env(lager, handlers, [{lager_console_backend, >> > debug}, {lager_common_test_backend, debug}]), >> > >> > And in the test-code I have this little snippet: >> > >> > ct:log("ENV ~p", [application:get_all_env(lager)]), >> > lager:warning("Now you see me"), >> > lager:debug("Now you don't"), >> > >> > Which outputs: >> > >> > *** User 2017-04-08 07:36:42.089 *** >> > ENV [{crash_log_count,5}, >> > {crash_log_date,"$D0"}, >> > {colored,false}, >> > {crash_log_msg_size,65536}, >> > {colors,[{debug,"\e[0;38m"}, >> > {info,"\e[1;37m"}, >> > {notice,"\e[1;36m"}, >> > {warning,"\e[1;33m"}, >> > {error,"\e[1;31m"}, >> > {critical,"\e[1;35m"}, >> > {alert,"\e[1;44m"}, >> > {emergency,"\e[1;41m"}]}, >> > {included_applications,[]}, >> > {async_threshold,20}, >> > {crash_log_size,10485760}, >> > {error_logger_whitelist,[cth_readable_failonly]}, >> > {error_logger_redirect,true}, >> > {async_threshold_window,5}, >> > {crash_log,"log/crash.log"}, >> > {error_logger_hwm,50}, >> > {handlers,[{lager_console_backend,debug}, >> > {lager_common_test_backend,debug}]}] >> > >> > *** User 2017-04-08 07:36:42.090 *** >> > 07:36:42.089 [warning] Now you see me >> > >> > >> > Missing the debug-line. >> > >> > So what am I missing? Is `lager_common_test_backend` the wrong flag? >> > Or is CT muting debug-messages anyway? >> > >> > >> > Best regards >> > >> > >> > Bjarne Wichmann Bagge Petersen >> > >> > Backend Developer, ShopGun >> > >> > >> > E: bp@REDACTED >> > L: dk.linkedin.com/in/bjarnewp >> > W: shopgun.com >> > A: Arne Jacobsens All? 16, Field's, 3rd floor, DK-2300 K?benhavn S, >> > Denmark >> > >> > ShopGun, your helping hand in shopping, is available at shopgun.com >> > and as the ShopGun app >> > . >> > >> > This email is intended solely for the addressee and may contain >> > confidential information and/or privileged confidential advice. If >> > you receive this email by mistake, please do not read, print, >> > transmit, store, or in any other way use the email or its >> > attachments. Instead, please notify the addresser immediately and >> > then permanently delete it. Thank you. >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > >> > >> > -- >> > PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- ShopGun, your helping hand in shopping, is available at shopgun.com and as the ShopGun app . This email is intended solely for the addressee and may contain confidential information and/or privileged confidential advice. If you receive this email by mistake, please do not read, print, transmit, store, or in any other way use the email or its attachments. Instead, please notify the addresser immediately and then permanently delete it. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Apr 19 05:56:27 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 19 Apr 2017 15:56:27 +1200 Subject: [erlang-questions] BEAM micro-optimization: term comparison (> 0) versus pattern-matching In-Reply-To: <22770.25282.691552.422821@gargle.gargle.HOWL> References: <22770.25282.691552.422821@gargle.gargle.HOWL> Message-ID: > On 16/04/2017, at 6:13 AM, Mikael Pettersson wrote: > > Suppose a frequently called function returns a positive integer (always a fixnum) > and a boolean "success" or "failure" indicator (even in the failure case the return > value is significant and will be used). Which is better from a performance > perspective: Measure it. I will be astonished if it makes any practical difference. You are probably likely to get more benefit from inlining, or caching, or somehow reducing the number of calls. > > 1. Tag the return value, {success, X} or {failure, Y}, and have callers pattern-match > on that > > or, > > 2. Indicate failure by negating Y, and have callers match on > > X when X > 0 -> success case; > MinusY -> failure case % MinusY =< 0 is implicit You have omitted option 3: frequently_called(Inputs, Success, Failure) -> ... success(Fixnum) ... ; ... failure(Other_Fixnum) ... And option 4: - Have the success case(s) return the usual number. - Have the failure case(s) raise an exception containing the other number. And option 5: - Have the success case return the usual number. - Have the failure case return {error,X} or {X} or something else And options 6 to 8 that I haven't thought of yet... > > Option 2 avoids the consing overheads of option 1, but I'm worried that the X > 0 > guard may be less optimized than a plain structural pattern-match. The last time > I checked, term-comparisons would check inline for identity, and otherwise call > the general cmp() function which would then do a lot of type-casing. I am much more worried about the possibility of things going wrong. I'm worried about returning zero or the wrong sign. I'm worried about making it harder to get the most out of the Dialyzer. Note that what is optimised and how well is something that *changes* over time. Just to give you an idea of what's possible here, an optimisation I looked into (for a different language) once goes like this: If there are many calls to f(...) in contexts like case f(...) of {t1,...} -> ... | {tn,...} -> ... end then make two copies of f. In one of the copies, ... {tk,E1,...,Ex} turns into R1,...,Rx := E1,...,Ex; return k and the case turns into switch (f(...)) { ... case k: /* data are in R1,...,Rx */ ... } This also applies to {tk,X1,...,Xx} = f(...) which is a degenerate 'case', Use the other copy when the call is not in such a context. This was inspired by the compiler for Janus, IIRC and *partly* motivated by multiple value returns in Common Lisp and Scheme. I'm not saying that this will ever be done by an Erlang compiler, but it *could* be done by an Erlang compiler, and it would make the tuple version the cheapest available. > In my pure Erlang code I'm using option 1, but I'm reimplementing the function > as a NIF, and would like to avoid the consing overheads is that's indeed better. I presume that under "consing overheads" you are including the cost of cleaning up the dead tuples in garbage collection. But seriously, there is no substitute for measurement. If the function is costly enough to be worth reimplementing as a NIF, this aspect of it is likely to be quite unimportant, but only measurement can tell you for sure. You have given us no idea what this function does. Have you profiled the arguments it is given to see if it is often called with similar enough arguments to make some kind of caching attractive? From frank.muller.erl@REDACTED Wed Apr 19 06:40:13 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 19 Apr 2017 04:40:13 +0000 Subject: [erlang-questions] Send VM metrics to Grafana In-Reply-To: <1486662916.1541324.875894104.2F2C3D79@webmail.messagingengine.com> References: <1486662916.1541324.875894104.2F2C3D79@webmail.messagingengine.com> Message-ID: Hi Tristan, Any update on that? /Frank wrote: > Yup, we do at work and a number of others do as well I know of. > > I spoke with Ilya yesterday on irc about a shared template for a Erlang > metrics dashboard, which he is going to start this weekend based on his > rabbitmq dashboard. > > I plan to do a full writeup on grafana, prometheus and alertmanager for > Erlang, more specifically Elli, in the near future -- once I get it all > working, hehe. > > -- > Tristan Sloughter > "I am not a crackpot" - Abe Simpson > t@REDACTED > > > On Thu, Feb 9, 2017, at 09:48 AM, Frank Muller wrote: > > Thanks Illya, found it already this morning. > > Does someone already used this library? Maybe she/he can share their > experience? > > /Frank > > Then you might want to try https://github.com/deadtrickster/prometheus.erl > > > > Regards, > > Ilya > > > > *From: *Frank Muller > *Sent: *Thursday, February 9, 2017 12:37 PM > *To: *Alexander Petrovsky ; Erlang-Questions Questions > ; spyl@REDACTED > *Subject: *Re: [erlang-questions] Send VM metrics to Grafana > > > Thanks for all the pointers guys. > > I've also heard about Prometheus (https://prometheus.io/). > > I like it because it's a single Go binary, very easy to deploy. > > Anyone has any experience with it and Grafana? > > /Frank > > > > Hello! > > There are bunch of projects for this, by example exometer, my metronome ( > https://github.com/juise/metronome), and so on. > > Just install influxdb and grafana > > ??, 9 ????. 2017 ?. ? 12:06, Frank Muller : > > > Hi everyone, > > Does anyone succeeded to push VM metrics to Grafana (http://grafana.org/). > > Here I'm looking for a simple integration with minimal dependencies if > possible. > > Thank you. > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > > Phone: +7 914 8 820 815 > > *_______________________________________________* > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tpodowd@REDACTED Wed Apr 19 07:51:38 2017 From: tpodowd@REDACTED (Tom O'Dowd) Date: Wed, 19 Apr 2017 14:51:38 +0900 Subject: [erlang-questions] exit status of run_erl Message-ID: I?m starting an application using systemd which uses run_erl in daemon mode. When I stop the application though by running init:stop(), run_erl seems to return an exit status of 1 rather than 0 so I always get a failed status for the service rather than an inactive status. I can set systemd SuccessExitStatus=1 to have it treat 1 as a success but then if for example the erlang node did die, it still gets marked as inactive rather than failed. Note that if I tell systemd that the MainPID is the pid of the beam process rather than run_erl, systemd spits out a warning Supervising process 96669 which is not our child. We'll most likely not notice when it exits. and sure enough, if you kill beam, systemd doesn?t notice. Main issue sees to be that run_erl doesn?t return the exiting VM status back as its own status. Currently using 19.2 although will move to 19.3 shortly. Any pointers? Been poking around on Google and this doesn?t seem like a common issue so maybe I am doing something wrong. Thanks, Tom. -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek@REDACTED Wed Apr 19 08:58:38 2017 From: abhishek@REDACTED (Abhishek Ranjan) Date: Wed, 19 Apr 2017 12:28:38 +0530 (IST) Subject: [erlang-questions] Extending the MUC so that a user sitting on diffrent PC can also join it Message-ID: <1855639338.26323.1492585118120@webmail.blacklightsw.com> Hi, I have created a multi-user chat room using ejabberd and Pidgin.The chatroom consists of three users, but my problem is that all the three user are accessed by me only and that too on from my PC. I want that people sitting on different PCs can also join my chatroom . Can Anybody tell me how to go about it. Regards, Abhishek Ranjan -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andras.Bekes@REDACTED Wed Apr 19 14:13:26 2017 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Wed, 19 Apr 2017 12:13:26 +0000 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> Message-ID: Did anyone have a chance to take a closer look? I still think that this is a bug in Erlang. From: Bekes, Andras G (IST) Sent: Tuesday, March 28, 2017 9:55 AM To: 'Alex S.' Cc: erlang-questions@REDACTED Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux. Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109. From: Alex S. [mailto:alex0player@REDACTED] Sent: Monday, March 27, 2017 8:00 PM To: Bekes, Andras G (IST) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes I have a strong suspicion the binary might be automatically zero-terminated. 27 ????? 2017 ?., ? 19:03, Bekes, Andras G > ???????(?): Hi All, I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). http://man7.org/linux/man-pages/man7/unix.7.html Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: > Sun_path_length=108. 108 > UDS_path = <<"whatever">>. <<"whatever">> > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). 792 > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,...>>} > Port = 0. 0 > gen_tcp:connect(Address,Port,[local],infinity). (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} ** exception exit: badarg in function gen_tcp:connect/4 (gen_tcp.erl, line 149) (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. Seems to me that this is a bug in the Erlang implementation. What do you think? Thank you very much, Andras G. Bekes _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor.clark@REDACTED Wed Apr 19 14:33:09 2017 From: igor.clark@REDACTED (Igor Clark) Date: Wed, 19 Apr 2017 13:33:09 +0100 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> <24009245-b22d-a245-fc71-c7c0cfcc8632@gmail.com> Message-ID: Hi folks, just checking in, I wonder if anyone has any further thoughts about this? To re-state: the identical Mac OS C API call inside a NIF succeeds (returns a valid string property) on R18/El Capitan, fails (returns null) inside the same NIF on R19/El Capitan & R18/R19/Sierra, yet succeeds in a minimal C program on both El Capitan (clang/LLVM 8.0.0) and Sierra (clang/LLVM 8.1.0). In all cases this happens regardless of whether the C API call is made inside the main thread or a separate thread. Would be really grateful for any suggestions, as this is way lower-level than I'm familiar with - happy to dig around further, but a bit lost where to look next. Thanks! Igor On 18/04/2017 07:54, Igor Clark wrote: > Sorry, to be clearer, that should have read "when run in a separate > thread in the NIF using enif_thread_create(), the API call succeeds > under R18 on El Capitan, fails under R19 on El Capitan, and fails > under R18 and R19 on Sierra". > > On Tue, Apr 18, 2017 at 7:52 AM, Igor Clark > wrote: > > Thanks Paul and Daniel, > > Just tested this: the exact same thing happens with threads as > without. The API call succeeds when run in a separate thread in > the minimal C program using pthread_create(), on both El Capitan > and Sierra; when run in a separate thread in the NIF using > enif_thread_create(), the API call succeeds under R18 on El > Capitan and fails under R19. > > (My NIF code hasn't been explicitly creating or using any threads > at all until now, if that makes any difference.) > > Does that give any further clues? > > Cheers, > Igor > > > On 18/04/2017 03:47, Daniel Goertzen wrote: >> Following Paul's idea, what happens in both the NIF and your >> minimal C program if you first launch a new thread and call the >> API from there? >> >> On Mon, Apr 17, 2017 at 5:52 PM Fisher, Paul >> > wrote: >> >> Complete wild guess, but is there a difference in nif >> initialization running off the main thread in non-working >> version? This would be a problem for some low-level Mach >> related things. Not near laptop to look through the code to >> see if that changed. >> >> >> Get Outlook for iOS >> ------------------------------------------------------------------------ >> *From:* erlang-questions-bounces@REDACTED >> >> > > on behalf of >> Igor Clark > >> *Sent:* Monday, April 17, 2017 4:38:11 PM >> *To:* erlang-questions@REDACTED >> >> *Subject:* [erlang-questions] NIF+device problem moving >> R18->R19 / Mac 10.11->10.12 >> Hi all, >> >> I'm having problems updating a NIF to work correctly with R19 >> and/or >> MacOS Sierra (10.12). The NIF uses Mac OS C API calls to talk >> to some >> hardware. It's been working just fine with R18 on El Capitan, >> but it >> fails in a manner I don't understand when I trying to run it >> on R19 or >> Sierra. I'm pretty baffled as to what's going on, and wonder >> if any of >> the following might ring bells or set off a spider-sense for >> anyone here? >> >> - The code continues to work 100% as expected on R18 / Mac OS >> El Capitan >> (10.11), as it has for the last year or so. >> >> - On R19 on El Capitan, 99% of the C code works as before, >> but one >> particular MacOS C API call fails unexpectedly. It doesn't >> crash or >> cause errors directly, but it returns null instead of a valid >> value as >> before. There are no build errors or warnings. (I'm using >> 'pc' plugin >> version 1.5.0 with rebar3). Other MacOS API calls talking to >> the same >> hardware work as expected, sending correct commands, getting the >> hardware to carry out operations as expected and returning >> expected >> values, without any problems. >> >> - On both R18 & R19 on Sierra, exactly the same problem >> occurs as above >> in R19 on El Capitan. >> >> - If I make the failing/null-returning OS API call in a plain, >> simple-as-possible C file compiled with gcc (Apple >> LLVM/clang), it works >> as expected, *on both OS versions*. >> >> - If I paste that same simple-as-possible C code verbatim >> into the NIF, >> as the first line in the load() function, and run it on >> R18/El Capitan, >> it continues to work as expected. >> >> - If I then try to run that identical pasted-verbatim NIF >> code with no >> changes on R18/Sierra or on R19 & either OS version, this one >> particular >> function call fails, as above. >> >> I realise it could be many things, and my initial thought was >> that it >> must be the OS change or even the hardware failing - but the >> hardware >> and C API call continue to work just fine on both OS versions >> when the >> NIF stuff is taken out of the picture, and on El Capitan, the >> previously-not-happening problem reliably (and only) starts >> happening >> when I switch from R18 to R19. >> >> Did anything change between R18 and R19 that might somehow >> make some, >> but not all, external C function calls fail on El >> Capitan/clang 8.0.0? >> And could that be something that also makes the same thing >> happen on >> both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps >> be something >> do with the port compiler & LD/CXX flags? >> >> Any ideas as to where else I could look to try to track this >> down, or >> what might be causing this? >> >> (BTW, I'm using homebrew erlang, and I've tried all the above >> with >> versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. The >> gcc/clang >> version on El Capitan is from Xcode 8.1, "Apple LLVM version >> 8.0.0 >> (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple >> LLVM >> version 8.1.0 (clang-802.0.41)" - but as above, the >> plain/non-NIF C code >> works perfectly and identically on both versions.) >> >> Thanks very much, >> Igor >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZWIKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= >> >> >> Confidentiality Notice | This email and any included >> attachments may be privileged, confidential and/or otherwise >> protected from disclosure. Access to this email by anyone >> other than the intended recipient is unauthorized. If you >> believe you have received this email in error, please contact >> the sender immediately and delete all copies. If you are not >> the intended recipient, you are notified that disclosing, >> copying, distributing or taking any action in reliance on the >> contents of this information is strictly prohibited. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Wed Apr 19 16:35:02 2017 From: grahamrhay@REDACTED (Graham Hay) Date: Wed, 19 Apr 2017 15:35:02 +0100 Subject: [erlang-questions] exit status of run_erl In-Reply-To: References: Message-ID: You can use Type=notify for your service, and then inform systemd of the main pid later (like a forking daemon, with a pidfile, once was). That's how rabbitmq works: https://github.com/binarin/rabbitmq-server/blob/master/src/rabbit.erl#L345 On 19 April 2017 at 06:51, Tom O'Dowd wrote: > I?m starting an application using systemd which uses run_erl in daemon > mode. When I stop the application though by running init:stop(), run_erl > seems to return an exit status of 1 rather than 0 so I always get a failed > status for the service rather than an inactive status. I can set > systemd SuccessExitStatus=1 to have it treat 1 as a success but then if for > example the erlang node did die, it still gets marked as inactive rather > than failed. > > Note that if I tell systemd that the MainPID is the pid of the beam > process rather than run_erl, systemd spits out a warning > Supervising process 96669 which is not our child. We'll most likely > not notice when it exits. > and sure enough, if you kill beam, systemd doesn?t notice. > > Main issue sees to be that run_erl doesn?t return the exiting VM status > back as its own status. Currently using 19.2 although will move to 19.3 > shortly. > > Any pointers? Been poking around on Google and this doesn?t seem like a > common issue so maybe I am doing something wrong. > > Thanks, > > Tom. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tpodowd@REDACTED Thu Apr 20 10:12:55 2017 From: tpodowd@REDACTED (Tom O'Dowd) Date: Thu, 20 Apr 2017 17:12:55 +0900 Subject: [erlang-questions] exit status of run_erl In-Reply-To: References: Message-ID: Oh. That?s an interesting idea. I will check that out. Thanks for the pointer. Tom. > On Apr 19, 2017, at 23:35, Graham Hay wrote: > > You can use Type=notify for your service, and then inform systemd of the main pid later (like a forking daemon, with a pidfile, once was). That's how rabbitmq works: > > https://github.com/binarin/rabbitmq-server/blob/master/src/rabbit.erl#L345 > > On 19 April 2017 at 06:51, Tom O'Dowd > wrote: > I?m starting an application using systemd which uses run_erl in daemon mode. When I stop the application though by running init:stop(), run_erl seems to return an exit status of 1 rather than 0 so I always get a failed status for the service rather than an inactive status. I can set systemd SuccessExitStatus=1 to have it treat 1 as a success but then if for example the erlang node did die, it still gets marked as inactive rather than failed. > > Note that if I tell systemd that the MainPID is the pid of the beam process rather than run_erl, systemd spits out a warning > Supervising process 96669 which is not our child. We'll most likely not notice when it exits. > and sure enough, if you kill beam, systemd doesn?t notice. > > Main issue sees to be that run_erl doesn?t return the exiting VM status back as its own status. Currently using 19.2 although will move to 19.3 shortly. > > Any pointers? Been poking around on Google and this doesn?t seem like a common issue so maybe I am doing something wrong. > > Thanks, > > Tom. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Thu Apr 20 12:32:34 2017 From: ali.sabil@REDACTED (Ali Sabil) Date: Thu, 20 Apr 2017 12:32:34 +0200 Subject: [erlang-questions] exit status of run_erl In-Reply-To: References: Message-ID: I personally avoid run_erl altogether, and use the "foreground" mode from the relx generated script. The systemd unit looks like this: [Unit] Description=My Server After=network.target auditd.service [Service] Type = simple ExecStart = /srv/my-server/bin/my-server foreground Restart = always User = my-server Group = my-server Environment = ERL_EPMD_ADDRESS=127.0.0.1 [Install] WantedBy = multi-user.target On Thu, Apr 20, 2017 at 10:12 AM, Tom O'Dowd wrote: > Oh. That?s an interesting idea. I will check that out. Thanks for the > pointer. > > Tom. > > On Apr 19, 2017, at 23:35, Graham Hay wrote: > > You can use Type=notify for your service, and then inform systemd of the > main pid later (like a forking daemon, with a pidfile, once was). That's > how rabbitmq works: > > https://github.com/binarin/rabbitmq-server/blob/master/src/rabbit.erl#L345 > > On 19 April 2017 at 06:51, Tom O'Dowd wrote: > >> I?m starting an application using systemd which uses run_erl in daemon >> mode. When I stop the application though by running init:stop(), run_erl >> seems to return an exit status of 1 rather than 0 so I always get a failed >> status for the service rather than an inactive status. I can set >> systemd SuccessExitStatus=1 to have it treat 1 as a success but then if for >> example the erlang node did die, it still gets marked as inactive rather >> than failed. >> >> Note that if I tell systemd that the MainPID is the pid of the beam >> process rather than run_erl, systemd spits out a warning >> Supervising process 96669 which is not our child. We'll most >> likely not notice when it exits. >> and sure enough, if you kill beam, systemd doesn?t notice. >> >> Main issue sees to be that run_erl doesn?t return the exiting VM status >> back as its own status. Currently using 19.2 although will move to 19.3 >> shortly. >> >> Any pointers? Been poking around on Google and this doesn?t seem like a >> common issue so maybe I am doing something wrong. >> >> Thanks, >> >> Tom. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peppe@REDACTED Thu Apr 20 13:03:19 2017 From: peppe@REDACTED (Peter Andersson E) Date: Thu, 20 Apr 2017 13:03:19 +0200 Subject: [erlang-questions] Common Test last test report In-Reply-To: <73d81ce9-ecbc-e823-36aa-e70d8a7366b3@ninenines.eu> References: <73d81ce9-ecbc-e823-36aa-e70d8a7366b3@ninenines.eu> Message-ID: On 02/25/2017 12:20 AM, Lo?c Hoguin wrote: > Hello, > > It's still difficult to get to the last test report in Common Test. > Currently the URI looks like this: > > file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@REDACTED/ninenines.cowboy.rfc7540_SUITE.logs/run.2017-02-25_00.02.29/suite.log.html > > > Which is great, I can always go back and compare, and sometimes that's > what I want (when I test across Erlang releases, for example). But > when I'm fixing a bug, I tend to run a small set of tests, maybe a > single suite, or sometimes even just a single group or a single test > case. When I do that, things becomes more difficult than they should > be because I can't simply refresh the page to get the new report, I > have to go back, refresh on the all_runs.html page, and follow the > links from there. > > Would it be possible for Common Test to maintain a separate directory > always containing the latest report? For example: > > file:///home/essen/ninenines/cowboy/logs/ct_run.ct_cowboy@REDACTED/ninenines.cowboy.rfc7540_SUITE.logs/run.latest/suite.log.html > > > With a URI like this I can repeat the test run, refresh in my browser > and get the new report instantly. (And keep the history separate > should I need to compare.) > > Alternatively, have an option to disable the timestamps so I can just > refresh when I need it, although the history wouldn't be preserved in > this case. > > Cheers, > Hi Lo?c + anyone else interested, I've made a solution to this problem as follows: In the top level log directory (i.e. where the top level index files, all_runs.html and index.html, are stored) an html file, suite.log.latest.html, is written at the start of each test run, which contains an redirection (immediate, without a click) to the latest saved suite.log.html (test case overview). There is a link to this redirection page in the footer of suite.log.html. This makes it possible to jump immediately to the latest test results from any test case overview page without having to navigate to the top level index first. So this is useful when you're running the same tests repeatedly and want to update and/or compare results. The shortcoming of this design, however, is that it ignores the test name. It's the latest report in 'logdir' that is jumped to, not the latest report for the current set of tests (like e.g. what you get in the top level index.html page). Example: If you run a set of tests, A, then repeat this set, A', then run a new set of tests, B, then if you're analyzing the results of A and click the latest results link, you're redirected to the report for B, not A'. The workaround in the above example is either to use different top level log directories for A and B test runs, or to navigate from suite.log.html for A to the top level index.html (via the link in the footer) and go to A' from there. My question is, what do you guys prefer? Is the current implementation as I descibe it sufficient and useful, or would a more complex solution with "name scoped redirections" make more sense? I'd appreciate the input! Best, Peter Ericsson AB, Erlang/OTP From jargon@REDACTED Thu Apr 20 20:17:35 2017 From: jargon@REDACTED (Johannes =?utf-8?B?V2Vpw59s?=) Date: Thu, 20 Apr 2017 20:17:35 +0200 Subject: [erlang-questions] Deprecation and removal of the non-smp emulator In-Reply-To: References: Message-ID: <20170420181735.byv7h4xk5qmg2ywv@molb.org> Hello Lukas, > If you have a very good reason for wanting to use the non-smp > emulator, please do let us know as soon as possible so that we can > consider whether we want to keep the non-smp version anyway. We use the non-smp emulator on small ARM Linux boxes (single CPU). I have tested the smp emulator over the last two weeks, here are my observations: - smp version increases VmSize by 10-35 MB, but VmRSS by only about 1 MB - smp version is slower, but not critically (can still run our software) - qemu-arm-static 2.8 has problems running the smp version So we will stick to the non-smp version until OTP-21, especially because of the broken qemu (used for test suite). Johannes From igor.clark@REDACTED Fri Apr 21 01:12:20 2017 From: igor.clark@REDACTED (Igor Clark) Date: Fri, 21 Apr 2017 00:12:20 +0100 Subject: [erlang-questions] NIF+device problem moving R18->R19 / Mac 10.11->10.12 In-Reply-To: References: <3b109312-7f0b-65ec-0710-90a6664d81e5@gmail.com> <24009245-b22d-a245-fc71-c7c0cfcc8632@gmail.com> Message-ID: <7d34c954-3fc8-1674-7755-502d112bdedb@gmail.com> Quick update - looks like it's not the hardware API at all, and I guess not erlang either, but Objective-C/Foundation stuff related to how I was initialising CFStringRef values whose addresses I passed to the API call. I was using a CFSTR macro that I took from an Apple code sample; now I'm using a CFString function to do it and it's working fine across the erlang & OS versions. I'm a bit baffled as to why the macro was working at all on R18, and why it still works in the minimal C program version, as it looks to me like it returns an immutable string ref; maybe it shouldn't work, but like I say, I took it from an Apple sample. I plan to look into that a bit more deeply. I'm also pretty baffled as to why the different erlang & OS/compiler versions made this manifest, when it wasn't previously - but regardless, taking it out seems to have side-stepped the problem. Thanks everyone, best, Igor On 19/04/2017 13:33, Igor Clark wrote: > Hi folks, just checking in, I wonder if anyone has any further > thoughts about this? > > To re-state: the identical Mac OS C API call inside a NIF succeeds > (returns a valid string property) on R18/El Capitan, fails (returns > null) inside the same NIF on R19/El Capitan & R18/R19/Sierra, yet > succeeds in a minimal C program on both El Capitan (clang/LLVM 8.0.0) > and Sierra (clang/LLVM 8.1.0). In all cases this happens regardless of > whether the C API call is made inside the main thread or a separate > thread. > > Would be really grateful for any suggestions, as this is way > lower-level than I'm familiar with - happy to dig around further, but > a bit lost where to look next. > > Thanks! > Igor > > On 18/04/2017 07:54, Igor Clark wrote: >> Sorry, to be clearer, that should have read "when run in a separate >> thread in the NIF using enif_thread_create(), the API call succeeds >> under R18 on El Capitan, fails under R19 on El Capitan, and fails >> under R18 and R19 on Sierra". >> >> On Tue, Apr 18, 2017 at 7:52 AM, Igor Clark > > wrote: >> >> Thanks Paul and Daniel, >> >> Just tested this: the exact same thing happens with threads as >> without. The API call succeeds when run in a separate thread in >> the minimal C program using pthread_create(), on both El Capitan >> and Sierra; when run in a separate thread in the NIF using >> enif_thread_create(), the API call succeeds under R18 on El >> Capitan and fails under R19. >> >> (My NIF code hasn't been explicitly creating or using any threads >> at all until now, if that makes any difference.) >> >> Does that give any further clues? >> >> Cheers, >> Igor >> >> >> On 18/04/2017 03:47, Daniel Goertzen wrote: >>> Following Paul's idea, what happens in both the NIF and your >>> minimal C program if you first launch a new thread and call the >>> API from there? >>> >>> On Mon, Apr 17, 2017 at 5:52 PM Fisher, Paul >>> > wrote: >>> >>> Complete wild guess, but is there a difference in nif >>> initialization running off the main thread in non-working >>> version? This would be a problem for some low-level Mach >>> related things. Not near laptop to look through the code to >>> see if that changed. >>> >>> >>> Get Outlook for iOS >>> ------------------------------------------------------------------------ >>> *From:* erlang-questions-bounces@REDACTED >>> >>> >> > on behalf of >>> Igor Clark > >>> *Sent:* Monday, April 17, 2017 4:38:11 PM >>> *To:* erlang-questions@REDACTED >>> >>> *Subject:* [erlang-questions] NIF+device problem moving >>> R18->R19 / Mac 10.11->10.12 >>> Hi all, >>> >>> I'm having problems updating a NIF to work correctly with >>> R19 and/or >>> MacOS Sierra (10.12). The NIF uses Mac OS C API calls to >>> talk to some >>> hardware. It's been working just fine with R18 on El >>> Capitan, but it >>> fails in a manner I don't understand when I trying to run it >>> on R19 or >>> Sierra. I'm pretty baffled as to what's going on, and wonder >>> if any of >>> the following might ring bells or set off a spider-sense for >>> anyone here? >>> >>> - The code continues to work 100% as expected on R18 / Mac >>> OS El Capitan >>> (10.11), as it has for the last year or so. >>> >>> - On R19 on El Capitan, 99% of the C code works as before, >>> but one >>> particular MacOS C API call fails unexpectedly. It doesn't >>> crash or >>> cause errors directly, but it returns null instead of a >>> valid value as >>> before. There are no build errors or warnings. (I'm using >>> 'pc' plugin >>> version 1.5.0 with rebar3). Other MacOS API calls talking to >>> the same >>> hardware work as expected, sending correct commands, getting >>> the >>> hardware to carry out operations as expected and returning >>> expected >>> values, without any problems. >>> >>> - On both R18 & R19 on Sierra, exactly the same problem >>> occurs as above >>> in R19 on El Capitan. >>> >>> - If I make the failing/null-returning OS API call in a plain, >>> simple-as-possible C file compiled with gcc (Apple >>> LLVM/clang), it works >>> as expected, *on both OS versions*. >>> >>> - If I paste that same simple-as-possible C code verbatim >>> into the NIF, >>> as the first line in the load() function, and run it on >>> R18/El Capitan, >>> it continues to work as expected. >>> >>> - If I then try to run that identical pasted-verbatim NIF >>> code with no >>> changes on R18/Sierra or on R19 & either OS version, this >>> one particular >>> function call fails, as above. >>> >>> I realise it could be many things, and my initial thought >>> was that it >>> must be the OS change or even the hardware failing - but the >>> hardware >>> and C API call continue to work just fine on both OS >>> versions when the >>> NIF stuff is taken out of the picture, and on El Capitan, the >>> previously-not-happening problem reliably (and only) starts >>> happening >>> when I switch from R18 to R19. >>> >>> Did anything change between R18 and R19 that might somehow >>> make some, >>> but not all, external C function calls fail on El >>> Capitan/clang 8.0.0? >>> And could that be something that also makes the same thing >>> happen on >>> both R18 and R19 under Sierra/clang 8.1.0? Could it perhaps >>> be something >>> do with the port compiler & LD/CXX flags? >>> >>> Any ideas as to where else I could look to try to track this >>> down, or >>> what might be causing this? >>> >>> (BTW, I'm using homebrew erlang, and I've tried all the >>> above with >>> versions 18.1, 18.3, 19.0.2 and 19.3 on both OS versions. >>> The gcc/clang >>> version on El Capitan is from Xcode 8.1, "Apple LLVM version >>> 8.0.0 >>> (clang-800.0.42.1)". On Sierra it's from Xcode 8.3.1, "Apple >>> LLVM >>> version 8.1.0 (clang-802.0.41)" - but as above, the >>> plain/non-NIF C code >>> works perfectly and identically on both versions.) >>> >>> Thanks very much, >>> Igor >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwICAg&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=PevFox_7LK44ZV_jlS5jRM2WItKtsHV4zN_CrbdT2aM&m=VYIypVbNEBkrDndUBzmlPhRtnzb4ZWIKbuytw-ZNFsQ&s=jm04c9Iw58-WmaHYaGFw-a9XQ3sFl2wqkf7RhdBDC6E&e= >>> >>> >>> Confidentiality Notice | This email and any included >>> attachments may be privileged, confidential and/or otherwise >>> protected from disclosure. Access to this email by anyone >>> other than the intended recipient is unauthorized. If you >>> believe you have received this email in error, please >>> contact the sender immediately and delete all copies. If you >>> are not the intended recipient, you are notified that >>> disclosing, copying, distributing or taking any action in >>> reliance on the contents of this information is strictly >>> prohibited. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stafil@REDACTED Thu Apr 20 22:34:45 2017 From: stafil@REDACTED (Stavros Filargyropoulos) Date: Thu, 20 Apr 2017 13:34:45 -0700 Subject: [erlang-questions] TCP client Message-ID: Hey, Just starting out on erlang, and trying to implement a message broker using erlang. I have a gen_fsm process for each client that handles the command and data receiving and I was wondering what is the best way to for the main broker process to send data to the client. Do I just send a message to the client process and handle the sending inside `handle_info`? How would I handle race conditions in this case? E.g. When the client process terminates just after the main process sent the message. My code looks something like this at the moment: -module(pq_test). -export([init/1,handle_info/3,handle_event/3]). -export([start/0]). -export([recv_cmd/2,recv_data/2]). -record(state, {socket, buffer, current_command}). start() -> spawn_link(fun start_link/0). start_link() -> {ok, ListenSocket} = gen_tcp:listen(11880, [{active, false}, binary]), accept(ListenSocket). accept(ListenSocket) -> {ok, Socket} = gen_tcp:accept(ListenSocket), {ok, Pid} = gen_fsm:start_link({local, ?MODULE}, ?MODULE, Socket, []), erlang:display(Pid), gen_tcp:controlling_process(Socket, Pid), accept(ListenSocket). init(Socket)-> inet:setopts(Socket, [{active, true}]), {ok, recv_cmd, #state{socket = Socket, buffer = <<"">>, current_command = null}}. cmd_parse(publish, [Topic, Key, Length], #state{socket = Socket})-> {Len, _} = string:to_integer(Length), erlang:display(Len), io:format("Publish -~s- -~s- -~B-~n", [Topic, Key, Len]), {next_state, recv_data, #state{socket = Socket, buffer = <<"">>, current_command = {publish, Len}}}; cmd_parse(_, _, #state{socket = Socket}) -> io:format("Unknown command~n"), {next_state, recv_cmd, #state{socket = Socket, buffer = <<"">>, current_command = null}}. cmd_process(CommandLine, #state{socket = Socket} = State) -> io:format("1~n"), case string:tokens(binary_to_list(CommandLine), [$\s, $\r, $\n]) of [Command | Args] -> cmd_parse(list_to_atom(Command), Args, State); _ -> {next_state, recv_cmd, #state{socket = Socket, buffer = <<"">>, current_command = null}} end. cmd_more(CommandLine, #state{socket = Socket}) -> {next_state, recv_cmd, #state{socket=Socket, buffer=CommandLine, current_command = null}}. recv_cmd({data, Data}, #state{buffer = Buffer} = State) -> CommandLine = <>, Last = binary:last(CommandLine), case Last of $\n -> cmd_process(CommandLine, State); _ -> cmd_more(CommandLine, State) end. recv_data({data, Data}, #state{socket = Socket, buffer = Buffer, current_command = {publish, Len}}) when Len > byte_size(Data) -> {next_state, recv_data, #state{socket = Socket, buffer = <>, current_command = {publish, Len - byte_size(Data)}}}; recv_data({data, Data}, #state{socket = Socket, buffer = Buffer}) -> io:format("Got all data: ~s~n", [<>]), {next_state, recv_cmd, #state{socket = Socket, buffer = <<"">>, current_command = null}}. handle_info({tcp, _, Message}, StateName, State) -> ?MODULE:StateName({data, Message}, State); handle_info(Event, StateName, State) -> io:format("handle info~n"), erlang:display(Event), {next_state, StateName, State}. handle_event(_, _, _) -> io:format("handle event"). -------------- next part -------------- An HTML attachment was scrubbed... URL: From arunp@REDACTED Fri Apr 21 08:00:29 2017 From: arunp@REDACTED (Arun) Date: Fri, 21 Apr 2017 11:30:29 +0530 Subject: [erlang-questions] Mnesia table is not available after restart Message-ID: <58F99FFD.7010200@utl.in> Hi, I am running an application on a single node and the application creates some *ram copies* tables on the same node, but the problem what I am facing is that, if I restart my system suddenly after successful creation of all tables, in the next start the table which got created last is missing, but if I give some delay after creation of tables and restart the system all tables will be available in the next start. In mnesia reference manual and users guide I could not find a proper solution to this issue, can somebody kindly explain what is the reason behind this behavior and is there any solution for this ?. Will the function mnesia:dump_log/0 be helpful ? Regards, Arun P -------------- next part -------------- An HTML attachment was scrubbed... URL: From Maxim.Minin@REDACTED Fri Apr 21 08:53:35 2017 From: Maxim.Minin@REDACTED (Minin Maxim) Date: Fri, 21 Apr 2017 06:53:35 +0000 Subject: [erlang-questions] Mnesia table is not available after restart In-Reply-To: <58F99FFD.7010200@utl.in> References: <58F99FFD.7010200@utl.in> Message-ID: <554dd866cf5044dfbf23f261c085c913@sv2085.baag> see - http://erlang.org/doc/man/mnesia.html#wait_for_tables-2 Cheers Maxim Von: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] Im Auftrag von Arun Gesendet: Freitag, 21. April 2017 08:00 An: erlang-questions@REDACTED Betreff: [erlang-questions] Mnesia table is not available after restart Hi, I am running an application on a single node and the application creates some ram copies tables on the same node, but the problem what I am facing is that, if I restart my system suddenly after successful creation of all tables, in the next start the table which got created last is missing, but if I give some delay after creation of tables and restart the system all tables will be available in the next start. In mnesia reference manual and users guide I could not find a proper solution to this issue, can somebody kindly explain what is the reason behind this behavior and is there any solution for this ?. Will the function mnesia:dump_log/0 be helpful ? Regards, Arun P -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5536 bytes Desc: not available URL: From arunp@REDACTED Fri Apr 21 11:58:06 2017 From: arunp@REDACTED (Arun) Date: Fri, 21 Apr 2017 15:28:06 +0530 Subject: [erlang-questions] Mnesia table is not available after restart In-Reply-To: <554dd866cf5044dfbf23f261c085c913@sv2085.baag> References: <58F99FFD.7010200@utl.in> <554dd866cf5044dfbf23f261c085c913@sv2085.baag> Message-ID: <58F9D7AE.9090903@utl.in> Hi Maxim, I already tried wait for tables, but it gives me time out. I strongly believe that in schema the information related to the table is getting missed or corrupted, can this be possible.? Regards, Arun P On Friday 21 April 2017 12:23 PM, Minin Maxim wrote: > > see - http://erlang.org/doc/man/mnesia.html#wait_for_tables-2 > > Cheers > > Maxim > > *Von:*erlang-questions-bounces@REDACTED > [mailto:erlang-questions-bounces@REDACTED] *Im Auftrag von *Arun > *Gesendet:* Freitag, 21. April 2017 08:00 > *An:* erlang-questions@REDACTED > *Betreff:* [erlang-questions] Mnesia table is not available after restart > > Hi, > > I am running an application on a single node and the application > creates some *ram copies* tables on the same node, but the problem > what I am facing is that, if I restart my system suddenly after > successful creation of all tables, in the next start the table which > got created last is missing, but if I give some delay after creation > of tables and restart the system all tables will be available in the > next start. In mnesia reference manual and users guide I could not > find a proper solution to this issue, can somebody kindly explain what > is the reason behind this behavior and is there any solution for this > ?. Will the function mnesia:dump_log/0 be helpful ? > > Regards, > Arun P > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Fri Apr 21 12:02:41 2017 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 21 Apr 2017 10:02:41 +0000 Subject: [erlang-questions] Mnesia table is not available after restart In-Reply-To: <58F9D7AE.9090903@utl.in> References: <58F99FFD.7010200@utl.in> <554dd866cf5044dfbf23f261c085c913@sv2085.baag> <58F9D7AE.9090903@utl.in> Message-ID: You are never guaranteed that every operation is written disc before quitting abruptly, dump_log() or mnesia:sync_log() may help or more nodes so you can handle single node failure. On Fri, Apr 21, 2017 at 11:58 AM Arun wrote: > Hi Maxim, > > I already tried wait for tables, but it gives me time out. I strongly > believe that in schema the information related to the table is getting > missed or corrupted, can this be possible.? > > Regards, > Arun P > > > On Friday 21 April 2017 12:23 PM, Minin Maxim wrote: > > see - http://erlang.org/doc/man/mnesia.html#wait_for_tables-2 > > > > Cheers > > Maxim > > > > *Von:* erlang-questions-bounces@REDACTED [ > mailto:erlang-questions-bounces@REDACTED > ] *Im Auftrag von *Arun > *Gesendet:* Freitag, 21. April 2017 08:00 > *An:* erlang-questions@REDACTED > *Betreff:* [erlang-questions] Mnesia table is not available after restart > > > > Hi, > > I am running an application on a single node and the application creates > some * ram copies* tables on the same node, but the problem what I am > facing is that, if I restart my system suddenly after successful creation > of all tables, in the next start the table which got created last is > missing, but if I give some delay after creation of tables and restart the > system all tables will be available in the next start. In mnesia reference > manual and users guide I could not find a proper solution to this issue, > can somebody kindly explain what is the reason behind this behavior and is > there any solution for this ?. Will the function mnesia:dump_log/0 be > helpful ? > > Regards, > Arun P > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Apr 21 16:52:18 2017 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 21 Apr 2017 16:52:18 +0200 Subject: [erlang-questions] TCP client In-Reply-To: References: Message-ID: Hi! gen_fsm will be deprecated in OTP 20, so I recommend using gen_statem instead. In gen_statem you can have the semantics of selective receive by using postpone. Regards Ingela 2017-04-20 22:34 GMT+02:00 Stavros Filargyropoulos : > Hey, > > Just starting out on erlang, and trying to implement a message broker > using erlang. I have a gen_fsm process for each client that handles the > command and data receiving and I was wondering what is the best way to for > the main broker process to send data to the client. > > Do I just send a message to the client process and handle the sending > inside `handle_info`? How would I handle race conditions in this case? E.g. > When the client process terminates just after the main process sent the > message. > > My code looks something like this at the moment: > > -module(pq_test). > -export([init/1,handle_info/3,handle_event/3]). > -export([start/0]). > -export([recv_cmd/2,recv_data/2]). > > -record(state, {socket, buffer, current_command}). > > start() -> > spawn_link(fun start_link/0). > > start_link() -> > {ok, ListenSocket} = gen_tcp:listen(11880, [{active, false}, binary]), > accept(ListenSocket). > > accept(ListenSocket) -> > {ok, Socket} = gen_tcp:accept(ListenSocket), > {ok, Pid} = gen_fsm:start_link({local, ?MODULE}, ?MODULE, Socket, []), > erlang:display(Pid), > gen_tcp:controlling_process(Socket, Pid), > accept(ListenSocket). > > init(Socket)-> > inet:setopts(Socket, [{active, true}]), > {ok, recv_cmd, > #state{socket = Socket, buffer = <<"">>, current_command = null}}. > > cmd_parse(publish, [Topic, Key, Length], #state{socket = Socket})-> > {Len, _} = string:to_integer(Length), > erlang:display(Len), > io:format("Publish -~s- -~s- -~B-~n", [Topic, Key, Len]), > {next_state, recv_data, > #state{socket = Socket, buffer = <<"">>, current_command = {publish, > Len}}}; > cmd_parse(_, _, #state{socket = Socket}) -> > io:format("Unknown command~n"), > {next_state, recv_cmd, > #state{socket = Socket, buffer = <<"">>, current_command = null}}. > > cmd_process(CommandLine, #state{socket = Socket} = State) -> > io:format("1~n"), > case string:tokens(binary_to_list(CommandLine), [$\s, $\r, $\n]) of > [Command | Args] -> > cmd_parse(list_to_atom(Command), Args, State); > _ -> {next_state, recv_cmd, > #state{socket = Socket, buffer = <<"">>, current_command = null}} > end. > > cmd_more(CommandLine, #state{socket = Socket}) -> > {next_state, recv_cmd, > #state{socket=Socket, buffer=CommandLine, current_command = null}}. > > recv_cmd({data, Data}, #state{buffer = Buffer} = State) -> > CommandLine = <>, > Last = binary:last(CommandLine), > case Last of > $\n -> cmd_process(CommandLine, State); > _ -> cmd_more(CommandLine, State) > end. > > recv_data({data, Data}, > #state{socket = Socket, > buffer = Buffer, > current_command = {publish, Len}}) when Len > byte_size(Data) -> > {next_state, recv_data, > #state{socket = Socket, buffer = <>, > current_command = {publish, Len - byte_size(Data)}}}; > recv_data({data, Data}, #state{socket = Socket, buffer = Buffer}) -> > io:format("Got all data: ~s~n", [<>]), > {next_state, recv_cmd, > #state{socket = Socket, buffer = <<"">>, current_command = null}}. > > handle_info({tcp, _, Message}, StateName, State) -> > ?MODULE:StateName({data, Message}, State); > handle_info(Event, StateName, State) -> > io:format("handle info~n"), > erlang:display(Event), > {next_state, StateName, State}. > > handle_event(_, _, _) -> > io:format("handle event"). > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Fri Apr 21 19:12:27 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Fri, 21 Apr 2017 17:12:27 +0000 Subject: [erlang-questions] Core Erlang apply target expression behaviour Message-ID: Whilst trying to compile some code to core erlang I came across a problem when the target expression (e0 in spec) to apply was another apply expression. According to the spec I thought that would be ok however when compiling the .core file with erlc I got an "no_file: Warning: invalid function call" error. What I then tried, not thinking it would work, was to wrap the inner apply expression in a let expression. This to my surprise worked just fine. I've included the code below. 'addSix` is the working function and `addSix2` is the dodgy one. Am I doing something wrong in `addSix2` to in terms of how I print my AST or is it simply that `addSix2` cannot be made to work? I also tried putting some parens around it but that also didn't work. 'add'/2 = fun (_a0,_b0) -> call 'erlang':'+'(_a0,_b0) 'addFive'/0 = fun () -> let <_a0> = 5 in fun (_b0) -> apply 'add'/2 (_a0,_b0) 'addSix'/1 = fun (_x0) -> call 'erlang':'+'(let <_fez0> = apply 'addFive'/0 () in apply _fez0 (_x0),1) 'addSix2'/1 = fun (_x0) -> call 'erlang':'+'(apply apply 'addFive'/0 () (_x0),1) Cheers Karl -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Sat Apr 22 09:23:14 2017 From: fxn@REDACTED (Xavier Noria) Date: Sat, 22 Apr 2017 09:23:14 +0200 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: Does the Erlang compiler go from the Erlang AST to Core Erlang source code as printed by +to_core? Or does it just manipulate the structure to convert it directly into Core Erlang AST? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sat Apr 22 09:56:33 2017 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sat, 22 Apr 2017 09:56:33 +0200 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: Here is the list of passes starting with the Erlang AST: https://github.com/erlang/otp/blob/52744948f4bbe85dee428cefdf9d205665a57b18/lib/compiler/src/compile.erl#L656 The passes that emit files based on the command line flags are in the format {iff, to_pp, {done, "P"}}, such as this one: https://github.com/erlang/otp/blob/52744948f4bbe85dee428cefdf9d205665a57b18/lib/compiler/src/compile.erl#L668 The pass that converts to core is in the same function. You can see that it has an option to emit -to_core0 via the command line. So that would be the format closest to your question: https://github.com/erlang/otp/blob/52744948f4bbe85dee428cefdf9d205665a57b18/lib/compiler/src/compile.erl#L679-L681 The format emitted by to_core is the one emitted after optimizations, inlining, etc: https://github.com/erlang/otp/blob/52744948f4bbe85dee428cefdf9d205665a57b18/lib/compiler/src/compile.erl#L699 You may continue following the passes to see what happens for the other formats as well. :) *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Sat, Apr 22, 2017 at 9:23 AM, Xavier Noria wrote: > Does the Erlang compiler go from the Erlang AST to Core Erlang source code > as printed by +to_core? Or does it just manipulate the structure to convert > it directly into Core Erlang AST? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fxn@REDACTED Sat Apr 22 10:13:48 2017 From: fxn@REDACTED (Xavier Noria) Date: Sat, 22 Apr 2017 10:13:48 +0200 Subject: [erlang-questions] Core erlang definition In-Reply-To: References: Message-ID: ?? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sat Apr 22 21:19:37 2017 From: rvirding@REDACTED (Robert Virding) Date: Sat, 22 Apr 2017 21:19:37 +0200 Subject: [erlang-questions] Core Erlang apply target expression behaviour In-Reply-To: References: Message-ID: I would try doing 2 things: - Write the equivalent Erlang code which does what you want the code to do, compile it with the to_core0 option and see what it generates. This then becomes your target code. This is how I did it with LFE. - Or you can try flattening out your code to remove the nested calls. While Core erlang in principle should be able to manage nested code the Erlang compiler generally flattens things. What I found when doing LFE was that while the language *in principle* allows a lot of things *in practice* later passes in the compiler, for example the optimisation passes, assume that the code looks like what the Erlang compiler compiler generates. One example of this is how literal structures are represented. to_core0 returns the Core erlang which has been generated by the erlang->core conversion pass which is then what is passed on to optimisation and and code generation. This is what you should be targeting. To check what the actual Core erlang data structures look like, not the pretty printed code, in the shell try doing: > c("foo", [binary,to_core0,return]). This call will *return* the actual data structure so you can put it in variable then pretty print it to a file so you can what you really need to get. Note that some of the attributes are important in later passes. What language are you working on? Robert On 21 April 2017 at 19:12, Karl Nilsson wrote: > Whilst trying to compile some code to core erlang I came across a problem > when the target expression (e0 in spec) to apply was another apply > expression. According to the spec I thought that would be ok however when > compiling the .core file with erlc I got an "no_file: Warning: invalid > function call" error. > > What I then tried, not thinking it would work, was to wrap the inner apply > expression in a let expression. This to my surprise worked just fine. I've > included the code below. 'addSix` is the working function and `addSix2` is > the dodgy one. Am I doing something wrong in `addSix2` to in terms of how I > print my AST or is it simply that `addSix2` cannot be made to work? I also > tried putting some parens around it but that also didn't work. > > 'add'/2 = > fun (_a0,_b0) -> > call 'erlang':'+'(_a0,_b0) > > 'addFive'/0 = > fun () -> > let <_a0> = 5 > in fun (_b0) -> > apply 'add'/2 (_a0,_b0) > > 'addSix'/1 = > fun (_x0) -> > call 'erlang':'+'(let <_fez0> = apply 'addFive'/0 () > in apply _fez0 (_x0),1) > > 'addSix2'/1 = > fun (_x0) -> > call 'erlang':'+'(apply apply 'addFive'/0 () (_x0),1) > > > Cheers > Karl > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sat Apr 22 21:23:50 2017 From: rvirding@REDACTED (Robert Virding) Date: Sat, 22 Apr 2017 21:23:50 +0200 Subject: [erlang-questions] =?utf-8?b?562U5aSNOiAgSXMgaXQgYSBjb21waWxlciBi?= =?utf-8?b?dWc/?= In-Reply-To: References: <16d77a7ee12d48c7a61ed60d585e8557@sv2085.baag> Message-ID: There is no *need* of it as such it just what the syntax allows. I think we should require the parentheses again and make the current usage a syntax error. Robert On 14 April 2017 at 04:24, ? ? wrote: > I still don?t get the sense , What is the need for such weird appending > use? > > > > ??? Windows 10 ??? ?? > > > > *???: *Robert Virding > *????: *2017?4?14? 8:15 > *???: *Dmytro Lytovchenko > *??: *Erlang Questions ; Stanislaw Klekot > > *??: *Re: [erlang-questions] Is it a compiler bug? > > > > IIRC originally parentheses were required around the first was required > which would have caught this case. The requirement was then removed, but I > don't know why. Perhaps to save the extra 2 chars. :-) > > > > On 13 April 2017 at 16:43, Dmytro Lytovchenko < > dmytro.lytovchenko@REDACTED> wrote: > > I can see how enforcing parentheses in map()map() syntax can be useful. > > A compiler warning would be awesome here. Also this is a nice thing for > Elvis style checker to do, but when it comes to running Elvis, your program > should already be correct ? so that is too late. > > > > In variable()map() syntax there is no confusion so that is fine. > > > > 2017-04-13 16:14 GMT+02:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > > No, it is not stupidity in any way. > > > > People hit this now and again. It makes sense to solve the problem this > way. I remember we considered the alternative, which is to reject the > notion, but this requires some special-handling in the compiler and isn't > clear-cut either. > > > > In short, regarding this as an invalid expression is to a certain extent > possible, and certainly desirable. But we run into subtle problems when we > want to reject it too, which is what complicates matters. > > > > I think it is healthy to challenge certain design decisions in Erlang, > especially because dynamically typed languages tend to have corner cases in > their semantics which are hard to handle in general. > > > > On Thu, Apr 13, 2017 at 4:04 PM Minin Maxim > wrote: > > I'm so stupid today J > > Thanks guys (Robert & Stanislaw) > > Cheers > > Maxim > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Sat Apr 22 21:58:46 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Sat, 22 Apr 2017 21:58:46 +0200 Subject: [erlang-questions] adding compile pass Message-ID: can one add custom compile pass when compiling module?? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Apr 23 02:07:30 2017 From: rvirding@REDACTED (Robert Virding) Date: Sun, 23 Apr 2017 02:07:30 +0200 Subject: [erlang-questions] Core Erlang apply target expression behaviour In-Reply-To: References: Message-ID: I have been thinking about f# as well, but from a different direction. I would want to use as much as possible of the existing f# compiler which is written in f# so I would write an interpreter in Erlang for AST generated by f# parser which you could then use that to run all the early parts of the compiler, like type-checking and such. Then hopefully you could replace the backend with a new bit which generates core erlang and use the erlang compiler for the rest. I fully realise it would not be as simple as it sounds :-) but I think it would be a doable of reusing as much as possible of what is written in f#. For me time is the limiting factor. Robert On 22 April 2017 at 21:52, Karl Nilsson wrote: > Thanks for the advice. A few things to think about there. > I have been writing things in erlang, compiling and comparing but rather > than writing a language from scratch I am working from the AST (or rather > the TAST) from another language, fsharp. As fsharp is an ML and core erlang > is ML-ish I was hoping to avoid having to do too much mangling but > flattening out applies and mod calls with let expressions is easy enough so > I'll definitely do that. > > Cheers > Karl > > On Sat, 22 Apr 2017 at 20:19 Robert Virding wrote: > >> I would try doing 2 things: >> >> - Write the equivalent Erlang code which does what you want the code to >> do, compile it with the to_core0 option and see what it generates. This >> then becomes your target code. This is how I did it with LFE. >> >> - Or you can try flattening out your code to remove the nested calls. >> While Core erlang in principle should be able to manage nested code the >> Erlang compiler generally flattens things. >> >> What I found when doing LFE was that while the language *in principle* >> allows a lot of things *in practice* later passes in the compiler, for >> example the optimisation passes, assume that the code looks like what the >> Erlang compiler compiler generates. One example of this is how literal >> structures are represented. >> >> to_core0 returns the Core erlang which has been generated by the >> erlang->core conversion pass which is then what is passed on to >> optimisation and and code generation. This is what you should be targeting. >> >> To check what the actual Core erlang data structures look like, not the >> pretty printed code, in the shell try doing: >> >> > c("foo", [binary,to_core0,return]). >> >> This call will *return* the actual data structure so you can put it in >> variable then pretty print it to a file so you can what you really need to >> get. Note that some of the attributes are important in later passes. >> >> What language are you working on? >> >> Robert >> >> >> On 21 April 2017 at 19:12, Karl Nilsson wrote: >> >>> Whilst trying to compile some code to core erlang I came across a >>> problem when the target expression (e0 in spec) to apply was another apply >>> expression. According to the spec I thought that would be ok however when >>> compiling the .core file with erlc I got an "no_file: Warning: invalid >>> function call" error. >>> >>> What I then tried, not thinking it would work, was to wrap the inner >>> apply expression in a let expression. This to my surprise worked just fine. >>> I've included the code below. 'addSix` is the working function and >>> `addSix2` is the dodgy one. Am I doing something wrong in `addSix2` to in >>> terms of how I print my AST or is it simply that `addSix2` cannot be made >>> to work? I also tried putting some parens around it but that also didn't >>> work. >>> >>> 'add'/2 = >>> fun (_a0,_b0) -> >>> call 'erlang':'+'(_a0,_b0) >>> >>> 'addFive'/0 = >>> fun () -> >>> let <_a0> = 5 >>> in fun (_b0) -> >>> apply 'add'/2 (_a0,_b0) >>> >>> 'addSix'/1 = >>> fun (_x0) -> >>> call 'erlang':'+'(let <_fez0> = apply 'addFive'/0 () >>> in apply _fez0 (_x0),1) >>> >>> 'addSix2'/1 = >>> fun (_x0) -> >>> call 'erlang':'+'(apply apply 'addFive'/0 () (_x0),1) >>> >>> >>> Cheers >>> Karl >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Sat Apr 22 21:52:27 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Sat, 22 Apr 2017 19:52:27 +0000 Subject: [erlang-questions] Core Erlang apply target expression behaviour In-Reply-To: References: Message-ID: Thanks for the advice. A few things to think about there. I have been writing things in erlang, compiling and comparing but rather than writing a language from scratch I am working from the AST (or rather the TAST) from another language, fsharp. As fsharp is an ML and core erlang is ML-ish I was hoping to avoid having to do too much mangling but flattening out applies and mod calls with let expressions is easy enough so I'll definitely do that. Cheers Karl On Sat, 22 Apr 2017 at 20:19 Robert Virding wrote: > I would try doing 2 things: > > - Write the equivalent Erlang code which does what you want the code to > do, compile it with the to_core0 option and see what it generates. This > then becomes your target code. This is how I did it with LFE. > > - Or you can try flattening out your code to remove the nested calls. > While Core erlang in principle should be able to manage nested code the > Erlang compiler generally flattens things. > > What I found when doing LFE was that while the language *in principle* > allows a lot of things *in practice* later passes in the compiler, for > example the optimisation passes, assume that the code looks like what the > Erlang compiler compiler generates. One example of this is how literal > structures are represented. > > to_core0 returns the Core erlang which has been generated by the > erlang->core conversion pass which is then what is passed on to > optimisation and and code generation. This is what you should be targeting. > > To check what the actual Core erlang data structures look like, not the > pretty printed code, in the shell try doing: > > > c("foo", [binary,to_core0,return]). > > This call will *return* the actual data structure so you can put it in > variable then pretty print it to a file so you can what you really need to > get. Note that some of the attributes are important in later passes. > > What language are you working on? > > Robert > > > On 21 April 2017 at 19:12, Karl Nilsson wrote: > >> Whilst trying to compile some code to core erlang I came across a problem >> when the target expression (e0 in spec) to apply was another apply >> expression. According to the spec I thought that would be ok however when >> compiling the .core file with erlc I got an "no_file: Warning: invalid >> function call" error. >> >> What I then tried, not thinking it would work, was to wrap the inner >> apply expression in a let expression. This to my surprise worked just fine. >> I've included the code below. 'addSix` is the working function and >> `addSix2` is the dodgy one. Am I doing something wrong in `addSix2` to in >> terms of how I print my AST or is it simply that `addSix2` cannot be made >> to work? I also tried putting some parens around it but that also didn't >> work. >> >> 'add'/2 = >> fun (_a0,_b0) -> >> call 'erlang':'+'(_a0,_b0) >> >> 'addFive'/0 = >> fun () -> >> let <_a0> = 5 >> in fun (_b0) -> >> apply 'add'/2 (_a0,_b0) >> >> 'addSix'/1 = >> fun (_x0) -> >> call 'erlang':'+'(let <_fez0> = apply 'addFive'/0 () >> in apply _fez0 (_x0),1) >> >> 'addSix2'/1 = >> fun (_x0) -> >> call 'erlang':'+'(apply apply 'addFive'/0 () (_x0),1) >> >> >> Cheers >> Karl >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Sun Apr 23 08:15:59 2017 From: kjnilsson@REDACTED (Karl Nilsson) Date: Sun, 23 Apr 2017 06:15:59 +0000 Subject: [erlang-questions] Core Erlang apply target expression behaviour In-Reply-To: References: Message-ID: That is pretty much the approach I am taking. I use the FSharp.Compiler.Service to do all the checking and get access to the typed AST, then I transform that to a core erlang AST which I then "pretty" (ugly) print to a core erlang file, all in fsharp. On Sun, 23 Apr 2017 at 01:07 Robert Virding wrote: > I have been thinking about f# as well, but from a different direction. I > would want to use as much as possible of the existing f# compiler which is > written in f# so I would write an interpreter in Erlang for AST generated > by f# parser which you could then use that to run all the early parts of > the compiler, like type-checking and such. Then hopefully you could replace > the backend with a new bit which generates core erlang and use the erlang > compiler for the rest. > > I fully realise it would not be as simple as it sounds :-) but I think it > would be a doable of reusing as much as possible of what is written in f#. > For me time is the limiting factor. > > Robert > > > On 22 April 2017 at 21:52, Karl Nilsson wrote: > >> Thanks for the advice. A few things to think about there. >> I have been writing things in erlang, compiling and comparing but rather >> than writing a language from scratch I am working from the AST (or rather >> the TAST) from another language, fsharp. As fsharp is an ML and core erlang >> is ML-ish I was hoping to avoid having to do too much mangling but >> flattening out applies and mod calls with let expressions is easy enough so >> I'll definitely do that. >> >> Cheers >> Karl >> >> On Sat, 22 Apr 2017 at 20:19 Robert Virding wrote: >> >>> I would try doing 2 things: >>> >>> - Write the equivalent Erlang code which does what you want the code to >>> do, compile it with the to_core0 option and see what it generates. This >>> then becomes your target code. This is how I did it with LFE. >>> >>> - Or you can try flattening out your code to remove the nested calls. >>> While Core erlang in principle should be able to manage nested code the >>> Erlang compiler generally flattens things. >>> >>> What I found when doing LFE was that while the language *in principle* >>> allows a lot of things *in practice* later passes in the compiler, for >>> example the optimisation passes, assume that the code looks like what the >>> Erlang compiler compiler generates. One example of this is how literal >>> structures are represented. >>> >>> to_core0 returns the Core erlang which has been generated by the >>> erlang->core conversion pass which is then what is passed on to >>> optimisation and and code generation. This is what you should be targeting. >>> >>> To check what the actual Core erlang data structures look like, not the >>> pretty printed code, in the shell try doing: >>> >>> > c("foo", [binary,to_core0,return]). >>> >>> This call will *return* the actual data structure so you can put it in >>> variable then pretty print it to a file so you can what you really need to >>> get. Note that some of the attributes are important in later passes. >>> >>> What language are you working on? >>> >>> Robert >>> >>> >>> On 21 April 2017 at 19:12, Karl Nilsson wrote: >>> >>>> Whilst trying to compile some code to core erlang I came across a >>>> problem when the target expression (e0 in spec) to apply was another apply >>>> expression. According to the spec I thought that would be ok however when >>>> compiling the .core file with erlc I got an "no_file: Warning: invalid >>>> function call" error. >>>> >>>> What I then tried, not thinking it would work, was to wrap the inner >>>> apply expression in a let expression. This to my surprise worked just fine. >>>> I've included the code below. 'addSix` is the working function and >>>> `addSix2` is the dodgy one. Am I doing something wrong in `addSix2` to in >>>> terms of how I print my AST or is it simply that `addSix2` cannot be made >>>> to work? I also tried putting some parens around it but that also didn't >>>> work. >>>> >>>> 'add'/2 = >>>> fun (_a0,_b0) -> >>>> call 'erlang':'+'(_a0,_b0) >>>> >>>> 'addFive'/0 = >>>> fun () -> >>>> let <_a0> = 5 >>>> in fun (_b0) -> >>>> apply 'add'/2 (_a0,_b0) >>>> >>>> 'addSix'/1 = >>>> fun (_x0) -> >>>> call 'erlang':'+'(let <_fez0> = apply 'addFive'/0 () >>>> in apply _fez0 (_x0),1) >>>> >>>> 'addSix2'/1 = >>>> fun (_x0) -> >>>> call 'erlang':'+'(apply apply 'addFive'/0 () (_x0),1) >>>> >>>> >>>> Cheers >>>> Karl >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silviu.cpp@REDACTED Sun Apr 23 13:33:40 2017 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Sun, 23 Apr 2017 14:33:40 +0300 Subject: [erlang-questions] method available similar with mb_encode_numericentity Message-ID: Hello, There is any library/method available in erlang that does the same with mb_encode_numericentity from PHP ? http://php.net/manual/ro/function.mb-encode-numericentity.php Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Apr 23 15:48:52 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 23 Apr 2017 13:48:52 +0000 Subject: [erlang-questions] method available similar with mb_encode_numericentity In-Reply-To: References: Message-ID: I've been staring at the PHP documentation for 5 minutes. I still don't understand what that function is supposed to do. On Sun, Apr 23, 2017 at 1:33 PM Caragea Silviu wrote: > Hello, > > There is any library/method available in erlang that does the same with > mb_encode_numericentity from PHP ? > > http://php.net/manual/ro/function.mb-encode-numericentity.php > > Silviu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex0player@REDACTED Mon Apr 24 01:17:09 2017 From: alex0player@REDACTED (Alex S.) Date: Mon, 24 Apr 2017 02:17:09 +0300 Subject: [erlang-questions] method available similar with mb_encode_numericentity In-Reply-To: References: Message-ID: It turns a Unicode code point into &#xxxx;. Or possibly into a named entity probably. 2017-04-23 16:48 GMT+03:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > I've been staring at the PHP documentation for 5 minutes. I still don't > understand what that function is supposed to do. > > On Sun, Apr 23, 2017 at 1:33 PM Caragea Silviu > wrote: > >> Hello, >> >> There is any library/method available in erlang that does the same with >> mb_encode_numericentity from PHP ? >> >> http://php.net/manual/ro/function.mb-encode-numericentity.php >> >> Silviu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Mon Apr 24 11:46:16 2017 From: rickard@REDACTED (Rickard Green) Date: Mon, 24 Apr 2017 11:46:16 +0200 Subject: [erlang-questions] Patch package OTP 19.3.2 released Message-ID: <0c6a02ba-6076-10d0-bb44-0c67bcf2af5c@erlang.org> Patch Package: OTP 19.3.2 Git Tag: OTP-19.3.2 Date: 2017-04-24 Trouble Report Id: OTP-14358 Seq num: System: OTP Release: 19 Application: erts-8.3.2 Predecessor: OTP 19.3.1 Check out the git tag OTP-19.3.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-8.3.2 ------------------------------------------------------ --------------------------------------------------------------------- Note! The erts-8.3.2 application can *not* be applied independently of other applications on an arbitrary OTP 19 installation. On a full OTP 19 installation, also the following runtime dependency has to be satisfied: -- sasl-3.0.1 (first satisfied in OTP 19.1) --- Fixed Bugs and Malfunctions --- OTP-14358 Application(s): erts Related Id(s): OTP-14085 The +Bi command line argument of erl erroneously caused SIGTERM to be ignored by the VM as well as of all its child processes. This bug was introduced in erts version 8.3. Full runtime dependencies of erts-8.3.2: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From raimo+erlang-questions@REDACTED Mon Apr 24 12:01:09 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 24 Apr 2017 12:01:09 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> Message-ID: <20170424100109.GA52860@erix.ericsson.se> On Wed, Apr 19, 2017 at 12:13:26PM +0000, Bekes, Andras G wrote: > Did anyone have a chance to take a closer look? > > I still think that this is a bug in Erlang. So far no-one have had any good suggestion on how to handle this in a platform independent way, nor any suggestion on how to detect this misfeature with configure. And digging into this is not on our prioritized list. I am leaning towards special treatment for this Linux abnormality by observing the leading 0 and in this special case use all bytes of the given binary. Then we might pass a length to the kernel that does not match the string length of the file path. But if such a platform uses zero termination there is no problem, and if it uses the size information there is no problem, it is only if it uses both that might be a problem, and that should be very unlikely... Something like this should do it: ------------------- erts/emulator/drivers/common/inet_drv.c ------------------- index 0fe5183..abcc289 100644 @@ -602,14 +602,6 @@ static size_t my_strnlen(const char *s, size_t maxlen) return i; } -/* Check that some character in the buffer != '\0' */ -static int is_nonzero(const char *s, size_t n) -{ - size_t i; - for (i = 0; i < n; i++) if (s[i] != '\0') return !0; - return 0; -} - #endif #ifdef VALGRIND @@ -4018,7 +4010,15 @@ static char* inet_set_address(int family, inet_address* dst, int n; if (*len == 0) return str_einval; n = *((unsigned char*)(*src)); /* Length field */ - if ((*len < 1+n) || (sizeof(dst->sal.sun_path) < n+1)) { + if (*len < 1+n) return str_einval; + /* Portability fix: Make sure the address gets zero terminated + * except when the first byte is \0 because then it is sort + * of zero terminated although the zero termination comes + * before the address... This fix handles Linux's + * abstract socket address nonportable extension. + */ + if (n + ((*len) > 1 && (*src)[1] == '\0' ? 0 : 1) + > sizeof(dst->sal.sun_path)) { return str_einval; } sys_memzero((char*)dst, sizeof(struct sockaddr_un)); @@ -4176,8 +4176,7 @@ static int inet_get_address(char* dst, inet_address* src, unsigned int* len) * nonportable extension. */ m = my_strnlen(src->sal.sun_path, n); - if ((m == 0) && is_nonzero(src->sal.sun_path, n)) - m = n; + if (m == 0) m = n; dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); sys_memcpy(dst+2, src->sal.sun_path, m); @@ -4241,8 +4240,7 @@ inet_address_to_erlang(char *dst, inet_address **src, SOCKLEN_T sz) { * nonportable extension. */ m = my_strnlen((*src)->sal.sun_path, n); - if ((m == 0) && is_nonzero((*src)->sal.sun_path, n)) - m = n; + if (m == 0) m = n; if (dst) { dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); > > From: Bekes, Andras G (IST) > Sent: Tuesday, March 28, 2017 9:55 AM > To: 'Alex S.' > Cc: erlang-questions@REDACTED > Subject: RE: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes > > I was also thinking about how and why Erlang puts its limit at 107 characters, but regardless of the answers, apparently I need to use 108 bytes, which seems to be a valid thing on Linux. > Note: I need to see the number 110 passed as the 3rd parameter to the connect syscall, as opposed to 109. > > From: Alex S. [mailto:alex0player@REDACTED] > Sent: Monday, March 27, 2017 8:00 PM > To: Bekes, Andras G (IST) > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes > > I have a strong suspicion the binary might be automatically zero-terminated. > 27 ????? 2017 ?., ? 19:03, Bekes, Andras G > ???????(?): > > > Hi All, > > I need to communicate with unix domain sockets using the abstract namespace and the server (which is not under my control) opens the port with all 108 bytes of the path used. > If I understand it correctly, the server/client can decide to indicate the number of actual bytes used in the 3rd parameter to the bind/connect syscalls (this what Erlang does), but it is also possible to use all 108 bytes and fill the unused part with 0 bytes (this is what my server does). > http://man7.org/linux/man-pages/man7/unix.7.html > > Apparently, I can fill the path with 0 bytes up to 107 only. If I try to pass a 108-byte address to gen_tcp:connect, I get a badarg exception: > > > Sun_path_length=108. > 108 > > > UDS_path = <<"whatever">>. > <<"whatever">> > > > Fill_length = 8*(Sun_path_length-byte_size(UDS_path)-1). > 792 > > > Address = {local,<<0, UDS_path/binary,0:Fill_length>>}. > {local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,...>>} > > > Port = 0. > 0 > > > gen_tcp:connect(Address,Port,[local],infinity). > (<0.236.0>) call gen_tcp:connect({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],infinity) > (<0.236.0>) call gen_tcp:connect1({local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0>>},0,[local],false) > (<0.236.0>) call gen_tcp:try_connect([{local,<<0,119,104,97,116,101,118,101,114,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, > 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0>>}],0,[],false,local_tcp,{error,einval}) > (<0.236.0>) returned from gen_tcp:try_connect/6 -> {error,einval} > ** exception exit: badarg > in function gen_tcp:connect/4 (gen_tcp.erl, line 149) > (<0.236.0>) returned from gen_tcp:connect1/4 -> {error,einval} > (<0.236.0>) exception_from {gen_tcp,connect,4} {exit,badarg} > > If I reduce the path to 107 bytes, it works. With strace, I can see the connect syscall: > connect(19, {sa_family=AF_FILE, path=@"whatever"...}, 109) = -1 ECONNREFUSED (Connection refused) > When I strace the official client of this server, the 3rd parameter to the connect syscall is 110 (108 + 2), regardless of the actual length of the path. > > Apparently, with Erlang it is not possible to use all 108 bytes. I should only get a badarg error at 109 bytes, not 108. > Seems to me that this is a bug in the Erlang implementation. What do you think? > > Thank you very much, > > Andras G. Bekes > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From natalia.chechina@REDACTED Mon Apr 24 12:41:03 2017 From: natalia.chechina@REDACTED (Natalia Chechina) Date: Mon, 24 Apr 2017 11:41:03 +0100 Subject: [erlang-questions] [ANN] 2nd Call for Papers: 16th ACM SIGPLAN Erlang Workshop 2017 Message-ID: 2nd CALL FOR PAPERS =============== Sixteenth ACM SIGPLAN Erlang Workshop http://erlang.org/workshop/2017/ ----------------------------------------------------------- Oxford, United Kingdom, 8 September 2017 Satellite event of the 22nd ACM SIGPLAN International Conference on Functional Programming (ICFP 2017) 3 - 9 September, 2017 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages. Note that this is a maximum length; we welcome shorter papers also, and the program committee will evaluate all papers on an equal basis independent of their lengths. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Scott Lystig Fritchie, VMware, USA Natalia Chechina, Glasgow University, UK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Clara Benac Earle, Universidad Politecnica de Madrid, Spain Richard Carlsson, Klarna, Sweden Laura M. Castro, University of A Coruna, Spain Viktoria F?rd?s, Klarna, Sweden James S. Larson, Google, USA Important Dates ----------------------- Submission deadline: Fri May 26, 2017 Author notification: Fri June 23, 2017 Final submission for the publisher: Sat July 15, 2017 Workshop date: Fri September 8, 2017 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2017" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2017 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. The proceedings will be freely available for download from the ACM Digital Library from one week before the start of the conference until two weeks after the conference. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2017 web site at: http://icfp17.sigplan.org/ Related Links -------------------- ICFP 2017 web site: http://icfp17.sigplan.org/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2017 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Atendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/CodeOfConduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: EW'17-poster.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 113025 bytes Desc: not available URL: From bhag.chandra.erl@REDACTED Mon Apr 24 17:26:14 2017 From: bhag.chandra.erl@REDACTED (Bhag Chandra) Date: Mon, 24 Apr 2017 20:56:14 +0530 Subject: [erlang-questions] Which version of ejabberd would be ideal to transition code from 2.1.13 Message-ID: Hi, I had written a server for my product using ejabberd 2.1.13 and now I want to upgrade it to latest ejabberd. My system is not live, so I have no issue in incurring downtime etc. I want to upgrade in order to take advantages of more efficient codebase that ejabberd has released over the years. They recently released 17.04 so I want to understand which was the most stable that I can migrate my code to. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Mon Apr 24 17:43:08 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Mon, 24 Apr 2017 15:43:08 +0000 Subject: [erlang-questions] Who called my gen_server? Message-ID: Hi guys, I've two connected nodes: N1, N2. N1 is mine and I've a gen_server running on it. N2 is external and doesn't belong to me. A process on N2 is calling my gen_server (on N1) via an rpc:call. Is there a way to know the Pid of that process? Or, can I log all rpc_call(s) calling my gen_server? /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Mon Apr 24 20:09:34 2017 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Mon, 24 Apr 2017 11:09:34 -0700 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170424100109.GA52860@erix.ericsson.se> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> Message-ID: On 04/24/2017 03:01 AM, Raimo Niskanen wrote: > So far no-one have had any good suggestion on how to handle this > in a platform independent way, nor any suggestion on how to detect > this misfeature with configure. This _appears_ to be a useful feature that seems to make domain sockets work more like IP sockets. [0] Why do you consider it a misfeature? [0] https://utcc.utoronto.ca/~cks/space/blog/linux/SocketAbstractNamespace -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From michal@REDACTED Tue Apr 25 08:51:10 2017 From: michal@REDACTED (=?utf-8?Q?Micha=C5=82_Muska=C5=82a?=) Date: Tue, 25 Apr 2017 08:51:10 +0200 Subject: [erlang-questions] Changing the representation of sets Message-ID: Hello everybody. The sets module implements an opaque structure to represent sets. The current implementation is based on hashes and nested tuples. Since OTP 18, we have maps that are an excellent and performant data structure for storing arbitrary keys & values. This means, they can be used to efficiently implement sets, by storing set elements as keys with some default value. My measurements [1] show that a set implementation based on maps is 2-4 times faster in all operations than the current implementation from the sets module. Comparing performance tradeoffs to other set implementations, it seems that map-based sets have exactly the same characteristics as current sets, except they are faster. Map-based sets also have a huge advantage of being easily readable when printed in the console. Given the sets data structure is opaque it should be possible to just switch implementations and make everybody's code faster without changing a line of code. The only issue might be incorrect uses of the sets module that assume some particular shape for the structure - I'm not sure how widespread that is. Another possibility would be to implement a new map_sets module, but I think 4 set implementation is already too many - no need to add another. There's also no need to write a new implementation since there's already a well tested, map-based set implementation available in OTP in the cerl_sets module in the compiler application [2]. A similar consideration could be made for the dict module, but I haven't looked closely at the performance differences there - I would expect something similar, though, since sets and dict use a similar implementation. Regards, Micha?. [1]:?https://gist.github.com/michalmuskala/9b0b55eee25b94d9339edbf7a0e261ea [2]:?https://github.com/erlang/otp/blob/master/lib/compiler/src/cerl_sets.erl -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Apr 25 09:18:38 2017 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 25 Apr 2017 00:18:38 -0700 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: <58FEF84E.7050905@gmail.com> On 04/24/2017 11:51 PM, Micha? Muska?a wrote: > Hello everybody. > > The sets module implements an opaque structure to represent sets. The current implementation is based on hashes and nested tuples. Since OTP 18, we have maps that are an excellent and performant data structure for storing arbitrary keys & values. This means, they can be used to efficiently implement sets, by storing set elements as keys with some default value. > > My measurements [1] show that a set implementation based on maps is 2-4 times faster in all operations than the current implementation from the sets module. Comparing performance tradeoffs to other set implementations, it seems that map-based sets have exactly the same characteristics as current sets, except they are faster. Map-based sets also have a huge advantage of being easily readable when printed in the console. Given the sets data structure is opaque it should be possible to just switch implementations and make everybody's code faster without changing a line of code. The only issue might be incorrect uses of the sets module that assume some particular shape for the structure - I'm not sure how widespread that is. > > Another possibility would be to implement a new map_sets module, but I think 4 set implementation is already too many - no need to add another. There's also no need to write a new implementation since there's already a well tested, map-based set implementation available in OTP in the cerl_sets module in the compiler application [2]. > > A similar consideration could be made for the dict module, but I haven't looked closely at the performance differences there - I would expect something similar, though, since sets and dict use a similar implementation. > > Regards, > Micha?. > > [1]: https://gist.github.com/michalmuskala/9b0b55eee25b94d9339edbf7a0e261ea > [2]: https://github.com/erlang/otp/blob/master/lib/compiler/src/cerl_sets.erl > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions It is best to avoid modifying dict and sets, due to unintended consequences. One I can think of is the impact on memory allocation when a dict or sets is sent through distributed Erlang, since the memory allocation for a map larger than 32 elements (|MAP_SMALL_MAP_LIMIT|) gets an allocation from |HASHMAP_ESTIMATED_HEAP_SIZE(SIZE) == ||(SIZE*3 + (2*SIZE/5)*2) which can severely |exaggerate the size of the map (details in|erts/emulator/beam/erl_map.h|). There may be other impacts also, but that is unclear. Keeping maps and dict/sets separate should be safest to avoid unintended changes in legacy source code. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnzeng91@REDACTED Tue Apr 25 09:14:32 2017 From: johnzeng91@REDACTED (=?UTF-8?B?6L6+5pu+?=) Date: Tue, 25 Apr 2017 15:14:32 +0800 Subject: [erlang-questions] What is "Rest" in the return of xmerl_scan:string() ? Message-ID: Hi all, I am using xmerl_scan to decode a xml string, but when calling xmerl_scan:sting, it returns in a tuple: {xmlElement{}, Rest} I've searched about this API, looks like we all ignore the Rest in return. So what is the Rest used for?? Should I ignore them?? Here is the defination of xmerl_scan:string/1 in erlang OTP 19, *string(Text::list()) -> {**xmlElement()**, Rest}* Great thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Tue Apr 25 13:03:40 2017 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Tue, 25 Apr 2017 13:03:40 +0200 Subject: [erlang-questions] What is "Rest" in the return of xmerl_scan:string() ? In-Reply-To: References: Message-ID: In all parsers, that would be the tail of input, which does not match the rules anymore. Empty string if your input is correct. 2017-04-25 9:14 GMT+02:00 ?? : > Hi all, > I am using xmerl_scan to decode a xml string, but when calling > xmerl_scan:sting, it returns in a tuple: > > {xmlElement{}, Rest} > > I've searched about this API, looks like we all ignore the Rest in return. > > So what is the Rest used for?? Should I ignore them?? > > Here is the defination of xmerl_scan:string/1 in erlang OTP 19, > > *string(Text::list()) -> {**xmlElement()**, Rest}* > > > Great thanks. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Tue Apr 25 13:17:44 2017 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 25 Apr 2017 13:17:44 +0200 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: 2017-04-25 8:51 GMT+02:00 Micha? Muska?a : [...] > My measurements [1] show that a set implementation based on maps is 2-4 > times faster in all operations than the current implementation from the sets > module. Comparing performance tradeoffs to other set implementations, it > seems that map-based sets have exactly the same characteristics as current > sets, except they are faster. Map-based sets also have a huge advantage of > being easily readable when printed in the console. Given the sets data > structure is opaque it should be possible to just switch implementations and > make everybody's code faster without changing a line of code. The only issue > might be incorrect uses of the sets module that assume some particular shape > for the structure - I'm not sure how widespread that is. It would affect everybody who saves the opaque data using an earlier OTP version, then reads it in newer OTP version (e.g. after upgrade). Or those who run two nodes on different OTP versions (e.g. during an upgrade). From jesper.louis.andersen@REDACTED Tue Apr 25 15:17:00 2017 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 25 Apr 2017 13:17:00 +0000 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: On Tue, Apr 25, 2017 at 1:17 PM Attila Rajmund Nohl wrote: > It would affect everybody who saves the opaque data using an earlier > OTP version, then reads it in newer OTP version (e.g. after upgrade). > Or those who run two nodes on different OTP versions (e.g. during an > upgrade). > I like to think of dict and sets to be "opaque" data structures. If you are relying on their internal representation, you will run into trouble. So changing the representation going forward should definitely be possible. This leaves backwards compatibility. If the maps-optimized module can read and dynamically change old sets implementations into the new format on the fly, it may be possible to gradually replace the old representations with the new. Some things to look out for in that process is data-at-rest, stored years ago in a database. At some point you would have to reprocess such data, or supply a conversion module which can handle these old formats. I also think that any system must provide some measure of pushing things forward. That is, each major release of Erlang could contain a limited set of things you now have to do differently, with a clear upgrade path from earlier versions. As long as the set is limited, we can probably handle the rewrites needed. If you value backwards compatibility for forever, you run the risk of getting stale, never upgrading anything. As for the increased memory copy pressure: I think this should be fixed in the context of "maps" and not be part of the argument as to why one would keep the old sets representation. As an aside: I've long wanted a way to "tag" an erlang term as "do-not-touch-this" That, is to provide functions: seal(Tag, Term) -> Sealed unseal(Tag, Sealed) -> Term which "hides" the representation of Term if printed, replacing it with some kind of opaque representation (Think a function). For debugging purposes, an auto-unseal could be necessary. One reason for this is that I can make a representation abstract so users of a library are unlikely to rely on the representation being a certain structure and by accident tightly coupling their code to my code. While we are all mutually consenting adults, experience has shown people tend to rely on internals quite often. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Apr 25 16:14:04 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 25 Apr 2017 16:14:04 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> Message-ID: <20170425141404.GA26809@erix.ericsson.se> On Mon, Apr 24, 2017 at 11:09:34AM -0700, Kenneth Lakin wrote: > On 04/24/2017 03:01 AM, Raimo Niskanen wrote: > > So far no-one have had any good suggestion on how to handle this > > in a platform independent way, nor any suggestion on how to detect > > this misfeature with configure. > > This _appears_ to be a useful feature that seems to make domain sockets > work more like IP sockets. [0] Why do you consider it a misfeature? Well, it is just because on all other Unix-like OS:es the address is (loosly/de-facto) defined to be zero terminated, and there are macros to calculate the address structure size that use strlen on the string. Therefore the Linux way to use the address type contradicts the BSD:s and I guess Solaris. Nobody has so far found a way to figure out how to use configure to detect if a size larger than the string length is OK or might crash the program some day. Other than that I am sure it is a useful feature, so maybe not a misfeature, more an API abuse. > > [0] https://utcc.utoronto.ca/~cks/space/blog/linux/SocketAbstractNamespace > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From marco.molteni@REDACTED Tue Apr 25 18:01:06 2017 From: marco.molteni@REDACTED (Marco Molteni) Date: Tue, 25 Apr 2017 18:01:06 +0200 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: <5B41658C-B209-48E4-BCAC-425954675E78@laposte.net> On 25 Apr 2017, at 15:17, Jesper Louis Andersen wrote: [..] > I also think that any system must provide some measure of pushing things forward. That is, each major release of Erlang could contain a limited set of things you now have to do differently, with a clear upgrade path from earlier versions. As long as the set is limited, we can probably handle the rewrites needed. If you value backwards compatibility for forever, you run the risk of getting stale, never upgrading anything. AMEN. Marco From michal@REDACTED Tue Apr 25 18:40:47 2017 From: michal@REDACTED (=?utf-8?Q?Micha=C5=82_Muska=C5=82a?=) Date: Tue, 25 Apr 2017 18:40:47 +0200 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: <58FEF84E.7050905@gmail.com> References: <58FEF84E.7050905@gmail.com> Message-ID: <567bdaa0-4604-479e-ab6e-22917d367046@Spark> I checked how the memory sizes compare. Map-based sets are indeed larger - about 70% larger in memory than current sets, however they are smaller when serialised with term_to_binary (assuming using [] for the bogus value) by about 5%. That is when storing plain integers - when storing bigger terms, the difference quickly melts. Maps are smaller than dict - by about 12% in memory and 40% when serialised. Micha?. On 25 Apr 2017, 09:18 +0200, Michael Truog , wrote: > On 04/24/2017 11:51 PM, Micha? Muska?a wrote: > > Hello everybody. > > > > The sets module implements an opaque structure to represent sets. The current implementation is based on hashes and nested tuples. Since OTP 18, we have maps that are an excellent and performant data structure for storing arbitrary keys & values. This means, they can be used to efficiently implement sets, by storing set elements as keys with some default value. > > > > My measurements [1] show that a set implementation based on maps is 2-4 times faster in all operations than the current implementation from the sets module. Comparing performance tradeoffs to other set implementations, it seems that map-based sets have exactly the same characteristics as current sets, except they are faster. Map-based sets also have a huge advantage of being easily readable when printed in the console. Given the sets data structure is opaque it should be possible to just switch implementations and make everybody's code faster without changing a line of code. The only issue might be incorrect uses of the sets module that assume some particular shape for the structure - I'm not sure how widespread that is. > > > > Another possibility would be to implement a new map_sets module, but I think 4 set implementation is already too many - no need to add another. There's also no need to write a new implementation since there's already a well tested, map-based set implementation available in OTP in the cerl_sets module in the compiler application [2]. > > > > A similar consideration could be made for the dict module, but I haven't looked closely at the performance differences there - I would expect something similar, though, since sets and dict use a similar implementation. > > > > Regards, > > Micha?. > > > > [1]:?https://gist.github.com/michalmuskala/9b0b55eee25b94d9339edbf7a0e261ea > > [2]:?https://github.com/erlang/otp/blob/master/lib/compiler/src/cerl_sets.erl > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > It is best to avoid modifying dict and sets, due to unintended consequences.? One I can think of is the impact on memory allocation when a dict or sets is sent through distributed Erlang, since the memory allocation for a map larger than 32 elements (MAP_SMALL_MAP_LIMIT) gets an allocation from HASHMAP_ESTIMATED_HEAP_SIZE(SIZE) == (SIZE*3 + (2*SIZE/5)*2) which can severely exaggerate the size of the map (details in erts/emulator/beam/erl_map.h).? There may be other impacts also, but that is unclear.? Keeping maps and dict/sets separate should be safest to avoid unintended changes in legacy source code. > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Apr 25 19:21:58 2017 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 25 Apr 2017 10:21:58 -0700 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: <58FF85B6.7010102@gmail.com> On 04/25/2017 06:17 AM, Jesper Louis Andersen wrote: > On Tue, Apr 25, 2017 at 1:17 PM Attila Rajmund Nohl > wrote: > > It would affect everybody who saves the opaque data using an earlier > OTP version, then reads it in newer OTP version (e.g. after upgrade). > Or those who run two nodes on different OTP versions (e.g. during an > upgrade). > > > I like to think of dict and sets to be "opaque" data structures. If you are relying on their internal representation, you will run into trouble. So changing the representation going forward should definitely be possible. > > This leaves backwards compatibility. If the maps-optimized module can read and dynamically change old sets implementations into the new format on the fly, it may be possible to gradually replace the old representations with the new. > > Some things to look out for in that process is data-at-rest, stored years ago in a database. At some point you would have to reprocess such data, or supply a conversion module which can handle these old formats. > > I also think that any system must provide some measure of pushing things forward. That is, each major release of Erlang could contain a limited set of things you now have to do differently, with a clear upgrade path from earlier versions. As long as the set is limited, we can probably handle the rewrites needed. If you value backwards compatibility for forever, you run the risk of getting stale, never upgrading anything. > > As for the increased memory copy pressure: I think this should be fixed in the context of "maps" and not be part of the argument as to why one would keep the old sets representation. The memory allocation related to maps is likely not something to solve, and should be related to its data structure (HAMT (Hash-Array Mapped Trie)). Keeping things stable is important and there should never be changes that sacrifice stability for a passing whim. It is easy to add new modules for maps used for sets and it is important to avoid adversely affecting legacy source code. That doesn't block future improvements, it only makes sure they are focused on being improvements. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From marklrh@REDACTED Tue Apr 25 20:32:38 2017 From: marklrh@REDACTED (Runhang Li) Date: Tue, 25 Apr 2017 11:32:38 -0700 Subject: [erlang-questions] CUFP 2017 Call for Tutorials Message-ID: <0E04FBF2-24DC-4C0B-B8D0-C170FCC125AD@gmail.com> Call for Tutorials Commercial Users of Functional Programming (CUFP) 2017 September 7th-9th, 2017, Oxford, United Kingdom Co-located with International Conference on Functional Programming (ICFP) Dear Erlang community, Commercial Users of Functional Programming (CUFP) 2017 is now seeking tutorials proposals. Co-located with International Conference on Functional Programming (ICFP), Commercial Users of Functional Programming (CUFP) will take place in Oxford, U.K. from 7th to 9th of September, 2017. CUFP is devoted to showcase the state of the art of functional programming in industrial settings. We are looking for tutors who would like to host half-day tutorial sessions of the following topics: * Introductions to functional programming languages, e.g., Haskell, OCaml, Scala, F#, Scheme, Erlang, Clojure. * Advanced programming languages, concepts, or applications, e.g., Agda, Idris, F*, Coq, Eff. * Applications of functional languages in particular areas, including web, high-performance computing, and finance. * Tools and techniques supporting state of the art functional programming, e.g., ReasonML, TypeScript, Elm, QuickCheck, FlowType. * Theory. Type theory, category theory, abstract algebra, ongoing or new research, or anything useful or interesting to functional programmers. **Proposal submission deadline is May 25th** Details can be found at: http://cufp.org/2017/call-for-tutorials.html Kindly, Runhang Li, CUFP 2017 tutorials co-chair From ok@REDACTED Wed Apr 26 03:07:47 2017 From: ok@REDACTED (ok@REDACTED) Date: Wed, 26 Apr 2017 13:07:47 +1200 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: Michael Muskala suggests *replacing* 'sets' (and maybe 'dict') with something based on maps, and offers numbers to show that *using* something based on maps would be faster. Primum non nocere. Yes, it would be good to have better set and dict modules. But they should be *new* modules. The old modules can be deprecated, that's fine, and even eventually removed (so that people have to pull them out of old distributions if they want to keep using them). But just up and changing the representation is likely to break programs with persistent data. From t@REDACTED Wed Apr 26 04:36:44 2017 From: t@REDACTED (Tristan Sloughter) Date: Tue, 25 Apr 2017 19:36:44 -0700 Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: <1493174204.1517681.956380936.2A178A5A@webmail.messagingengine.com> > But they should be *new* modules. > The old modules can be deprecated, that's fine, > and even eventually removed (so that people have to pull > them out of old distributions if they want to keep using > them). Time for sets2 and dict2 :). We've already got pg2, phash2, rebar3... Putting numbers at the end of functions/modules/programs seems to be a pattern in Erlang. I'd definitely want it deprecated and removed eventually if it were determined the memory size tradeoff is worth it for the speed improvements. Personally I'd prefer to just get a map when my current code upgraded OTP versions, but understand that especially since sets and dicts aren't types, so binary_to_term can't do the conversion (aside from checking the structure of the tuple it decodes but that is scary), this is asking for trouble. Tristan From ok@REDACTED Wed Apr 26 05:52:52 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 26 Apr 2017 15:52:52 +1200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170425141404.GA26809@erix.ericsson.se> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> Message-ID: > On 26/04/2017, at 2:14 AM, Raimo Niskanen wrote: > > Well, it is just because on all other Unix-like OS:es the address is > (loosly/de-facto) defined to be zero terminated, and there are macros to > calculate the address structure size that use strlen on the string. It's not *that* loose. POSIX explicitly defines sun_path to hold a "pathname" and a path name is defined as a character string and those are defined to be NUL terminated. (See and then check definitions for "path name" and "string".) > > Therefore the Linux way to use the address type contradicts the BSD:s and I > guess Solaris. And POSIX. And HP-UX and AIX, as far as I can tell, and to the degree that anyone cares. From manojy@REDACTED Wed Apr 26 07:58:00 2017 From: manojy@REDACTED (Manoj) Date: Wed, 26 Apr 2017 11:28:00 +0530 Subject: [erlang-questions] Disable erlang crash dump generation Message-ID: <6d1b826d-43db-f915-efce-e7416f0eb320@utl.in> Dear all, I'm running erlang in an embedded environment, and there 's a requirement to maintain 15MB free memory at any point of time. To ensure the same I need to disable the generation of erlang crash dump file(erl_crash.dump) file since it may grow in size. How can this be done?? I've tried the following options: i. "+d" option when beginning to run the erlang node ii. Set the value of the environment variable ERL_CRASH_DUMP to "/dev/null"(tried this both when running the erlang application, and also setting the environment variable explicitly after running the application) *PS: I'm running the my application on Erlang/OTP 19 [erts-8.0]* -- Regards, Manoj -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Wed Apr 26 08:32:41 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Wed, 26 Apr 2017 08:32:41 +0200 (CEST) Subject: [erlang-questions] Changing the representation of sets In-Reply-To: References: Message-ID: <842449454.1047552.1493188361774.JavaMail.zimbra@tpip.net> Hi, ----- On Apr 25, 2017, at 1:17 PM, Attila Rajmund Nohl attila.r.nohl@REDACTED wrote: [...] > It would affect everybody who saves the opaque data using an earlier > OTP version, then reads it in newer OTP version (e.g. after upgrade). That's easily solved by having the sets module accept the old representation and convert it on the fly. > Or those who run two nodes on different OTP versions (e.g. during an > upgrade). That is a different beast. One to solve it would be to not automatically convert between old and new representation. Keep the old structure (and code) and add a parameter to all function converting to sets to choose between the old and new internal representation. We could also have some global setting telling the sets to module to auto-convert or not (defaulting to "do not convert). Andreas From lukas@REDACTED Wed Apr 26 09:42:03 2017 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 26 Apr 2017 09:42:03 +0200 Subject: [erlang-questions] Disable erlang crash dump generation In-Reply-To: <6d1b826d-43db-f915-efce-e7416f0eb320@utl.in> References: <6d1b826d-43db-f915-efce-e7416f0eb320@utl.in> Message-ID: Hello, You can set ERL_CRASH_DUMP_SECONDS to 0 and it will not generate a erl_crash.dump. Also since Erlang/OTP 19.2 there is the ERL_CRASH_DUMP_BYTES variable that can be used to limit the size of the erl_crash.dump. http://erlang.org/doc/man/erl.html#id199588 Lukas On Wed, Apr 26, 2017 at 7:58 AM, Manoj wrote: > Dear all, > > I'm running erlang in an embedded environment, and there 's a requirement > to maintain 15MB free memory > > at any point of time. To ensure the same I need to disable the generation > of erlang crash dump file(erl_crash.dump) > > file since it may grow in size. How can this be done?? > > I've tried the following options: > > i. "+d" option when beginning to run the erlang node > > ii. Set the value of the environment variable ERL_CRASH_DUMP to > "/dev/null"(tried this both when running the erlang application, and > > also setting the environment variable explicitly after running the > application) > > *PS: I'm running the my application on Erlang/OTP 19 [erts-8.0]* > > -- > Regards, > > Manoj > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Apr 26 10:55:58 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 26 Apr 2017 10:55:58 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> Message-ID: <20170426085558.GA87429@erix.ericsson.se> On Wed, Apr 26, 2017 at 03:52:52PM +1200, Richard A. O'Keefe wrote: > > > On 26/04/2017, at 2:14 AM, Raimo Niskanen wrote: > > > > Well, it is just because on all other Unix-like OS:es the address is > > (loosly/de-facto) defined to be zero terminated, and there are macros to > > calculate the address structure size that use strlen on the string. > > It's not *that* loose. POSIX explicitly defines sun_path to hold a "pathname" > and a path name is defined as a character string and those are defined to be > NUL terminated. (See and then check definitions for "path name" > and "string".) Ok. That is not as loose as I thought. I can not find (in Posix) any clue to which address_len you shall use with a sockaddr_un structure. Is it the size of the whole structure, or is it the size of the structure ending with the terminating 0 in the path? There is an example: http://pubs.opengroup.org/onlinepubs/9699919799/ -> bind - bind a name to a socket that uses the size of the structure, ignoring the path length in .sun_path. That would speak for that the Linux use is creative but not violating Posix since the path is 0 terminated and the address size is large enough. They just encode an address after the empty path... But Posix also states that bind() shall fail with [ENOENT] if the pathname is an empty string. :-) And e.g FreeBSD unix(4) tells you to use the macro SUN_LEN() that uses strlen(.sun_path) to calculate the address size. But it seems Posix does not define this macro. So it seems it is not a violation of Posix to use a su->sun_path that starts with a 0 combined with an address_len of sizeof(*su). Or maybe even between SUN_LEN(su) and sizeof(*su)? Or? > > > > Therefore the Linux way to use the address type contradicts the BSD:s and I > > guess Solaris. > > And POSIX. And HP-UX and AIX, as far as I can tell, and to the degree that > anyone cares. I am sorry, since I am not a native english speaker; what does "and to the degree that anyone cares" mean in this context? -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From arif.ishaq@REDACTED Wed Apr 26 12:18:19 2017 From: arif.ishaq@REDACTED (Arif Ishaq) Date: Wed, 26 Apr 2017 10:18:19 +0000 Subject: [erlang-questions] What is "Rest" in the return of xmerl_scan:string() ? Message-ID: In Telecom networks we often have ASN.1 documents that are bundled into files, one after the other, even though this is not correct ASN.1. The Rest variable will help you decode such bundled documents. Date: Tue, 25 Apr 2017 13:03:40 +0200 From: Dmytro Lytovchenko To: ?? Cc: Erlang/OTP discussions Subject: Re: [erlang-questions] What is "Rest" in the return of xmerl_scan:string() ? Message-ID: Content-Type: text/plain; charset="utf-8" In all parsers, that would be the tail of input, which does not match the rules anymore. Empty string if your input is correct. 2017-04-25 9:14 GMT+02:00 ?? : > Hi all, > I am using xmerl_scan to decode a xml string, but when calling > xmerl_scan:sting, it returns in a tuple: > > {xmlElement{}, Rest} > > I've searched about this API, looks like we all ignore the Rest in return. > > So what is the Rest used for?? Should I ignore them?? > > Here is the defination of xmerl_scan:string/1 in erlang OTP 19, > > *string(Text::list()) -> {**xmlElement()**, Rest}* > > > Great thanks. > From kuna.prime@REDACTED Wed Apr 26 13:23:35 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Wed, 26 Apr 2017 13:23:35 +0200 Subject: [erlang-questions] Private message regarding: adding compile pass In-Reply-To: References: Message-ID: it's a shame, looking at source of compliler.erl this feature would be easy to add. Are you aware of any such proposal? Also it would be interesting to hear opinions on adding such proposal. thank you On Wed, Apr 26, 2017 at 10:29 AM, Juan Facorro wrote: > On Saturday, April 22, 2017 at 8:58:59 PM UTC+1, Karlo Kuna wrote: > > can one add custom compile pass when compiling module?? > > It is not possible to arbitrarily insert a compiler pass in the Erlang > compiler. Only parse transforms are currently available as a hook as a sort > of preprocessing pass that handle Erlang AST. > > Cheers, > Juan -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Apr 26 15:25:42 2017 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 26 Apr 2017 15:25:42 +0200 Subject: [erlang-questions] Private message regarding: adding compile pass In-Reply-To: References: Message-ID: You can add a core transform pass, just like a parse transform, but there is no support for adding passes below that level. In the source file: -compile({core_transform,Module}). or on the command line: erlc +'{core_transform,Module}' will cause Module:core_transform(Code, Options) to be called during compilation. /Richard 2017-04-26 13:23 GMT+02:00 Karlo Kuna : > it's a shame, looking at source of compliler.erl this feature would be > easy to add. Are you aware of any such proposal? Also it would be > interesting to hear opinions on adding such proposal. > > thank you > > On Wed, Apr 26, 2017 at 10:29 AM, Juan Facorro > wrote: > >> On Saturday, April 22, 2017 at 8:58:59 PM UTC+1, Karlo Kuna wrote: >> > can one add custom compile pass when compiling module?? >> >> It is not possible to arbitrarily insert a compiler pass in the Erlang >> compiler. Only parse transforms are currently available as a hook as a sort >> of preprocessing pass that handle Erlang AST. >> >> Cheers, >> Juan > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juan.facorro@REDACTED Wed Apr 26 15:27:52 2017 From: juan.facorro@REDACTED (=?UTF-8?B?SnVhbiBNYXJ0w61u?=) Date: Wed, 26 Apr 2017 13:27:52 +0000 Subject: [erlang-questions] Private message regarding: adding compile pass In-Reply-To: References: Message-ID: Nice! I didn't know about that one, obviously :P. On Wed, Apr 26, 2017, 2:25 PM Richard Carlsson wrote: > You can add a core transform pass, just like a parse transform, but there > is no support for adding passes below that level. > > In the source file: -compile({core_transform,Module}). > > or on the command line: erlc +'{core_transform,Module}' > > will cause Module:core_transform(Code, Options) to be called during > compilation. > > > > /Richard > > 2017-04-26 13:23 GMT+02:00 Karlo Kuna : > >> it's a shame, looking at source of compliler.erl this feature would be >> easy to add. Are you aware of any such proposal? Also it would be >> interesting to hear opinions on adding such proposal. >> >> thank you >> >> On Wed, Apr 26, 2017 at 10:29 AM, Juan Facorro >> wrote: >> >>> On Saturday, April 22, 2017 at 8:58:59 PM UTC+1, Karlo Kuna wrote: >>> > can one add custom compile pass when compiling module?? >>> >>> It is not possible to arbitrarily insert a compiler pass in the Erlang >>> compiler. Only parse transforms are currently available as a hook as a sort >>> of preprocessing pass that handle Erlang AST. >>> >>> Cheers, >>> Juan >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Wed Apr 26 21:50:05 2017 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Wed, 26 Apr 2017 12:50:05 -0700 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170426085558.GA87429@erix.ericsson.se> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> Message-ID: <2c35d583-572a-f4a2-446d-a7daa20676b9@gmail.com> Disclaimer: I've only read the man pages, I've not actually worked with this stuff. On 04/26/2017 01:55 AM, Raimo Niskanen wrote: > Well, it is just because on all other Unix-like OS:es > the address is (loosly/de-facto) defined to be zero > terminated... unix(7) on my Linux system [0] agrees that sockaddr_un.sun_path is typically null-terminated. (Though, it sounds like abstract socket addresses are always 108 bytes long, as embedded NULLs other than the very first one are not treated specially.) The BUGS section claims to describe two different portable methods to retrieve the specified socket address when there might be exactly 108 bytes of address. The man page makes it really sound like Linux will Do The Right Thing, as long as sun_path is no longer than 108 bytes. So, just make sure that the Erlang string or binary containing the socket address isn't longer than 108 bytes, fill sun_path (or the entire sockaddr_un) with 0 before you copy the address in, and you should be fine? I'm _fairly_ certain the second paragraph in the BUGS section is talking about the behavior of non-Linux systems, and can be ignored when writing Linux-specific code. Someone _please_ correct me if I'm wrong about that. > Nobody has so far found a way to figure out how to use > configure to detect if a size larger than the string length > is OK or might crash the program some day. As mentioned above, it _looks_ like sun_paths up to 108 bytes are A-OK. > I can not find (in Posix) any clue to which address_len you shall use with > a sockaddr_un structure. Is it the size of the whole structure[?] Yeah, it _looks_ like it is the size of the whole structure. The "Pathname sockets" section of the unix(7) man page suggests: "or, more simply, addrlen can be specified as sizeof(struct sockaddr_un)." The EXAMPLE section in bind(2) [1] as well as the EXAMPLE section in unix(7) indicates that passing sizeof(struct sockaddr_un) as bind's third argument is the correct way to specify addrlen to bind a Unix domain socket. _Additionally_, in the POSIX reference that you linked to in your message, the man page for bind(2) [2] agrees with the Linux man page that sizeof(struct sockaddr_un) is the correct thing to pass to bind's third argument. Or did I misunderstand, and you're talking about a different address length parameter? [0] http://man7.org/linux/man-pages/man7/unix.7.html [1] http://man7.org/linux/man-pages/man2/bind.2.html [2] http://pubs.opengroup.org/onlinepubs/9699919799/functions/bind.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From mikpelinux@REDACTED Wed Apr 26 22:00:24 2017 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Wed, 26 Apr 2017 22:00:24 +0200 Subject: [erlang-questions] Who called my gen_server? In-Reply-To: References: Message-ID: <22784.64600.968446.430903@gargle.gargle.HOWL> Frank Muller writes: > Hi guys, > > I've two connected nodes: N1, N2. > > N1 is mine and I've a gen_server running on it. > N2 is external and doesn't belong to me. > > A process on N2 is calling my gen_server (on N1) via an rpc:call. > > Is there a way to know the Pid of that process? Based on a half hour study of lib/kernel/src/rpc.erl in OTP-19.3 I didn't think so, but now I think I may have found a way. On the callee node, rpc has a gen_server (rex) which handles incoming calls. When it calls the {Mod,Fun,Args} from the rpc:call, it does so asynchronously via a temporary process (see handle_call_call/6). There is a mapping from that process' Pid to the From passed to rex' handle_call (called 'To' in the code) in rex' state, which you could dig out using sys:get_state or :get_status. But on the caller node, rpc:call also uses a temporary process when doing the gen_server:call to the callee node (see do_call/3), and the only association between that process and the application process doing the rpc:call is buried in the local variables of do_call/3 and the temporary process. (So if you do the sys:get_state on the callee node, you'd end up with the Pid of the temporary process on the caller node.) ... but now it struck me that the temporary process is monitored by the process doing the rpc:call, which erlang:process_info(Pid, monitored_by) should be able to retrieve. So a reverse rpc:call to the caller node to do that could work. Not tested, will break if the rpc implementation changes, etc. Why do you need this anyway? > Or, can I log all rpc_call(s) calling my gen_server? rex doesn't have that built-in, but you could modify it to do so (it's on the node you control) /Mikael From kuna.prime@REDACTED Wed Apr 26 23:14:45 2017 From: kuna.prime@REDACTED (Karlo Kuna) Date: Wed, 26 Apr 2017 23:14:45 +0200 Subject: [erlang-questions] Private message regarding: adding compile pass In-Reply-To: References: Message-ID: thank you, i was looking something like command line version, although if would be nice to compile module programatically and non invasive (-compile option). On Wed, Apr 26, 2017 at 3:27 PM, Juan Mart?n wrote: > Nice! I didn't know about that one, obviously :P. > > On Wed, Apr 26, 2017, 2:25 PM Richard Carlsson > wrote: > >> You can add a core transform pass, just like a parse transform, but there >> is no support for adding passes below that level. >> >> In the source file: -compile({core_transform,Module}). >> >> or on the command line: erlc +'{core_transform,Module}' >> >> will cause Module:core_transform(Code, Options) to be called during >> compilation. >> >> >> >> /Richard >> >> 2017-04-26 13:23 GMT+02:00 Karlo Kuna : >> >>> it's a shame, looking at source of compliler.erl this feature would be >>> easy to add. Are you aware of any such proposal? Also it would be >>> interesting to hear opinions on adding such proposal. >>> >>> thank you >>> >>> On Wed, Apr 26, 2017 at 10:29 AM, Juan Facorro >>> wrote: >>> >>>> On Saturday, April 22, 2017 at 8:58:59 PM UTC+1, Karlo Kuna wrote: >>>> > can one add custom compile pass when compiling module?? >>>> >>>> It is not possible to arbitrarily insert a compiler pass in the Erlang >>>> compiler. Only parse transforms are currently available as a hook as a sort >>>> of preprocessing pass that handle Erlang AST. >>>> >>>> Cheers, >>>> Juan >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Apr 27 02:46:53 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 27 Apr 2017 12:46:53 +1200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170426085558.GA87429@erix.ericsson.se> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> Message-ID: > On 26/04/2017, at 8:55 PM, Raimo Niskanen wrote: > > Ok. That is not as loose as I thought. > > I can not find (in Posix) any clue to which address_len you shall use with > a sockaddr_un structure. Is it the size of the whole structure, or is it > the size of the structure ending with the terminating 0 in the path? The bind() description says that "address_len Specifies the length of the sockaddr structure pointed to by the address argument." It does not say the *size* of the structure but the *length* of the structure. Unfortunately, I can find no definition of "length of a structure". Sigh. > There is an example: > http://pubs.opengroup.org/onlinepubs/9699919799/ -> > bind - bind a name to a socket > that uses the size of the structure, ignoring the path length in .sun_path. > That would speak for that the Linux use is creative but not violating Posix > since the path is 0 terminated and the address size is large enough. The Linux extension puts a NUL byte at the front of the path, thus unambiguously providing the path "". Bytes after that are not part of the required path name. It is *deeply* regrettable that the Linux extenders did not do the clean thing and add a new address family. I feel that I have to say this: if you look at the TOP500 list, 498 out of the top 500 supercomputers in the world run "Linux". (The other 2 run "Unix".) While I enjoy exegesis of POSIX, at some point reality must be acknowledged... Solaris documentation is clear that you should use bind(s, (struct sockaddr *) &addr, strlen(addr.sun_path) + sizeof (addr.sun_family)) Poking around in the OpenSolaris kernel source code, the function copyin_name() suggests to me that any value between that and the full size of the structure should work. > So it seems it is not a violation of Posix to use a su->sun_path that starts > with a 0 combined with an address_len of sizeof(*su). It may not be, but the path is still "" and thus a run-time error. Turning errors into extensions is common enough, so this leaves the situation where we found it, really. POSIX systems *shouldn't* do this, but Linux *does*. > >>> Therefore the Linux way to use the address type contradicts the BSD:s and I >>> guess Solaris. >> >> And POSIX. And HP-UX and AIX, as far as I can tell, and to the degree that >> anyone cares. > > I am sorry, since I am not a native english speaker; what does "and to the > degree that anyone cares" mean in this context? What I meant in that paragraph was "I have checked the AIX and HP-UX manuals and they certainly do not mention the Linux extension and can be read as consistent with POSIX and the BSDs, so Erlang code using this Linux feature would not port to AIX or HP-UX, but does that have any practical significance? Does anyone *care* about Erlang on AIX or HP-UX?" From frank.muller.erl@REDACTED Thu Apr 27 06:10:37 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 27 Apr 2017 04:10:37 +0000 Subject: [erlang-questions] Who called my gen_server? In-Reply-To: <22784.64600.968446.430903@gargle.gargle.HOWL> References: <22784.64600.968446.430903@gargle.gargle.HOWL> Message-ID: Hi Mikael, Thank you for the hint about rpc module. I'll try to read it to understand its behaviour I'm still considering myself an Erlang newbie. The idea here is to log all calls to my gen_server for debugging purposes: who called me, from where, when, with which arguments ... For "handle_call", it's more or less easy. Not so for "handle_cast" and "handle_info". Finally, I would like to generalize this idea to more than one gen_server. It'll be very tedious if I've to add log calls to every gen_server in my app. Maybe there's a better way. Thanks again for your time Mikael. /Frank wrote : > Frank Muller writes: > > Hi guys, > > > > I've two connected nodes: N1, N2. > > > > N1 is mine and I've a gen_server running on it. > > N2 is external and doesn't belong to me. > > > > A process on N2 is calling my gen_server (on N1) via an rpc:call. > > > > Is there a way to know the Pid of that process? > > Based on a half hour study of lib/kernel/src/rpc.erl in OTP-19.3 I didn't > think so, but now I think I may have found a way. > > On the callee node, rpc has a gen_server (rex) which handles incoming > calls. > When it calls the {Mod,Fun,Args} from the rpc:call, it does so > asynchronously > via a temporary process (see handle_call_call/6). There is a mapping from > that process' Pid to the From passed to rex' handle_call (called 'To' in > the code) > in rex' state, which you could dig out using sys:get_state or :get_status. > > But on the caller node, rpc:call also uses a temporary process when doing > the > gen_server:call to the callee node (see do_call/3), and the only > association > between that process and the application process doing the rpc:call is > buried > in the local variables of do_call/3 and the temporary process. (So if you > do > the sys:get_state on the callee node, you'd end up with the Pid of the > temporary > process on the caller node.) > > ... but now it struck me that the temporary process is monitored by the > process > doing the rpc:call, which erlang:process_info(Pid, monitored_by) should be > able > to retrieve. So a reverse rpc:call to the caller node to do that could > work. > > Not tested, will break if the rpc implementation changes, etc. > > Why do you need this anyway? > > > Or, can I log all rpc_call(s) calling my gen_server? > > rex doesn't have that built-in, but you could modify it to do so (it's on > the > node you control) > > /Mikael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Thu Apr 27 07:51:38 2017 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Wed, 26 Apr 2017 22:51:38 -0700 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> Message-ID: NB: The observations of a Linux weenie (that is, me) follow: On 04/26/2017 05:46 PM, Richard A. O'Keefe wrote: > Solaris documentation is clear that you should use > bind(s, (struct sockaddr *) &addr, > strlen(addr.sun_path) + sizeof (addr.sun_family)) Which is _really_ confusing to me. Even on Solaris, sockaddr_un.sun_path is an array of fixed size, right? And for INET and INET6 sockets, it seem that the Solaris documentation says that argument three to bind is the length of the _entire struct_. Why does it make sense to force client code to special-case UNIX domain sockets and get strlen involved in calculating bind's third argument? Why not have the kernel switch on sa_family_t and have any required special-casing neatly hidden from the client? [0] >> So it seems it is not a violation of Posix to use a su->sun_path that starts >> with a 0 combined with an address_len of sizeof(*su). > > It may not be, but the path is still "" and thus a run-time error. > ... > POSIX systems *shouldn't* do this, but Linux *does*. My search skills might be weak, but I'm unable to find the section in the POSIX documentation that fully addresses UNIX domain sockets. I think the most it says on the topic is in the documentation for sys/un.h, which pretty much says "sockaddr_un must have sun_family and sun_path members. sun_path might be any size at all. Cast this struct to struct_sockaddr when you use it.". [1] What did I miss? If we move from thinking of sockaddr_un.sun_path as a null-terminated string, and move to thinking of it as a fixed-length sequence of bytes that might happen to be a null-terminated string, but might also be interpreted in any other way by the underlying OS, does Linux's behavior seem more sensible? I mean, if we're no longer constrained by the restriction that sun_path _must_ be something that's a valid filename, [2] we can get very creative with the interpretation of sun_path, no? From a practical perspective, it seems to me that as long as you zero out the entirety of the sockaddr_un before you start filling it and ensure that whatever you copy into sun_path is not longer than sun_path, then your code _should_ just work. And, mostly unrelated to all that: > It does not say the *size* of the structure but the > *length* of the structure. To be fair, the documentation seems to consistently uses "length of [the] structure" where one might expect to see "size". See the documentation for connect, for example. ;) [0] And while we're talking about inconsistent interfaces, why do the BSDs have a one-byte length as the first member of all(?) of its socket structs? This wart appears to be absent in both Solaris and Linux. [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html [2] Because Linux's abstract namespace UNIX sockets are (just like IP sockets) AFAICT _not_ represented as files in the filesystem. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From aschultz@REDACTED Thu Apr 27 08:52:12 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 27 Apr 2017 08:52:12 +0200 (CEST) Subject: [erlang-questions] Who called my gen_server? In-Reply-To: References: Message-ID: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> Hi, ----- On Apr 24, 2017, at 5:43 PM, Frank Muller wrote: > Hi guys, > I've two connected nodes: N1, N2. > N1 is mine and I've a gen_server running on it. > N2 is external and doesn't belong to me. > A process on N2 is calling my gen_server (on N1) via an rpc:call. You do know that the rpc server is exactly one process per Erlang node. Pushing to many calls through it will degrade performance. If you have any influence on the design of node N2, change it to call your N1 gen_server directly without the intermediate rpc process. You could either register your server with a global name, use another global registry (e.g. gproc), or a myriad of other ways to reach your gen_server without the rpc module. That way, `From` in the handle_call method will contain the PID of the caller. Andreas > Is there a way to know the Pid of that process? > Or, can I log all rpc_call(s) calling my gen_server? > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From botanyzh@REDACTED Thu Apr 27 08:02:52 2017 From: botanyzh@REDACTED (=?gb2312?B?1dQguro=?=) Date: Thu, 27 Apr 2017 06:02:52 +0000 Subject: [erlang-questions] =?gb2312?b?tPC4tDogIFByb3Bvc2FsIHRvIHJlbW92?= =?gb2312?b?ZSB0dXBsZSBkaXNwYXRjaGVzIGZyb20gRXJsYW5n?= In-Reply-To: References: Message-ID: This is really cool for both erlang and elixir defmodule MyBianry do def size({_, binary}) do :erlang.size(binary) end end defmodule MyTuple do def size({_, tuple} ) do :erlang.size(tuple) end end defmodule MyList do def size({_, list}) do length(list) end end iex(24)> var={MyBianry, <<1,2,3,4>>} {MyBianry, <<1, 2, 3, 4>>} iex(25)> var.size 4 iex(26)> var={MyTuple, {1,2,3,4}} {MyTuple, {1, 2, 3, 4}} iex(27)> var.size 4 iex(28)> var={MyList, [1,2,3,4,5,6]} {MyList, [1, 2, 3, 4, 5, 6]} iex(29)> var.size 6 iex(9)> c "myList.erl" [:myList] iex(10)> c "myBinary.erl" [:myBinary] iex(11)> c "myTuple.erl" [:myTuple] iex(12)> :myList.msize({:a,[1,2,3,4]}) 4 iex(13)> var={:myList,[1,2,3,4]} {:myList, [1, 2, 3, 4]} iex(14)> var.msize 4 iex(15)> var={:myBinary,<<1,2,3,4>>} {:myBinary, <<1, 2, 3, 4>>} iex(16)> var.msize 4 iex(17)> var={:myTuple,{1,2,3,4}} {:myTuple, {1, 2, 3, 4}} iex(18)> var.msize 4 iex(19)> ??? Windows 10 ????? ???: Jos? Valim ????: 2017?4?14? 20:13 ???: Erlang ??: [erlang-questions] Proposal to remove tuple dispatches from Erlang Hi everyone, I would like to propose to remove "tuple dispatches" from Erlang. The tuple dispatch is the ability to invoke a function or call erlang:apply/3 on a tuple as first argument: Eshell V9.0 (abort with ^G) 1> Var = dict:new(). {dict,0,16,16,8,80,48, {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}} 2> Var:size(). 0 This behaviour is considered by most in the community to be undesired and confusing, as it obfuscates the meaning of the code and adds indirection. I have also heard this behaviour made it harder to add some optimizations to the VM. I would love if someone more knowledgeable on the area could confirm or deny this. If true, it is also a strong argument to remove such behaviour. Another reason for removing it is that the behaviour can be implemented as needed by adding is_tuple/1 checks to the code or more programmatically by using a parse transforms (see note 1 at the bottom for a limitation though). Therefore those who need the behaviour can include it only when necessary and we don't impose it as a semantics to the whole language (and ecosystem). I can think of two strategies for removing the behaviour: 1. Clean-cut: the code responsible for tuple dispatching will be completely removed from the VM and a parse transform will be made available. The parse transform could be part of Erlang/OTP or a separate repository. This change is backwards incompatible at the BEAM level. Code that relies on tuple dispatch without the parse transform on OTP 19 will not work on OTP 20. However, the parse transform should work with any OTP version, so if the parse transform is used during compilation, the code is guaranteed to work on OTP 19 and earlier as well as on OTP 20 onwards. 2. New byte codes: if we don't want to break backwards compatibility at the BEAM level, I believe our only option is to introduce new bytecodes and a new apply BIF. Usage of the old BIFs and bytecode could emit warnings while we phase them out. A compiler option or parse transform should still be made available if a developer relying on those features wants their code to run without warnings. Please let me know if there are other options available, I will be glad to send patches and implement the required parse-transforms if this is accepted by the OTP team. Jos? Valim www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D Note 1. A parse-transform would be unable to make the following code work in the same way as today: erlang:apply(erlang, apply, [dict:new(), size, []]) Although I would consider it highly unlikely to exist so it should not be a point of contention. -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Apr 27 09:49:28 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 27 Apr 2017 07:49:28 +0000 Subject: [erlang-questions] Who called my gen_server? In-Reply-To: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> References: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> Message-ID: Hi Andreas, No I didn't know that issue with "rpc". But is switching to a different solution as you mentioned (gproc, etc.) will help me solve "the who's calling me"? /Frank Schultz wrote : > Hi, > > ----- On Apr 24, 2017, at 5:43 PM, Frank Muller < > frank.muller.erl@REDACTED> wrote: > > Hi guys, > > I've two connected nodes: N1, N2. > > N1 is mine and I've a gen_server running on it. > N2 is external and doesn't belong to me. > > A process on N2 is calling my gen_server (on N1) via an rpc:call. > > > You do know that the rpc server is exactly one process per Erlang > node. Pushing to many calls through it will degrade performance. > > If you have any influence on the design of node N2, change it to call > your N1 gen_server directly without the intermediate rpc process. > > You could either register your server with a global name, use another > global registry (e.g. gproc), or a myriad of other ways to reach your > gen_server without the rpc module. > > That way, `From` in the handle_call method will contain the PID of > the caller. > > Andreas > > > Is there a way to know the Pid of that process? > Or, can I log all rpc_call(s) calling my gen_server? > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Thu Apr 27 10:11:55 2017 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 27 Apr 2017 10:11:55 +0200 (CEST) Subject: [erlang-questions] Who called my gen_server? In-Reply-To: References: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> Message-ID: <867180675.1062318.1493280715935.JavaMail.zimbra@tpip.net> Hi Frank, ----- On Apr 27, 2017, at 9:49 AM, Frank Muller wrote: > Hi Andreas, > No I didn't know that issue with "rpc". > But is switching to a different solution as you mentioned (gproc, etc.) will > help me solve "the who's calling me"? Only for handle_call case. handle_cast and handle_info won't get that information. You have to include the source in the actual message you are sending. Andreas > /Frank > Schultz< [ mailto:aschultz@REDACTED | aschultz@REDACTED ] > wrote : >> Hi, >> ----- On Apr 24, 2017, at 5:43 PM, Frank Muller < [ >> mailto:frank.muller.erl@REDACTED | frank.muller.erl@REDACTED ] > wrote: >>> Hi guys, >>> I've two connected nodes: N1, N2. >>> N1 is mine and I've a gen_server running on it. >>> N2 is external and doesn't belong to me. >>> A process on N2 is calling my gen_server (on N1) via an rpc:call. >> You do know that the rpc server is exactly one process per Erlang >> node. Pushing to many calls through it will degrade performance. >> If you have any influence on the design of node N2, change it to call >> your N1 gen_server directly without the intermediate rpc process. >> You could either register your server with a global name, use another >> global registry (e.g. gproc), or a myriad of other ways to reach your >> gen_server without the rpc module. >> That way, `From` in the handle_call method will contain the PID of >> the caller. >> Andreas >>> Is there a way to know the Pid of that process? >>> Or, can I log all rpc_call(s) calling my gen_server? >>> /Frank >>> _______________________________________________ >>> erlang-questions mailing list >>> [ mailto:erlang-questions@REDACTED | erlang-questions@REDACTED ] >>> [ http://erlang.org/mailman/listinfo/erlang-questions | >>> http://erlang.org/mailman/listinfo/erlang-questions ] -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Thu Apr 27 11:30:51 2017 From: v@REDACTED (Valentin Micic) Date: Thu, 27 Apr 2017 11:30:51 +0200 Subject: [erlang-questions] Who called my gen_server? In-Reply-To: References: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> Message-ID: <5BB26E00-1711-46F0-AC6F-0850B96348F8@pharos-avantgard.com> > Or, can I log all rpc_call(s) calling my gen_server? > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions General comments about appropriateness of rpc notwithstanding?. Consider following statements (you may run them from the shell): (tcap2@REDACTED)225> f(REX), REX=whereis(rex). <0.12.0> (tcap2@REDACTED)226> unregister( rex ). true (tcap2@REDACTED)227> catch register( rex, self() ). true Then, you could go to the remote node's shell and run, say: (tcap3@REDACTED)3> rpc:call( tcap2@REDACTED, erlang, now, [], 1000 ). (NOTE 1: the caller wil timeout after one second) At this point, your first node will receive a message, such that: (tcap2@REDACTED)228> flush(). Shell got {'$gen_call',{<7155.38.0>,#Ref<7155.0.0.59>}, {call,erlang,now,[],<7155.31.0>}} ok (NOTE 2: <<7155.38.0>> is pid() of the process that issued the call) (NOTE 3: <<7155.31.0>> is pid() of the group leader) Thus, you could exploit this in order to write your own logger, which would front rex: Consider below for incorporation in your code: ... -export( [init/1, logger/0] ). ... -record( state, {rex, logger} ). ... init( _ ) -> ... % --------------- % Redirecting REX % --------------- OldREX = whereis( REX ), unregister( rex ), register( rex, self() ), main_loop( #state{rex=OldREX, logger=spawn(?MODULE, logger, []} ) . main_loop( S=#state{} ) -> NewS = receive Request -> fwd_request(Request, S) end, main_loop( NewS ) ; main_loop( {shutdown, #state{rex=OldREX, logger=LPID}} ) -> % --------------- % Exit gracefully % --------------- unregister( rex ), register( rex, OldRex ), LPID ! shutdown, exit( normal ) . fwd_request( Request={'$gen_call',{CallerPID,CallerRef}, {call,Mod,Func,Args,_CallerGroupLeader}}, S=#state{rex=OldRex, logger=LPID} ) -> OldRex ! Request, LPID ! {log, CallerPid, CallerRef, Mod, Func, Args}, S ; fwd_request( shutdown, S ) -> {shutdown, S}; fwd_request( _, S ) -> S. % ------ % LOGGER % ------ logger() -> receive {log, CallerPid, CallerRef, Mod, Func, Args} -> ... Do some logging stuff ... ; shutdown -> exit( normal ); _ -> void end, logger() . Kind regards V/ PS Just want to state the obvious: I did not compile this code. It could contain bugs, but it illustrates the point. -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Apr 27 15:18:34 2017 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 27 Apr 2017 13:18:34 +0000 Subject: [erlang-questions] Who called my gen_server? In-Reply-To: <5BB26E00-1711-46F0-AC6F-0850B96348F8@pharos-avantgard.com> References: <1971935372.1061286.1493275932862.JavaMail.zimbra@tpip.net> <5BB26E00-1711-46F0-AC6F-0850B96348F8@pharos-avantgard.com> Message-ID: Hi Valentin, I clearly see your idea. I'll play with it and let you know. To everyone: thanks guys. I've learnt a lot today. /Frank Valentin Micic wrote : > Or, can I log all rpc_call(s) calling my gen_server? >> >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > General comments about appropriateness of rpc notwithstanding?. > > Consider following statements (you may run them from the shell): > > (tcap2@REDACTED)225> f(REX), REX=whereis(rex). > <0.12.0> > (tcap2@REDACTED)226> unregister( rex ). > true > (tcap2@REDACTED)227> catch register( rex, self() ). > true > > Then, you could go to the remote node's shell and run, say: > > (tcap3@REDACTED)3> *rpc:call( tcap2@REDACTED, erlang, now, [], 1000 ).* > > *(NOTE 1: the caller wil timeout after one second)* > > At this point, your first node will receive a message, such that: > > (tcap2@REDACTED)228> flush(). > Shell got {'$gen_call',{<7155.*38.0*>,#Ref<7155.0.0.59>}, > {call,erlang,now,[],<7155.*31.0*>}} > ok > > *(NOTE 2: <<7155.38.0>> is pid() of the process that issued the call)* > *(NOTE 3: <<7155.31.0>> is pid() of the group leader)* > > Thus, you could exploit this in order to write your own logger, which > would front *rex*: > > Consider below for incorporation in your code: > > > ... > -export( [init/1, logger/0] ). > ... > -record( state, {rex, logger} ). > ... > init( _ ) > -> > ... > % --------------- > % Redirecting REX > % --------------- > OldREX = whereis( REX ), > unregister( rex ), > register( rex, self() ), > main_loop( #state{rex=OldREX, logger=spawn(?MODULE, logger, []} ) > . > > main_loop( S=#state{} ) > -> > NewS = receive > Request -> fwd_request(Request, S) > end, > main_loop( NewS ) > ; > main_loop( {shutdown, #state{rex=OldREX, logger=LPID}} ) > -> > % --------------- > % Exit gracefully > % --------------- > unregister( rex ), > register( rex, OldRex ), > LPID ! shutdown, > exit( normal ) > . > > fwd_request( Request={'$gen_call',{CallerPID,CallerRef}, > {call,Mod,Func,Args,_CallerGroupLeader}}, S=#state{rex=OldRex, > logger=LPID} ) > -> > OldRex ! Request, > LPID ! {log, CallerPid, CallerRef, Mod, Func, Args}, > S > ; > fwd_request( shutdown, S ) -> {shutdown, S}; > fwd_request( _, S ) -> S. > > % ------ > % LOGGER > % ------ > logger() > -> > receive > {log, CallerPid, CallerRef, Mod, Func, Args} > -> > ... *Do some logging stuff ...* > ; > shutdown -> exit( normal ); > _ -> void > end, > logger() > . > > Kind regards > > V/ > > PS > Just want to state the obvious: I did not compile this code. It could > contain bugs, but it illustrates the point. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Apr 27 15:53:54 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 27 Apr 2017 15:53:54 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> Message-ID: <20170427135354.GA93725@erix.ericsson.se> On Wed, Apr 26, 2017 at 10:51:38PM -0700, Kenneth Lakin wrote: > NB: The observations of a Linux weenie (that is, me) follow: > > On 04/26/2017 05:46 PM, Richard A. O'Keefe wrote: > > Solaris documentation is clear that you should use > > bind(s, (struct sockaddr *) &addr, > > strlen(addr.sun_path) + sizeof (addr.sun_family)) > > Which is _really_ confusing to me. > > Even on Solaris, sockaddr_un.sun_path is an array of fixed size, right? > And for INET and INET6 sockets, it seem that the Solaris documentation > says that argument three to bind is the length of the _entire struct_. Well, maybe. But AF_INET and AF_INET6 socket addresses are not variable length. That it is an array of fixed size only means that you have a pre-allocated area whithin which the address shall be stored. The alternative would be to have a pointer but then we get the problem of having to allocate and free the memory for the address correctly. This is an example of an variable length address. In fact Linux does not always use all 108 bytes. You can create a socket with an empty address and then you get an address with a leading '\0' and some 6 characters not corresponding to a filesystem name. > > Why does it make sense to force client code to special-case UNIX domain > sockets and get strlen involved in calculating bind's third argument? > Why not have the kernel switch on sa_family_t and have any required > special-casing neatly hidden from the client? [0] Since you for AF_UNIX have to know that the address corresponds to a filesystem path name and that you have to explicitly use unlink(2) to remove the file that bind(3) creates you already are forced to specialize client code. > > >> So it seems it is not a violation of Posix to use a su->sun_path that starts > >> with a 0 combined with an address_len of sizeof(*su). > > > > It may not be, but the path is still "" and thus a run-time error. > > ... > > POSIX systems *shouldn't* do this, but Linux *does*. > > My search skills might be weak, but I'm unable to find the section in > the POSIX documentation that fully addresses UNIX domain sockets. I > think the most it says on the topic is in the documentation for > sys/un.h, which pretty much says "sockaddr_un must have sun_family and > sun_path members. sun_path might be any size at all. Cast this struct to > struct_sockaddr when you use it.". [1] What did I miss? There is an example in the Posix documentation of bind() that says sun_path is a path name and that the empty string should result in ENOENT. > > If we move from thinking of sockaddr_un.sun_path as a null-terminated > string, and move to thinking of it as a fixed-length sequence of bytes > that might happen to be a null-terminated string, but might also be > interpreted in any other way by the underlying OS, does Linux's behavior > seem more sensible? I mean, if we're no longer constrained by the > restriction that sun_path _must_ be something that's a valid filename, > [2] we can get very creative with the interpretation of sun_path, no? Yes. Creative. But it would have been more kosher if Linux had defined a new address family. They have violated old standards in a creative way. > > From a practical perspective, it seems to me that as long as you zero > out the entirety of the sockaddr_un before you start filling it and > ensure that whatever you copy into sun_path is not longer than sun_path, > then your code _should_ just work. Yes. But what happens if you do that on FreeBSD? Probably nothing since sun_path is zero terminated within the size of the structure. But unix(4) tells you to use SUN_LEN(su). So there is no way to fulfill both Linux's and FreeBSD's API definitions. And so far nobody has provided a way to configure test for it. Thank you Linus! I am inclined to just ignore the FreeBSD man page advice since it _may_ allow SUN_LEN(su) =< su->sun_len =< sizeof(*su)... > > And, mostly unrelated to all that: > > > It does not say the *size* of the structure but the > > *length* of the structure. > > To be fair, the documentation seems to consistently uses "length of > [the] structure" where one might expect to see "size". See the > documentation for connect, for example. ;) > > [0] And while we're talking about inconsistent interfaces, why do the > BSDs have a one-byte length as the first member of all(?) of its socket > structs? This wart appears to be absent in both Solaris and Linux. This simplifies handling variable length addresses for example for routing sockets. [3] It was added with 4.3BSD-Reno (1990) and probably existed in SunOS 4 but did not propagate to Solaris. > [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html > [2] Because Linux's abstract namespace UNIX sockets are (just like IP > sockets) AFAICT _not_ represented as files in the filesystem. In blatant contradiction to Posix. > > [3]: W. Richard Stevens: Unix Network Programming Volume 1 3:rd Edition Chapter 3.2 Socket Address Structures. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From comptekki@REDACTED Thu Apr 27 19:01:19 2017 From: comptekki@REDACTED (Wes James) Date: Thu, 27 Apr 2017 11:01:19 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: I tried this today with 19.3. I have to set the service to Automatic (Delaye Start) for it to work, like in 18.3. I looked in the eventviewer and I see these errors: Error Service Control Manager Under General Tab The service terminated with the following error: The configuration registry key could not be read. A little further down Error ErlSrv Under General Tab servicename: Could not get registry keys for erlang service. Which is odd since it does work when delayed. It's like it's waiting for some other service to start but hasn't yet so it quits. I'm not sure what it's depending on before starting successfully in delayed start. -wes On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson wrote: > Well you got it run at all, that is something, I have disabled our testing > of > services on Windows 10 for now, master branch, since I could not get it > run at all. > > From what I googled and understood you will need start the services from > an elevated shell, an administer user is not enough. > But you might already do that or I got it wrong. > > Anyway, our Windows knowledge, and priority, is not at the highest level. > So any help would be appreciated. > > /Dan > > On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: > >> I use erlsrv to install an erlang application on windows. Windows 7 >> works fine, but on windows 10, I have to go in and change the service from >> automatic to Automatic (delayed start), otherwise the app never runs on >> system start up. Any idea how to fix this? >> >> I just found this windows command: >> >> sc config SVCNAME start= delayed-auto >> >> I can use that in the install script. >> >> I'd still like it to start with just automatic. With the delayed option, >> it takes several extra minutes for the service to start running. >> >> Thanks, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Apr 28 06:46:17 2017 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 28 Apr 2017 16:46:17 +1200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> Message-ID: <489BD294-2B8A-43A0-B16E-F442A99A433B@cs.otago.ac.nz> > On 27/04/2017, at 5:51 PM, Kenneth Lakin wrote: > > NB: The observations of a Linux weenie (that is, me) follow: > > On 04/26/2017 05:46 PM, Richard A. O'Keefe wrote: >> Solaris documentation is clear that you should use >> bind(s, (struct sockaddr *) &addr, >> strlen(addr.sun_path) + sizeof (addr.sun_family)) > > Which is _really_ confusing to me. > > Even on Solaris, sockaddr_un.sun_path is an array of fixed size, right? So? It shouldn't be that confusing. The third parameter of bind() is the length of the *address*, not the size of the *struct*. > And for INET and INET6 sockets, For IPv4 and IPv6 addresses, the length of the address and the size of the struct happen to be the same number. For Unix domain address, the length of the address (that is, the number of bytes in the struct that are actual information- bearing payload) is LESS than the size of the struct. > it seem that the Solaris documentation > says that argument three to bind is the length of the _entire struct_. It really doesn't help that there are two socket interfaces in Solaris. bind(3SOCKET) calls the 3rd argument "namelen" (note "len", not "sz"), doesn't really say what the arguments are, but notes Binding a name in the UNIX domain creates a socket in the file system that must be deleted by the caller when it is no longer needed by using unlink(2). The rules used in name binding vary between communication domains. bind(3XNET) calls the third argument "address_len" (which is more accurate, because the length includes some bytes before the name starts; note that it is still "len" not "sz") No, that's not what it says. First, the manpage I'm looking at, second the Programming Interfaces Guide is careful to say "length" (not "size") and says that it "Specifies the length of" the address, not the size. Neither manual page includes an example. https://docs.oracle.com/cd/E26502_01/html/E35299/sockets-18552.html#sockets-6 has an example, and the example uses a sockaddr_in6, and the example uses sizeof because for a sockaddr_in6 the size and length coincide (there is nothing NUL-terminated). It's https://docs.oracle.com/cd/E26502_01/html/E35299/portmapper-51908.html#scrolltoc that discusses UNIX-domain sockets (appendix A of the Sockets guide). And it is really pretty explicit: bind (s, name, namelen); The socket handle is s. The bound name is a byte string that is interpreted by the supporting protocols. UNIX family names contain a path name and a family. The example shows binding the name /tmp/foo to a UNIX family socket. #include ... struct sockaddr_un addr; ... strlcpy(addr.sun_path, "/tmp/foo", sizeof(addr.sun_path)); addr.sun_family = AF_UNIX; bind(s, (struct sockaddr *) &addr, strlen(addr.sun_path) + sizeof (addr.sun_family)); When determining the size of an AF_UNIX socket address, null bytes are not counted, which is why you can use strlen(3C). For what it is worth, I have /usr/include/sys/un.h open in an OpenBSD window, and I read therein /* actual length of an initialized sockaddr_un */ #define SUN_LEN(su) \ (sizeof(*(su)) - sizeof((su)->sun_path) + strlen((su)->sun_path)) and in Mac OS X 10.11.6, we find #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) /* actual length of an initialized sockaddr_un */ #define SUN_LEN(su) \ (sizeof(*(su)) - sizeof((su)->sun_path) + strlen((su)->sun_path)) #endif /* (!_POSIX_C_SOURCE || _DARWIN_C_SOURCE) */ So it seems pretty clear that the right way to determine the *LENGTH* of a socket address (a) may and does differ between address families (b) is to use the sizeof the whole record for IPv4 and IPv6 -- I'm actually wondering whether any attention is paid -- to it at all in those cases, since the sun_family field -- determines everything else (c) involves computing the length of the string for UNIX domain sockets (d) looks different between Solaris and *BSD because Solaris doesn't *have* a sun_len field, so #define AF_UNIX_LENGTH(su) \ sizeof su - sizeof su.sun_path + strlen(su.sun_path) or perhaps offsetof(struct sockaddr_un, sun_path) + strlen(su.sun_path) > Why does it make sense to force client code to special-case UNIX domain > sockets and get strlen involved in calculating bind's third argument? The client *already* has to special-case UNIX domain sockets: you don't have to unlink() an IPv4 or IPv6 address when you've finished with it. > Why not have the kernel switch on sa_family_t and have any required > special-casing neatly hidden from the client? [0] You are asking me to read the minds of people who decided that Unix domain socket names should be UNIX-style path names that actually identified objects in the file system *BUT* the array that holds the name should not be big enough for even the longest legal *component* of a file name? Perhaps they intended that (a) the size of the struct should be a tolerable default but (b) people should be able to get away with using something like the struct hack to make *bigger* records if they really needed them, in which case (c) the 3rd parameter might be *bigger* than the sizeof the official struct and still work. From long experience with other operating systems before BSD got a socket interface, I find the entire idea of *binary* IPv4 and IPv6 addresses rather disgusting and above all things like storing the binary data in *network* order rather then native order? FEH! The interface should in my view have been bind(socket_fd, AF_WHATEVER, "/") and the Linux extension should have been announced by a new AF_LINUX. In short, I'm not saying the interface is REASONABLE, I'm saying it is what it is. > > My search skills might be weak, but I'm unable to find the section in > the POSIX documentation that fully addresses UNIX domain sockets. I > think the most it says on the topic is in the documentation for > sys/un.h, which pretty much says "sockaddr_un must have sun_family and > sun_path members. sun_path might be any size at all. Cast this struct to > struct_sockaddr when you use it.". [1] What did I miss? How much are you paying me for this? > > If we move from thinking of sockaddr_un.sun_path as a null-terminated > string Then we move away from what *IS* documented into the realm of wishful thinking. To repeat: what Solaris, Darwin, and OpenBSD all actually say boils down to + and the AIX documentation also says to use SUN_LEN. I don't have an HP-UX system to try things on, but chapter 6 of http://www.cs.put.poznan.pl/wswitala/download/pdf/B2355-90136.pdf - says that sun_path is 92 bytes long - says to use "size of struct sockaddr_un". which disagrees with the others. Disagreement between existing systems is probably the best explanation of vagueness in POSIX. Having looked at the OpenSolaris kernel, I reported earlier that it actually checks within the size you give it and stops early if it finds a NUL, so I *suspect* that the shorter size will actually work in HP-UX. Ad fontes! An Advanced 4.3BSD Interprocess Communication Tutorial by Samuel J. Leffler, Robert S. Fabry, William N. Joy, Phil Lapsley (UCB) Steve Miller, and Chris Torek (UM) If one wanted to bind the name ??/tmp/foo?? to a UNIX domain socket, the following code would be used: #include ... struct sockaddr_un addr; ... strcpy(addr.sun_path, "/tmp/foo"); addr.sun_family = AF_UNIX; bind(s, (struct sockaddr *) &addr, strlen(addr.sun_path) + sizeof (addr.sun_family)); (I had a 4.2 BSD documentation set as recently as 2 years ago but had to discard it when the University took back a wing of our building, so this is as early as I can readily go.) > , and move to thinking of it as a fixed-length sequence of bytes > that might happen to be a null-terminated string, The designers have always said "The bound name is a VARIABLE LENGTH BYTE STRING which is interpreted by the supporting protocol(s)." > But might also be > interpreted in any other way by the underlying OS, does Linux's behavior > seem more sensible? I mean, if we're no longer constrained by the > restriction that sun_path _must_ be something that's a valid filename, > [2] we can get very creative with the interpretation of sun_path, no? I do not understand why we are no longer constrained by that restriction. An *operating system* (such as Linux) may turn an otherwise illegal argument (an empty path) into something else. But if we want code to be portable, *we* can't unilaterally abandon all compatibility restraints. > > From a practical perspective, it seems to me that as long as you zero > out the entirety of the sockaddr_un before you start filling it and > ensure that whatever you copy into sun_path is not longer than sun_path, > then your code _should_ just work. I believe I said in a previous message that I *believed* any length between the actual length and the size of the struct would work in Solaris -- in UNIX systems where sockaddr_un has an sun_len field that field must be omitted from the size calculation -- would work and *suspected* that work in other Unices. But I really have other things to do with my time (like writing exam questions, it's that time of year here) than fossick around in kernels. One is enough. Your code "_should_ just work" if you do what the available documentation says to do". In this case, the HP-UX documentation says that would be OK; Solaris, BSD, Darwin, AIX, and POSIX do not. Maybe it will, maybe it won't, maybe it will today but not tomorrow. > > And, mostly unrelated to all that: > >> It does not say the *size* of the structure but the >> *length* of the structure. > > To be fair, the documentation seems to consistently uses "length of > [the] structure" where one might expect to see "size". See the > documentation for connect, for example. ;) That's not a different issue, that's the SAME issue. It is talking about the *length* (not size) of a socket address, and in Solaris, *BSD, Darwin, &c it really should be the length, not the size. > > [0] And while we're talking about inconsistent interfaces, why do the > BSDs have a one-byte length as the first member of all(?) of its socket > structs? This wart appears to be absent in both Solaris and Linux. It was a change between 4.3BSD and 4.4BSD. Presumably the answer can still be found in a revision history in a dusty archive, possibly orbiting Barnard's Star. > [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html > [2] Because Linux's abstract namespace UNIX sockets are (just like IP > sockets) AFAICT _not_ represented as files in the filesystem. To the best of my knowledge, that's the *point* of them. If you want to use a path name as a UNIX domain address, you have to ensure that it is a valid path name that you have permission to create (but does not exist before you create the first socket with it) and that you must unlink when you have finished with it. Given that, the "special casing" required to get the length right is a minor issue, don't you think? My argument goes like this: - Linux is too important to ignore - The Linux extension is a genuinely useful feature done in a rather unpleasant way - If you are using AF_UNIX in the portable way, you should provide the *length* of the address, not the *size* of the struct. I remember it being that way in 4.2 BSD and have cited material from some of the key BSD people that that was certainly the intent in 4.3BSD. - But if you care about HP-UX, you had better do some experiments. Honestly, just doing the right thing is less effort than arguing about it. From raimo+erlang-questions@REDACTED Fri Apr 28 10:42:00 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 28 Apr 2017 10:42:00 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <489BD294-2B8A-43A0-B16E-F442A99A433B@cs.otago.ac.nz> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> <489BD294-2B8A-43A0-B16E-F442A99A433B@cs.otago.ac.nz> Message-ID: <20170428084200.GA79301@erix.ericsson.se> On Fri, Apr 28, 2017 at 04:46:17PM +1200, Richard A. O'Keefe wrote: : > > My argument goes like this: > - Linux is too important to ignore > - The Linux extension is a genuinely useful feature > done in a rather unpleasant way > - If you are using AF_UNIX in the portable way, you should > provide the *length* of the address, not the *size* of the > struct. I remember it being that way in 4.2 BSD and have > cited material from some of the key BSD people that that was > certainly the intent in 4.3BSD. > - But if you care about HP-UX, you had better do some experiments. > > Honestly, just doing the right thing is less effort than arguing > about it. Precisely. Well put! But what is the "right thing"? The current implementation passes the address given to it as is, but ensures it is correctly terminated as in appending a '\0' at the end that is not counted for address_len, and if that does not fit in the defined size it bails out. That felt like the "right thing" when I wrote it, but you can not set the last byte of the address to non-zero. The new "right thing" could be to add an exception to the above when there is a '\0' first in the address, since that hopefully will count as a correct termination on OS:es that do not understand abstract addresses. The question is if there is any danger in this because we then give a contradiction to e.g bind() because we have not used SUN_LEN(su) as we were told to do...? I believe not, and will try to find the time to do this after our OTP-20.rc1 pre-release. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From S.J.Thompson@REDACTED Fri Apr 28 10:50:56 2017 From: S.J.Thompson@REDACTED (Simon Thompson) Date: Fri, 28 Apr 2017 09:50:56 +0100 Subject: [erlang-questions] Trends in Functional Programming in Education TFPIE 2017, Canterbury 22 June, SECOND CALL FOR PAPERS Message-ID: <801F8F64-AE0E-4C5F-8A08-11C7CDDEAAEC@kent.ac.uk> TFPIE 2017 Trends in Functional Programming in Education, 2017 https://www.cs.kent.ac.uk/people/staff/sjt/TFPIE2017/ The sixth workshop on Trends in Functional Programming in Education, 2017, which is to be held on the Canterbury campus of the University of Kent on Thursday, 22 June, following the 2017 TFP meeting on 19?21 June. TFPIE workshops have previously been held in St Andrews, Scotland (2012), Provo Utah, USA (2013), Soesterberg, The Netherlands (2014), and Sophia-Antipolis, France (2015), College Park, USA (2016). A particular topic of this year's TFPIE will be MOOCs and other online learning, and as well as a session on this, we're looking forward to announcing a keynote speaker in this area very soon. The goal of TFPIE is to gather researchers, teachers and professionals that use, or are interested in the use of, functional programming in education. TFPIE aims to be a venue where novel ideas, classroom-tested ideas and work-in-progress on the use of functional programming in education are discussed. The one-day workshop will foster a spirit of open discussion by having a review process for publication after the workshop. The program chair of TFPIE 2017 will screen submissions to ensure that all presentations are within scope and are of interest to participants. After the workshop, presenters will be invited to submit revised versions of their articles for publication in the journal Electronic Proceedings in Theoretical Computer Science (EPTCS). Second call for papers TFPIE 2017 welcomes submissions describing techniques used in the classroom, tools used in and/or developed for the classroom and any creative use of functional programming (FP) to aid education in or outside Computer Science. Topics of interest include, but are not limited to: - FP and beginning CS students - FP and Computational Thinking - FP and Artificial Intelligence - FP in Robotics - FP and Music - Advanced FP for undergraduates - FP in graduate education - Engaging students in research using FP - FP in Programming Languages - FP in the high school curriculum - FP as a stepping stone to other CS topics - FP and Philosophy - The pedagogy of teaching FP - FP and e-learning: MOOCs, automated assessment etc. - Best Lectures ? more details below In addition to papers, we are requesting best lecture presentations. What?s your best lecture topic in an FP related course? Do you have a fun way to present FP concepts to novices or perhaps an especially interesting presentation of a difficult topic? In either case, please consider sharing it. Best lecture topics will be selected for presentation based on a short abstract describing the lecture and its interest to TFPIE attendees. Submission Potential presenters are invited to submit an extended abstract (4-6 pages) or a draft paper (up to 16 pages) in EPTCS style. The authors of accepted presentations will have their preprints and their slides made available on the workshop's website. Papers and abstracts can be submitted via easychair at the following link: https://easychair.org/conferences/?conf=tfpie2017 After the workshop, presenters will be invited to submit (a revised version of) their article for review. The PC will select the best articles for publication in the journal Electronic Proceedings in Theoretical Computer Science (EPTCS). Articles rejected for presentation and extended abstracts will not be formally reviewed by the PC. Programme committee Dr Laura Castro, University of A Coru?a Prof Ralf L?mmel, University of Koblenz-Landau Dr Elena Machkasova, University of Minnesota, Morris Prof Michel Mauny, Inria, Paris Dr Jeremy Singer, University of Glasgow Prof Simon Thompson, University of Kent (chair) Important dates Submissions of draft papers: 10 May, 2017 Notification: 17 May, 2017 Registration: 11 June, 2017 Workshop: 22 June 2017 Submission for formal review: 18 August, 2017 Notification of acceptance: 6 October, 2017 Camera ready paper: 3 November, 2017 Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From ali.sabil@REDACTED Fri Apr 28 11:34:18 2017 From: ali.sabil@REDACTED (Ali Sabil) Date: Fri, 28 Apr 2017 09:34:18 +0000 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: <20170428084200.GA79301@erix.ericsson.se> References: <0CFB90D6-7921-4AB0-9EC9-9FAA3CB851EA@gmail.com> <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> <489BD294-2B8A-43A0-B16E-F442A99A433B@cs.otago.ac.nz> <20170428084200.GA79301@erix.ericsson.se> Message-ID: Just a suggestion, but would it make sense to handle abstract unix sockets differently at the Erlang level? Basically if the path is a binary or list consider it a standard path and append the null byte, but if it is for example a tuple `{abstract, string() | binary()}` then prepend the null byte instead of appending? On Fri, 28 Apr 2017 at 10:42, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Fri, Apr 28, 2017 at 04:46:17PM +1200, Richard A. O'Keefe wrote: > : > > > > My argument goes like this: > > - Linux is too important to ignore > > - The Linux extension is a genuinely useful feature > > done in a rather unpleasant way > > - If you are using AF_UNIX in the portable way, you should > > provide the *length* of the address, not the *size* of the > > struct. I remember it being that way in 4.2 BSD and have > > cited material from some of the key BSD people that that was > > certainly the intent in 4.3BSD. > > - But if you care about HP-UX, you had better do some experiments. > > > > Honestly, just doing the right thing is less effort than arguing > > about it. > > Precisely. Well put! But what is the "right thing"? > > The current implementation passes the address given to it as is, but > ensures it is correctly terminated as in appending a '\0' at the end > that is not counted for address_len, and if that does not fit > in the defined size it bails out. > > That felt like the "right thing" when I wrote it, but you can not set the > last byte of the address to non-zero. > > The new "right thing" could be to add an exception to the above when there > is a '\0' first in the address, since that hopefully will count as a > correct termination on OS:es that do not understand abstract addresses. > > The question is if there is any danger in this because we then give a > contradiction to e.g bind() because we have not used SUN_LEN(su) as we were > told to do...? > > I believe not, and will try to find the time to do this > after our OTP-20.rc1 pre-release. > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Fri Apr 28 12:11:03 2017 From: mbike2000ru@REDACTED (Dmitry) Date: Fri, 28 Apr 2017 10:11:03 +0000 (UTC) Subject: [erlang-questions] cowlib compiling gives such error : cow_sse.erl:32: syntax error before: ':=' References: <1286139026.305055.1493374263861.ref@mail.yahoo.com> Message-ID: <1286139026.305055.1493374263861@mail.yahoo.com> Hello cowlib - the latest master branch is not compiled in erlang versions less then 19+. Cowlib(master branch) is pulled during get-deps by cowbow How to tell cowboy to pull, say branch 1.3 (not master)? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Apr 28 12:33:18 2017 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 28 Apr 2017 12:33:18 +0200 Subject: [erlang-questions] cowlib compiling gives such error : cow_sse.erl:32: syntax error before: ':=' In-Reply-To: <1286139026.305055.1493374263861@mail.yahoo.com> References: <1286139026.305055.1493374263861.ref@mail.yahoo.com> <1286139026.305055.1493374263861@mail.yahoo.com> Message-ID: <2264df3b-2421-682d-0633-2147eaad1e06@ninenines.eu> On 04/28/2017 12:11 PM, Dmitry wrote: > Hello > > cowlib - the latest master branch is not compiled in erlang versions > less then 19+. > > Cowlib(master branch) is pulled during get-deps by cowbow > > How to tell cowboy to pull, say branch 1.3 (not master)? Cowboy master (and pre-releases) pulls Cowlib master at this time. Your own project can depend on Cowlib (BUILD_DEPS if using Erlang.mk) to fetch it before Cowboy can, and override the version. All three related projects (Cowboy, Cowlib and Gun) will have their next version released this summer require at least Erlang/OTP 19.0 and above. As far as Cowboy is concerned, this decision was taken in August last year and is clearly written in the introduction. Expect more changes to both Cowlib and Cowboy that require 19+ in the next few weeks. -- Lo?c Hoguin https://ninenines.eu From raimo+erlang-questions@REDACTED Fri Apr 28 14:28:48 2017 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 28 Apr 2017 14:28:48 +0200 Subject: [erlang-questions] unix domain sockets with abstract namespace: can't use all 108 bytes In-Reply-To: References: <20170424100109.GA52860@erix.ericsson.se> <20170425141404.GA26809@erix.ericsson.se> <20170426085558.GA87429@erix.ericsson.se> <489BD294-2B8A-43A0-B16E-F442A99A433B@cs.otago.ac.nz> <20170428084200.GA79301@erix.ericsson.se> Message-ID: <20170428122848.GB79301@erix.ericsson.se> On Fri, Apr 28, 2017 at 09:34:18AM +0000, Ali Sabil wrote: > Just a suggestion, but would it make sense to handle abstract unix sockets > differently at the Erlang level? > > Basically if the path is a binary or list consider it a standard path and > append the null byte, but if it is for example a tuple `{abstract, string() > | binary()}` then prepend the null byte instead of appending? > On Fri, 28 Apr 2017 at 10:42, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: It is difficult to reliably detect the other direction i.e in the driver when you get an address from e.g getsockname(); is it an empty string or an abstract address? > > > On Fri, Apr 28, 2017 at 04:46:17PM +1200, Richard A. O'Keefe wrote: > > : > > > > > > My argument goes like this: > > > - Linux is too important to ignore > > > - The Linux extension is a genuinely useful feature > > > done in a rather unpleasant way > > > - If you are using AF_UNIX in the portable way, you should > > > provide the *length* of the address, not the *size* of the > > > struct. I remember it being that way in 4.2 BSD and have > > > cited material from some of the key BSD people that that was > > > certainly the intent in 4.3BSD. > > > - But if you care about HP-UX, you had better do some experiments. > > > > > > Honestly, just doing the right thing is less effort than arguing > > > about it. > > > > Precisely. Well put! But what is the "right thing"? > > > > The current implementation passes the address given to it as is, but > > ensures it is correctly terminated as in appending a '\0' at the end > > that is not counted for address_len, and if that does not fit > > in the defined size it bails out. > > > > That felt like the "right thing" when I wrote it, but you can not set the > > last byte of the address to non-zero. > > > > The new "right thing" could be to add an exception to the above when there > > is a '\0' first in the address, since that hopefully will count as a > > correct termination on OS:es that do not understand abstract addresses. > > > > The question is if there is any danger in this because we then give a > > contradiction to e.g bind() because we have not used SUN_LEN(su) as we were > > told to do...? > > > > I believe not, and will try to find the time to do this > > after our OTP-20.rc1 pre-release. > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From info@REDACTED Fri Apr 28 16:48:04 2017 From: info@REDACTED (RDlab) Date: Fri, 28 Apr 2017 16:48:04 +0200 Subject: [erlang-questions] DDDS/DNS oriented project considerations In-Reply-To: References: Message-ID: Gentlemen, We would like to check if our understanding is correct. We consider deploying and documenting an user oriented set of distributed local IoT/internet OS independent services/applications. We wish to test the concept through a DDDs proposition (the DNS is a DDDS) oriented towards local DNS and Named Data Networking as a fork of BIND. We understand that an Erlang/Elixir Beam based solution could be a alternative and that OTP could provide a full communictions library for such a project. However, we have no knowledge in your area, and therefore before heavily investing into learnng it, we would be rather interested in knowing your opinions: - would such a project make sense from your experience - would there be some existing projects/solution that could be of interest to us. We are a young non-profit Lab in Montpellier, South of France. Thank you for you help. JFC Morfin From comptekki@REDACTED Sat Apr 29 21:09:51 2017 From: comptekki@REDACTED (Wes James) Date: Sat, 29 Apr 2017 13:09:51 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: I found a fix that works for me. Note this issue is only on windows 10. Set this registry entry: Create AutoStartDelay as DWORD 32 bit and set to decimail 1 in this key: HKLM\SYSTEM\CurrentControlSet\Control\ -wes On Thu, Apr 27, 2017 at 11:01 AM, Wes James wrote: > > I tried this today with 19.3. I have to set the service to Automatic (Delaye Start) for it to work, like in 18.3. I looked in the eventviewer and I see these errors: > > Error Service Control Manager > Under General Tab > The service terminated with the following error: > The configuration registry key could not be read. > > A little further down > > Error ErlSrv > Under General Tab > servicename: Could not get registry keys for erlang service. > > Which is odd since it does work when delayed. It's like it's waiting for some other service to start but hasn't yet so it quits. I'm not sure what it's depending on before starting successfully in delayed start. > > -wes > > On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson wrote: >> >> Well you got it run at all, that is something, I have disabled our testing of >> services on Windows 10 for now, master branch, since I could not get it run at all. >> >> From what I googled and understood you will need start the services from an elevated shell, an administer user is not enough. >> But you might already do that or I got it wrong. >> >> Anyway, our Windows knowledge, and priority, is not at the highest level. >> So any help would be appreciated. >> >> /Dan >> >> On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: >>> >>> I use erlsrv to install an erlang application on windows. Windows 7 works fine, but on windows 10, I have to go in and change the service from automatic to Automatic (delayed start), otherwise the app never runs on system start up. Any idea how to fix this? >>> >>> I just found this windows command: >>> >>> sc config SVCNAME start= delayed-auto >>> >>> I can use that in the install script. >>> >>> I'd still like it to start with just automatic. With the delayed option, it takes several extra minutes for the service to start running. >>> >>> Thanks, >>> >>> -wes >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuanbeo0502@REDACTED Sun Apr 30 00:25:54 2017 From: tuanbeo0502@REDACTED (Tuan Cao) Date: Sat, 29 Apr 2017 15:25:54 -0700 Subject: [erlang-questions] Erlang Lexer/Parser for Oracle Message-ID: Hi, I'm in the process of writing a lexer/parser for Oracle. It is possible that there are some projects out there which already did that. For example, I found sqlparse which is a LALR grammar based SQL Parser (not specific to Oracle but cover the basic SQL syntax). To avoid reinventing the wheel, if you have any pointers on a open source Oracle language parser in Erlang, kindly let me know. Thanks, Tuan -------------- next part -------------- An HTML attachment was scrubbed... URL: