From stu.bailey@REDACTED Fri Feb 1 00:01:20 2019 From: stu.bailey@REDACTED (Stu Bailey) Date: Thu, 31 Jan 2019 15:01:20 -0800 Subject: [erlang-questions] Docker/Rebar3/Hex Basic Issue: "No project builder is configured for type mix" Message-ID: Hi all, I'm trying to efficiently get up and running with Erlang, and I've immediately hit a roadblock and did not see any clear indication on causes or suggestions from a quick Google search so I thought I'd post here. I can make a new app with the Erlang Docker container no problem: bash-3.2$ docker run -it -v /Users/me/scratch:/usr/local/src erlang bash root@REDACTED:/# cd /usr/local/src root@REDACTED:/usr/local/src# root@REDACTED:/usr/local/src# rebar3 new app myapp rebar3 new app myapp ===> Writing myapp/src/myapp_app.erl ===> Writing myapp/src/myapp_sup.erl ===> Writing myapp/src/myapp.app.src ===> Writing myapp/rebar.config ===> Writing myapp/.gitignore ===> Writing myapp/LICENSE ===> Writing myapp/README.md root@REDACTED:/usr/local/src# rebar3 shell rebar3 shell ===> Verifying dependencies... Erlang/OTP 21 [erts-10.2.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] Eshell V10.2.3 (abort with ^G) 1> application:start(myapp). application:start(myapp). 2> C-c C-c Then I follow this for using packages https://medium.com/erlang-central/building-your-first-erlang-app-using-rebar3-25f40b109aad root@REDACTED:/usr/local/src/myapp# cat rebar.config cat rebar.config {erl_opts, [debug_info]}. {deps, []}. {shell, [ % {config, "config/sys.config"}, {apps, [myapp]} ]}. {plugins, [rebar3_hex]}. root@REDACTED:/usr/local/src/myapp# root@REDACTED:/usr/local/src/myapp# rebar3 update rebar3 update ===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"6.4.0">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/rebar3_hex-6.4.0.tar ===> Compiling rebar3_hex So far so good then I search for a specific package "restclient" root@REDACTED:/usr/local/src/myapp# rebar3 hex search restclient rebar3 hex search restclient Results: rest_client: RestClient is a generic REST client library. It generates structs and functions for use with APIs. Now I add rest_client as a dep and attempt rebar3 compile and this where the problem occurs root@REDACTED:/usr/local/src/myapp# cat rebar.config cat rebar.config {erl_opts, [debug_info]}. {deps, [rest_client]}. {shell, [ % {config, "config/sys.config"}, {apps, [myapp]} ]}. {plugins, [rebar3_hex]}. root@REDACTED:/usr/local/src/myapp# rebar3 compile rebar3 compile ===> Verifying dependencies... ===> Fetching rest_client ({pkg,<<"rest_client">>,<<"0.0.1">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/rest_client-0.0.1.tar ===> Fetching httpotion ({pkg,<<"httpotion">>,<<"2.1.0">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/httpotion-2.1.0.tar ===> Fetching inflex ({pkg,<<"inflex">>,<<"1.4.1">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/inflex-1.4.1.tar ===> Fetching mock ({pkg,<<"mock">>,<<"0.1.3">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/mock-0.1.3.tar ===> Fetching poison ({pkg,<<"poison">>,<<"1.5.2">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/poison-1.5.2.tar ===> Fetching meck ({pkg,<<"meck">>,<<"0.8.13">>}) ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/meck-0.8.13.tar ===> Compiling httpotion ===> Error building application httpotion: No project builder is configured for type mix Thoughts? Any help would be greatly appreciated. Thanks! Stu -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Fri Feb 1 00:04:08 2019 From: t@REDACTED (Tristan Sloughter) Date: Thu, 31 Jan 2019 18:04:08 -0500 Subject: [erlang-questions] =?utf-8?q?Docker/Rebar3/Hex_Basic_Issue=3A_=22?= =?utf-8?q?No_project_builder_is_configured_for_type_mix=22?= In-Reply-To: References: Message-ID: `rest_client` is an Elixir/mix project. To use it you'll need Elixir as well. Tristan On Thu, Jan 31, 2019, at 16:01, Stu Bailey wrote: > Hi all, > > I'm trying to efficiently get up and running with Erlang, and I've immediately hit a roadblock and did not see any clear indication on causes or suggestions from a quick Google search so I thought I'd post here. > > I can make a new app with the Erlang Docker container no problem: > > bash-3.2$ docker run -it -v /Users/me/scratch:/usr/local/src erlang bash > root@REDACTED:/# > cd /usr/local/src > root@REDACTED:/usr/local/src# > root@REDACTED:/usr/local/src# rebar3 new app myapp > rebar3 new app myapp > ===> Writing myapp/src/myapp_app.erl > ===> Writing myapp/src/myapp_sup.erl > ===> Writing myapp/src/myapp.app.src > ===> Writing myapp/rebar.config > ===> Writing myapp/.gitignore > ===> Writing myapp/LICENSE > ===> Writing myapp/README.md > root@REDACTED:/usr/local/src# rebar3 shell > rebar3 shell > ===> Verifying dependencies... > Erlang/OTP 21 [erts-10.2.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] > > Eshell V10.2.3 (abort with ^G) > 1> application:start(myapp). > application:start(myapp). > 2> C-c C-c > > > Then I follow this for using packages https://medium.com/erlang-central/building-your-first-erlang-app-using-rebar3-25f40b109aad > > root@REDACTED:/usr/local/src/myapp# cat rebar.config > cat rebar.config > {erl_opts, [debug_info]}. > {deps, []}. > > {shell, [ > % {config, "config/sys.config"}, > {apps, [myapp]} > ]}. > {plugins, [rebar3_hex]}. > root@REDACTED:/usr/local/src/myapp# > root@REDACTED:/usr/local/src/myapp# rebar3 update > rebar3 update > ===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"6.4.0">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/rebar3_hex-6.4.0.tar > ===> Compiling rebar3_hex > > So far so good then I search for a specific package "restclient" > > root@REDACTED:/usr/local/src/myapp# rebar3 hex search restclient > rebar3 hex search restclient > Results: > > rest_client: RestClient is a generic REST client library. It generates structs and functions > for use with APIs. > > Now I add rest_client as a dep and attempt rebar3 compile and this where the problem occurs > > root@REDACTED:/usr/local/src/myapp# cat rebar.config > cat rebar.config > {erl_opts, [debug_info]}. > {deps, [rest_client]}. > > {shell, [ > % {config, "config/sys.config"}, > {apps, [myapp]} > ]}. > {plugins, [rebar3_hex]}. > root@REDACTED:/usr/local/src/myapp# rebar3 compile > rebar3 compile > ===> Verifying dependencies... > ===> Fetching rest_client ({pkg,<<"rest_client">>,<<"0.0.1">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/rest_client-0.0.1.tar > ===> Fetching httpotion ({pkg,<<"httpotion">>,<<"2.1.0">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/httpotion-2.1.0.tar > ===> Fetching inflex ({pkg,<<"inflex">>,<<"1.4.1">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/inflex-1.4.1.tar > ===> Fetching mock ({pkg,<<"mock">>,<<"0.1.3">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/mock-0.1.3.tar > ===> Fetching poison ({pkg,<<"poison">>,<<"1.5.2">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/poison-1.5.2.tar > ===> Fetching meck ({pkg,<<"meck">>,<<"0.8.13">>}) > ===> Downloaded package, caching at /root/.cache/rebar3/hex/hexpm/packages/meck-0.8.13.tar > ===> Compiling httpotion > ===> Error building application httpotion: > No project builder is configured for type mix > > Thoughts? Any help would be greatly appreciated. > > Thanks! > Stu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Fri Feb 1 00:16:14 2019 From: stu.bailey@REDACTED (Stu Bailey) Date: Thu, 31 Jan 2019 15:16:14 -0800 Subject: [erlang-questions] Docker/Rebar3/Hex Basic Issue: "No project builder is configured for type mix" In-Reply-To: References: Message-ID: Thanks for the quick response. On Thu, Jan 31, 2019 at 3:04 PM Tristan Sloughter wrote: > `rest_client` is an Elixir/mix project. To use it you'll need Elixir as > well. > > Tristan > > On Thu, Jan 31, 2019, at 16:01, Stu Bailey wrote: > > Hi all, > > I'm trying to efficiently get up and running with Erlang, and I've > immediately hit a roadblock and did not see any clear indication on causes > or suggestions from a quick Google search so I thought I'd post here. > > I can make a new app with the Erlang Docker container no problem: > > bash-3.2$ docker run -it -v /Users/me/scratch:/usr/local/src erlang bash > root@REDACTED:/# > cd /usr/local/src > root@REDACTED:/usr/local/src# > root@REDACTED:/usr/local/src# rebar3 new app myapp > rebar3 new app myapp > ===> Writing myapp/src/myapp_app.erl > ===> Writing myapp/src/myapp_sup.erl > ===> Writing myapp/src/myapp.app.src > ===> Writing myapp/rebar.config > ===> Writing myapp/.gitignore > ===> Writing myapp/LICENSE > ===> Writing myapp/README.md > root@REDACTED:/usr/local/src# rebar3 shell > rebar3 shell > ===> Verifying dependencies... > Erlang/OTP 21 [erts-10.2.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] > [async-threads:1] [hipe] > > Eshell V10.2.3 (abort with ^G) > 1> application:start(myapp). > application:start(myapp). > 2> C-c C-c > > > Then I follow this for using packages > https://medium.com/erlang-central/building-your-first-erlang-app-using-rebar3-25f40b109aad > > root@REDACTED:/usr/local/src/myapp# cat rebar.config > cat rebar.config > {erl_opts, [debug_info]}. > {deps, []}. > > {shell, [ > % {config, "config/sys.config"}, > {apps, [myapp]} > ]}. > {plugins, [rebar3_hex]}. > root@REDACTED:/usr/local/src/myapp# > root@REDACTED:/usr/local/src/myapp# rebar3 update > rebar3 update > ===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"6.4.0">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/rebar3_hex-6.4.0.tar > ===> Compiling rebar3_hex > > So far so good then I search for a specific package "restclient" > > root@REDACTED:/usr/local/src/myapp# rebar3 hex search restclient > rebar3 hex search restclient > Results: > > rest_client: RestClient is a generic REST client library. It generates > structs and functions > for use with APIs. > > Now I add rest_client as a dep and attempt rebar3 compile and this where > the problem occurs > > root@REDACTED:/usr/local/src/myapp# cat rebar.config > cat rebar.config > {erl_opts, [debug_info]}. > {deps, [rest_client]}. > > {shell, [ > % {config, "config/sys.config"}, > {apps, [myapp]} > ]}. > {plugins, [rebar3_hex]}. > root@REDACTED:/usr/local/src/myapp# rebar3 compile > rebar3 compile > ===> Verifying dependencies... > ===> Fetching rest_client ({pkg,<<"rest_client">>,<<"0.0.1">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/rest_client-0.0.1.tar > ===> Fetching httpotion ({pkg,<<"httpotion">>,<<"2.1.0">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/httpotion-2.1.0.tar > ===> Fetching inflex ({pkg,<<"inflex">>,<<"1.4.1">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/inflex-1.4.1.tar > ===> Fetching mock ({pkg,<<"mock">>,<<"0.1.3">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/mock-0.1.3.tar > ===> Fetching poison ({pkg,<<"poison">>,<<"1.5.2">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/poison-1.5.2.tar > ===> Fetching meck ({pkg,<<"meck">>,<<"0.8.13">>}) > ===> Downloaded package, caching at > /root/.cache/rebar3/hex/hexpm/packages/meck-0.8.13.tar > ===> Compiling httpotion > ===> Error building application httpotion: > No project builder is configured for type mix > > Thoughts? Any help would be greatly appreciated. > > Thanks! > Stu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Feb 1 05:16:00 2019 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 1 Feb 2019 05:16:00 +0100 Subject: [erlang-questions] [ann] erlang-rocksdb 1.0.0 has been released In-Reply-To: <20190131221636.GG1915@ferdmbp.local> References: <20190131221636.GG1915@ferdmbp.local> Message-ID: That's good to hear :) Thank you! Beno?t On Thu, Jan 31, 2019 at 11:16 PM Fred Hebert wrote: > Congrats on finally reaching 1.0.0! > > I have not yet had a project where rocksdb would make sense but the > project always looked super interesting. > > Keep up the good work :) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Fri Feb 1 06:12:53 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 1 Feb 2019 06:12:53 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: <20190131225247.GH1915@ferdmbp.local> References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: The original proxy design was like this: External TCP Source -> Proxy Process (binary transformation and analysis) -> External TCP Sink The current design added a new stage to the proxy by having 2 processes instead of 1: External TCP Source -> Proxy Process1 -> Proxy Process2 -> External TCP Sink Process1 forward the binaries (>64B) to Process2 which also inspects them (by logging suspicious packets). I will perform more checks today and report back. On 01/31, Scott Ribe wrote: > >> On Jan 31, 2019, at 3:26 PM, Frank Muller > wrote: > >> > >> The refc binary is the one using most of VM?s memory (in GB) compared > to other allocators (few MB). > > > >So, could be the classic leak where you match on a binary and pass a > sub-binary to another process? That creates a ref to the original binary, > and if the first process is a long-lived one that doesn't gc often, even > though the second one finishes, the original binary hangs around... > > Yeah, that's a good bet. I'd check in the new feature where a sub-binary > of a large binary is taken, and would call `NewBin = binary:copy(Bin)` > on it to free the ref as early as possible. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Fri Feb 1 11:27:58 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 1 Feb 2019 11:27:58 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: Thanks Gerhard. I will definitely give it a go. I was able to identify the process which is leaking memory, thanks to recon. It was the process added to handle the new feature. The binary_alloc seems to be the one misbehaving (not 100% sure, see attached screenshot). Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson maybe? /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IMG_6340.jpg Type: image/jpg Size: 95928 bytes Desc: not available URL: From gerhard@REDACTED Fri Feb 1 11:44:13 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Fri, 1 Feb 2019 10:44:13 +0000 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: Hi Frank, The binary_alloc seems to be the one misbehaving (not 100% sure, see > attached screenshot). > I agree, it's your binary_alloc. How many single block carriers in binary_alloc do you have? What about multi-block carriers? Can you share a recon_alloc snapshot? > Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson > maybe? > For us, RabbitMQ, defaulting to a lower multi-block carrier size (+MBlmbcs ) as well as changing the allocation strategy (+MBas ) made a positive difference. From the screenshot that you've shared, I am guessing that this won't work for you. This is the process that we went through at RabbitMQ: https://groups.google.com/forum/#!msg/rabbitmq-users/LSYaac9frYw/LNZDZUlrBAAJ I found Lukas' Erlang Memory Management Battle Stories from 2014 to be a great erts_alloc companion: http://www.erlang-factory.com/static/upload/media/139454517145429lukaslarsson.pdf & https://www.youtube.com/watch?v=nuCYL0X-8f4 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IMG_6339.jpg Type: image/jpeg Size: 137037 bytes Desc: not available URL: From gerhard@REDACTED Fri Feb 1 12:49:31 2019 From: gerhard@REDACTED (Gerhard Lazu) Date: Fri, 1 Feb 2019 11:49:31 +0000 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: A text file would work a lot better, the important information is missing from your last screenshot. This should fit binary_alloc stats for the first scheduler in a screenshot (your system has 48 schedulers): [_|[SCHED0|_]] = erlang:system_info({allocator, binary_alloc}), io:format("~p~n", [SCHED0]). On Fri, Feb 1, 2019 at 11:14 AM Frank Muller wrote: > Here is the beginning of recon_alloc:snapshot/1. > Yes, Lukas can help on this. VM?s memory management looks like a mystery. > > > > Hi Frank, >> >> The binary_alloc seems to be the one misbehaving (not 100% sure, see >>> attached screenshot). >>> >> >> I agree, it's your binary_alloc. How many single block carriers in >> binary_alloc do you have? What about multi-block carriers? Can you share a >> recon_alloc snapshot? >> >> >>> Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson >>> maybe? >>> >> >> For us, RabbitMQ, defaulting to a lower multi-block carrier size ( >> +MBlmbcs ) as well as >> changing the allocation strategy (+MBas >> ) made a positive >> difference. From the screenshot that you've shared, I am guessing that this >> won't work for you. This is the process that we went through at RabbitMQ: >> https://groups.google.com/forum/#!msg/rabbitmq-users/LSYaac9frYw/LNZDZUlrBAAJ >> >> I found Lukas' Erlang Memory Management Battle Stories from 2014 to be a >> great erts_alloc companion: >> http://www.erlang-factory.com/static/upload/media/139454517145429lukaslarsson.pdf >> & https://www.youtube.com/watch?v=nuCYL0X-8f4 >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IMG_6342.jpg Type: image/jpeg Size: 199468 bytes Desc: not available URL: From frank.muller.erl@REDACTED Fri Feb 1 12:14:01 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 1 Feb 2019 12:14:01 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: Here is the beginning of recon_alloc:snapshot/1. Yes, Lukas can help on this. VM?s memory management looks like a mystery. Hi Frank, > > The binary_alloc seems to be the one misbehaving (not 100% sure, see >> attached screenshot). >> > > I agree, it's your binary_alloc. How many single block carriers in > binary_alloc do you have? What about multi-block carriers? Can you share a > recon_alloc snapshot? > > >> Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson >> maybe? >> > > For us, RabbitMQ, defaulting to a lower multi-block carrier size (+MBlmbcs > ) as well as changing > the allocation strategy (+MBas > ) made a positive > difference. From the screenshot that you've shared, I am guessing that this > won't work for you. This is the process that we went through at RabbitMQ: > https://groups.google.com/forum/#!msg/rabbitmq-users/LSYaac9frYw/LNZDZUlrBAAJ > > I found Lukas' Erlang Memory Management Battle Stories from 2014 to be a > great erts_alloc companion: > http://www.erlang-factory.com/static/upload/media/139454517145429lukaslarsson.pdf > & https://www.youtube.com/watch?v=nuCYL0X-8f4 > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IMG_6342.jpg Type: image/jpg Size: 199468 bytes Desc: not available URL: From frank.muller.erl@REDACTED Fri Feb 1 17:11:11 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 1 Feb 2019 17:11:11 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: I tried two solutions to reduce the memory usage of the problematic process: 1. calling garbage:collect/0 after processing N packets (varying N=10..128). Nothing changed at all and the bin_alloc memory stayed fragmented as you can see: http://147.135.36.188:3400/observer_cli_BEFORE.jpg The call to instrument:carriers/0: http://147.135.36.188:3400/instrument_carriers.jpg The call to instrument:allocations/0: http://147.135.36.188:3400/instrument_allocations.jpg 2. Hibernating the process after processing N packets (varying N=10..128). The HIT rates went above 90% immediately. http://147.135.36.188:3400/observer_cli_AFTER.jpg What is the effect of hibernating this process on the long term? This process is receiving about ~1200 packets/sec under normal load and can reach ~3000 packets/sec under heavy load. Is there a better way of solving the problem by tweeting the bin allocator SBC/MBC? /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Fri Feb 1 17:07:30 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 1 Feb 2019 17:07:30 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: I tried two solutions to reduce the memory usage of the problematic process: 1. calling garbage:collect/0 after processing N packets (varying N=10..128). Nothing changed at all and the bin_alloc memory stayed fragmented as you can see: http://147.135.36.188:3400/observer_cli_BEFORE.jpg The call to instrument:carriers/0: http://147.135.36.188:3400/instrument_carriers.jpg The call to instrument:allocations/0: http://147.135.36.188:3400/instrument_allocations.jpg 2. Hibernating the process after processing N packets (varying N=10..128). The HIT rate went above 90% immediately. http://147.135.36.188:3400/observer_cli_AFTER.jpg What is the effect of frequent hibernating on the long term? This process is receiving about ~1200 packets/sec under normal load and can reach ~3000 packets/sec under heavy load. Is there a better way of solving this memory issue by tweeting the bin allocator SBC/MBC? /Frank A text file would work a lot better, the important information is missing > from your last screenshot. > > This should fit binary_alloc stats for the first scheduler in a screenshot > (your system has 48 schedulers): > > [_|[SCHED0|_]] = erlang:system_info({allocator, binary_alloc}), > io:format("~p~n", [SCHED0]). > > > > On Fri, Feb 1, 2019 at 11:14 AM Frank Muller > wrote: > >> Here is the beginning of recon_alloc:snapshot/1. >> Yes, Lukas can help on this. VM?s memory management looks like a mystery. >> >> >> >> Hi Frank, >>> >>> The binary_alloc seems to be the one misbehaving (not 100% sure, see >>>> attached screenshot). >>>> >>> >>> I agree, it's your binary_alloc. How many single block carriers in >>> binary_alloc do you have? What about multi-block carriers? Can you share a >>> recon_alloc snapshot? >>> >>> >>>> Is there a way to tweak this allocator strategy SBC/MBC? Lukas Larsson >>>> maybe? >>>> >>> >>> For us, RabbitMQ, defaulting to a lower multi-block carrier size ( >>> +MBlmbcs ) as well >>> as changing the allocation strategy (+MBas >>> ) made a positive >>> difference. From the screenshot that you've shared, I am guessing that this >>> won't work for you. This is the process that we went through at RabbitMQ: >>> https://groups.google.com/forum/#!msg/rabbitmq-users/LSYaac9frYw/LNZDZUlrBAAJ >>> >>> I found Lukas' Erlang Memory Management Battle Stories from 2014 to be a >>> great erts_alloc companion: >>> http://www.erlang-factory.com/static/upload/media/139454517145429lukaslarsson.pdf >>> & https://www.youtube.com/watch?v=nuCYL0X-8f4 >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IMG_6342.jpg Type: image/jpeg Size: 199468 bytes Desc: not available URL: From sunboshan@REDACTED Fri Feb 1 17:54:38 2019 From: sunboshan@REDACTED (Boshan Sun) Date: Fri, 1 Feb 2019 08:54:38 -0800 Subject: [erlang-questions] [ann] erlang-rocksdb 1.0.0 has been released In-Reply-To: References: <20190131221636.GG1915@ferdmbp.local> Message-ID: Congrats! 1.0.0 have much smaller size in my deps folder comparing to 0.26.0. What optimization did you do? Boshan Benoit Chesneau ?2019?1?31??? ??8:16??? > That's good to hear :) Thank you! > > Beno?t > > On Thu, Jan 31, 2019 at 11:16 PM Fred Hebert wrote: > >> Congrats on finally reaching 1.0.0! >> >> I have not yet had a project where rocksdb would make sense but the >> project always looked super interesting. >> >> Keep up the good work :) >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Feb 1 22:54:34 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 1 Feb 2019 16:54:34 -0500 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: On Fri, Feb 1, 2019 at 11:11 AM Frank Muller wrote: > > 2. Hibernating the process after processing N packets (varying N=10..128). > The HIT rates went above 90% immediately. > http://147.135.36.188:3400/observer_cli_AFTER.jpg > > What is the effect of hibernating this process on the long term? > This process is receiving about ~1200 packets/sec under normal load and > can reach ~3000 packets/sec under heavy load. > > Is there a better way of solving the problem by tweeting the bin allocator > SBC/MBC? > So hibernation will do a few things: - a full-sweep garbage collection - drop the stack - memory compaction on the process. Unless specified otherwise, a call to `erlang:garbage_collect(Pid)` forces a major collection so it may look like what you have is more or less a case of a process spiking with a lot of memory, and then becoming mostly idle while still holding enough references to refc binaries to not GC old data. Rince and repeat and you get a lot of old stuff. Fragmentation of this kind is often resolved with settings such as `+MBas aobf +MBlmbcs 512` being passed to the emulator, which changes the allocation strategy for one that favors lower addresses, and reduces the size of a multiblock carrier to use more of them. The objective of this being to reuse existing blocks and make it easier to deallocate some by reducing the chance some binary keeps it active. If what you have is really one process though, you may get better results by running some hibernation from time to time, but only experimentation will confirm. If the allocator strategies don't cut it (or can't be used because you want to keep the 5 years live upgrade streak going), do something like count the packets you received, and every N of them (pick a large value so you might run a compaction once every 10 minutes or so; you can pick based on the leaking rate), force a hibernation to shed some memory. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Feb 2 00:10:46 2019 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 1 Feb 2019 15:10:46 -0800 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> On 2/1/19 8:11 AM, Frank Muller wrote: > I tried two solutions to reduce the memory usage of the problematic > process: > > 1. calling garbage:collect/0 after processing N packets (varying > N=10..128). > Nothing changed at all and the bin_alloc memory stayed fragmented as > you can see: > http://147.135.36.188:3400/observer_cli_BEFORE.jpg > > The call to instrument:carriers/0: > http://147.135.36.188:3400/instrument_carriers.jpg > > The call to instrument:allocations/0: > http://147.135.36.188:3400/instrument_allocations.jpg > > > 2. Hibernating the process after processing N packets (varying N=10..128). > The HIT rates went above 90% immediately. > http://147.135.36.188:3400/observer_cli_AFTER.jpg > > What is the effect of hibernating this process on the long term? > This process is receiving about ~1200 packets/sec under normal load > and can reach ~3000 packets/sec under heavy load. > > Is there a better way of solving the problem by tweeting the bin > allocator SBC/MBC? > > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions If you move the creation of temporary binaries out of any Erlang processes you have that are long-lived, into short-lived Erlang processes, you would no longer have this problem.? The tuning discussions, allocator options, hibernate use, etc. is not solving the cause of the problem.? Source code should not need to call garbage:collect/0 and using temporary Erlang processes makes the garbage collection occur naturally, at a pace that shouldn't require special tuning. Best Regards, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat Feb 2 02:11:15 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 1 Feb 2019 20:11:15 -0500 Subject: [erlang-questions] VM leaking memory In-Reply-To: <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> References: <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> Message-ID: <20190202011114.GI1915@ferdmbp.local> On 02/01, Michael Truog wrote: > >If you move the creation of temporary binaries out of any Erlang >processes you have that are long-lived, into short-lived Erlang >processes, you would no longer have this problem.? The tuning >discussions, allocator options, hibernate use, etc. is not solving the >cause of the problem.? Source code should not need to call >garbage:collect/0 and using temporary Erlang processes makes the >garbage collection occur naturally, at a pace that shouldn't require >special tuning. > >Best Regards, >Michael While agree on principle, both calling the garbage collector, tuning the allocators, or moving the creation of a sub-binary to a temporary process in the current contexts are workarounds over the current limitation where the most natural pattern results in a memory leak. You're suggesting another pattern that conflates code compared to its ideal format, which just leaks right now. The problem is not the workaround, it's needing it in the first place. From frank.muller.erl@REDACTED Sat Feb 2 07:53:38 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 2 Feb 2019 07:53:38 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> Message-ID: Hi Michael All packets in transit have a ?seq_id? (sequential number). This means that in theory packet1, packet2...packetN can be checked in parallel and in any order (which is not the case in my current design), but they must be send to the next processing stage in order: packet1 first, then packet2... I would love to hear from you how can I turn this long-lived process to multiple short-lived ones while enforcing ordering. /Frank On 2/1/19 8:11 AM, Frank Muller wrote: > > I tried two solutions to reduce the memory usage of the problematic > process: > > 1. calling garbage:collect/0 after processing N packets (varying > N=10..128). > Nothing changed at all and the bin_alloc memory stayed fragmented as you > can see: > http://147.135.36.188:3400/observer_cli_BEFORE.jpg > > The call to instrument:carriers/0: > http://147.135.36.188:3400/instrument_carriers.jpg > > The call to instrument:allocations/0: > http://147.135.36.188:3400/instrument_allocations.jpg > > > 2. Hibernating the process after processing N packets (varying N=10..128). > The HIT rates went above 90% immediately. > http://147.135.36.188:3400/observer_cli_AFTER.jpg > > What is the effect of hibernating this process on the long term? > This process is receiving about ~1200 packets/sec under normal load and > can reach ~3000 packets/sec under heavy load. > > Is there a better way of solving the problem by tweeting the bin allocator > SBC/MBC? > > > /Frank > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > If you move the creation of temporary binaries out of any Erlang processes > you have that are long-lived, into short-lived Erlang processes, you would > no longer have this problem. The tuning discussions, allocator options, > hibernate use, etc. is not solving the cause of the problem. Source code > should not need to call garbage:collect/0 and using temporary Erlang > processes makes the garbage collection occur naturally, at a pace that > shouldn't require special tuning. > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Feb 2 07:56:32 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 2 Feb 2019 07:56:32 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> Message-ID: Hi Fred I will implement your hibernation idea every N-minutes on monday and report back. Thanks again. /Frank > > On Fri, Feb 1, 2019 at 11:11 AM Frank Muller > wrote: > >> >> 2. Hibernating the process after processing N packets (varying N=10..128). >> The HIT rates went above 90% immediately. >> http://147.135.36.188:3400/observer_cli_AFTER.jpg >> >> What is the effect of hibernating this process on the long term? >> This process is receiving about ~1200 packets/sec under normal load and >> can reach ~3000 packets/sec under heavy load. >> >> Is there a better way of solving the problem by tweeting the bin >> allocator SBC/MBC? >> > > So hibernation will do a few things: > > - a full-sweep garbage collection > - drop the stack > - memory compaction on the process. > > Unless specified otherwise, a call to `erlang:garbage_collect(Pid)` forces > a major collection so it may look like what you have is more or less a case > of a process spiking with a lot of memory, and then becoming mostly idle > while still holding enough references to refc binaries to not GC old data. > Rince and repeat and you get a lot of old stuff. > > Fragmentation of this kind is often resolved with settings such as `+MBas > aobf +MBlmbcs 512` being passed to the emulator, which changes the > allocation strategy for one that favors lower addresses, and reduces the > size of a multiblock carrier to use more of them. The objective of this > being to reuse existing blocks and make it easier to deallocate some by > reducing the chance some binary keeps it active. > > If what you have is really one process though, you may get better results > by running some hibernation from time to time, but only experimentation > will confirm. If the allocator strategies don't cut it (or can't be used > because you want to keep the 5 years live upgrade streak going), do > something like count the packets you received, and every N of them (pick a > large value so you might run a compaction once every 10 minutes or so; you > can pick based on the leaking rate), force a hibernation to shed some > memory. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat Feb 2 08:59:54 2019 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 1 Feb 2019 23:59:54 -0800 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> Message-ID: <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> On 2/1/19 10:53 PM, Frank Muller wrote: > Hi Michael > > All packets in transit have a ?seq_id? (sequential number). > > This means that in theory packet1, packet2...packetN can be checked in > parallel and in any order (which is not the case in my current > design), but they must be send to the next processing stage in order: > packet1 first, then packet2... > > I would love to hear from you how can I turn this long-lived process > to multiple short-lived ones while enforcing ordering. An easy way to think about it comes from a module I used in the past called immediate_gc (https://gist.github.com/okeuday/dee991d580eeb00cd02c).? The sync_fun/2 function is below: sync_fun(F, A) when is_function(F), is_list(A) -> ??? Parent = self(), ??? Child = erlang:spawn_opt(fun() -> ??????? Parent ! {self(), erlang:apply(F, A)}, ??????? erlang:garbage_collect() ??? end, [link, {fullsweep_after, 0}]), ??? receive ??????? {Child, Result} -> Result ??? end. That is all you need to use a temporary process in a blocking way that consumes all the temporary binary data as quickly as the BEAM allows.? However, that example is more complex than it needs to be, with the child process using the fullsweep_after option and erlang:garbage_collect/0.? The extra complexity in the example is really not necessary or desirable, though it does force the garbage collection to occur as quickly as possible when consuming the temporary binary data (binary data that nothing else references). Spawn a similar child process before you start decoding a large binary, so the temporary Erlang process has a lifetime the length of the single request (packet in your situation) or less. Spawning Erlang processes is cheap, so you shouldn't hesitate to use them, just ensure they are linked so their failures may be tracked. CloudI (https://cloudi.org) internal services use temporary processes for handling service requests, in a way that is tunable with the service configuration options request_pid_uses and info_pid_uses , so you can control how many requests are processed in a temporary Erlang process before a new one is created (with the exit exception being used to terminate the Erlang process with its last result).? CloudI internal services also have the hibernate service configuration option, with the hibernate based on request rate checked every few seconds (the service configuration options are described at http://cloudi.org/api.html#2_services_add_config_opts). The idea of using a temporary Erlang process for consuming temporary binary data, is likely unusual for people new to Erlang/Elixir and may not have found its way into Erlang/Elixir books, though it is important to know about if you want to avoid excessive memory consumption (and potentially causing the BEAM to die due to memory use). Best Regards, Michael From frank.muller.erl@REDACTED Sat Feb 2 10:27:31 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 2 Feb 2019 10:27:31 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> Message-ID: Hello Michael Thanks again for the tip. I will update my design accordingly and let you know how it goes. /Frank On 2/1/19 10:53 PM, Frank Muller wrote: > > Hi Michael > > > > All packets in transit have a ?seq_id? (sequential number). > > > > This means that in theory packet1, packet2...packetN can be checked in > > parallel and in any order (which is not the case in my current > > design), but they must be send to the next processing stage in order: > > packet1 first, then packet2... > > > > I would love to hear from you how can I turn this long-lived process > > to multiple short-lived ones while enforcing ordering. > > An easy way to think about it comes from a module I used in the past > called immediate_gc > (https://gist.github.com/okeuday/dee991d580eeb00cd02c). The sync_fun/2 > function is below: > > > sync_fun(F, A) when is_function(F), is_list(A) -> > Parent = self(), > Child = erlang:spawn_opt(fun() -> > Parent ! {self(), erlang:apply(F, A)}, > erlang:garbage_collect() > end, [link, {fullsweep_after, 0}]), > receive > {Child, Result} -> Result > end. > > That is all you need to use a temporary process in a blocking way that > consumes all the temporary binary data as quickly as the BEAM allows. > However, that example is more complex than it needs to be, with the > child process using the fullsweep_after option and > erlang:garbage_collect/0. The extra complexity in the example is really > not necessary or desirable, though it does force the garbage collection > to occur as quickly as possible when consuming the temporary binary data > (binary data that nothing else references). > > Spawn a similar child process before you start decoding a large binary, > so the temporary Erlang process has a lifetime the length of the single > request (packet in your situation) or less. Spawning Erlang processes is > cheap, so you shouldn't hesitate to use them, just ensure they are > linked so their failures may be tracked. > > CloudI (https://cloudi.org) internal services use temporary processes > for handling service requests, in a way that is tunable with the service > configuration options request_pid_uses and info_pid_uses , so you can > control how many requests are processed in a temporary Erlang process > before a new one is created (with the exit exception being used to > terminate the Erlang process with its last result). CloudI internal > services also have the hibernate service configuration option, with the > hibernate based on request rate checked every few seconds (the service > configuration options are described at > http://cloudi.org/api.html#2_services_add_config_opts). > > The idea of using a temporary Erlang process for consuming temporary > binary data, is likely unusual for people new to Erlang/Elixir and may > not have found its way into Erlang/Elixir books, though it is important > to know about if you want to avoid excessive memory consumption (and > potentially causing the BEAM to die due to memory use). > > Best Regards, > Michael > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Mon Feb 4 15:30:44 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 4 Feb 2019 15:30:44 +0100 Subject: [erlang-questions] Patch Package OTP 21.2.5 Released Message-ID: <20190204143044.GA32472@erix.ericsson.se> Patch Package: OTP 21.2.5 Git Tag: OTP-21.2.5 Date: 2019-02-04 Trouble Report Id: OTP-15554 Seq num: ERIERL-289 System: OTP Release: 21 Application: inets-7.0.5 Predecessor: OTP 21.2.4 Check out the git tag OTP-21.2.5, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- inets-7.0.5 ----------------------------------------------------- --------------------------------------------------------------------- The inets-7.0.5 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15554 Application(s): inets Related Id(s): ERIERL-289 Fixed bug that causes a crash in http client when using hostnames (e.g. localhost) with the the option ipv6_host_with_brackets set to true. This change also fixes a regression: httpc:request fails with connection error (nxdomain) if option ipv6_host_with_brackets set to true and host component of the URI is an IPv6 address. Full runtime dependencies of inets-7.0.5: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From frank.muller.erl@REDACTED Wed Feb 6 11:53:32 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 6 Feb 2019 11:53:32 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131221414.GF1915@ferdmbp.local> <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> Message-ID: Hello Michael and Fred Combing both solutions gave the best result. No need to even touch the memory allocators at all and the system look stable for the last days. Michael: I?m passing a sub_binary to the child process created with your sync_fun/2. Is it still a good idea to call binary:copy/1 on this sub_bin inside the Fun? I think it?s useless because this process is garbage collected ASAP anyway. /Frank Hello Michael > > Thanks again for the tip. I will update my design accordingly and let you > know how it goes. > > /Frank > > On 2/1/19 10:53 PM, Frank Muller wrote: >> > Hi Michael >> > >> > All packets in transit have a ?seq_id? (sequential number). >> > >> > This means that in theory packet1, packet2...packetN can be checked in >> > parallel and in any order (which is not the case in my current >> > design), but they must be send to the next processing stage in order: >> > packet1 first, then packet2... >> > >> > I would love to hear from you how can I turn this long-lived process >> > to multiple short-lived ones while enforcing ordering. >> >> An easy way to think about it comes from a module I used in the past >> called immediate_gc >> (https://gist.github.com/okeuday/dee991d580eeb00cd02c). The sync_fun/2 >> function is below: >> >> >> sync_fun(F, A) when is_function(F), is_list(A) -> >> Parent = self(), >> Child = erlang:spawn_opt(fun() -> >> Parent ! {self(), erlang:apply(F, A)}, >> erlang:garbage_collect() >> end, [link, {fullsweep_after, 0}]), >> receive >> {Child, Result} -> Result >> end. >> >> That is all you need to use a temporary process in a blocking way that >> consumes all the temporary binary data as quickly as the BEAM allows. >> However, that example is more complex than it needs to be, with the >> child process using the fullsweep_after option and >> erlang:garbage_collect/0. The extra complexity in the example is really >> not necessary or desirable, though it does force the garbage collection >> to occur as quickly as possible when consuming the temporary binary data >> (binary data that nothing else references). >> >> Spawn a similar child process before you start decoding a large binary, >> so the temporary Erlang process has a lifetime the length of the single >> request (packet in your situation) or less. Spawning Erlang processes is >> cheap, so you shouldn't hesitate to use them, just ensure they are >> linked so their failures may be tracked. >> >> CloudI (https://cloudi.org) internal services use temporary processes >> for handling service requests, in a way that is tunable with the service >> configuration options request_pid_uses and info_pid_uses , so you can >> control how many requests are processed in a temporary Erlang process >> before a new one is created (with the exit exception being used to >> terminate the Erlang process with its last result). CloudI internal >> services also have the hibernate service configuration option, with the >> hibernate based on request rate checked every few seconds (the service >> configuration options are described at >> http://cloudi.org/api.html#2_services_add_config_opts). >> >> The idea of using a temporary Erlang process for consuming temporary >> binary data, is likely unusual for people new to Erlang/Elixir and may >> not have found its way into Erlang/Elixir books, though it is important >> to know about if you want to avoid excessive memory consumption (and >> potentially causing the BEAM to die due to memory use). >> >> Best Regards, >> Michael >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From areege.alzubaidi@REDACTED Wed Feb 6 15:48:15 2019 From: areege.alzubaidi@REDACTED (Areege Al Zubaidi) Date: Wed, 6 Feb 2019 14:48:15 +0000 Subject: [erlang-questions] Schedule announced for Code BEAM SF 2019! Message-ID: Hi all, I wanted to drop you a line and let you know that the schedule for Code BEAM SF has arrived! At Code BEAM SF this year, we have the return of Kazoo training , as well as certification exams for Erlang and OTP, and a host of talks from leaders and innovators in the community, including: - Jose Valim , author of Elixir, who will be keynoting! - Renee Orser , VP of Engineering at NS1, will also join us as a keynote speaker! - Tyler Bettilyon answers the question ?Are programmers heading towards another bursting bubble?? in his keynote - Miriam Pena , voted one of the women to watch in tech by Women 2.0 - Fred Hebert , Erlang and distributed systems enthusiast, author of Learn you some Erlang - Igors Istocnicks shares how WhatsApp moved 1.5B users across data centres - Anna Neyzberg , co-founder of ElixirBridge - Brujo Benavides will share his tips on OTP behaviours and how to behave around them - Hannah Howard , programmer and life long iconoclast - Lukas Larsson , Erlang VM core committer, will present an update from the OTP team - Jacqui Manzi will join us to share her experience using OTP: presence to power real-time analytics - Maxim Fedorov will share mid-air airplane repair: troubleshooting at WhatsApp - ? and more! Learn more about the conference, and the 2019 speakers: https://codesync.global/conferences/code-beam-sf-2019/#Speakers What: Code BEAM SF 2019 When: 28 February - 01 March 2019 Where: Hyatt Centric Fisherman?s Wharf, San Francisco How to attend: - Attend the conference Secure an standard rate ticket to the conference by 23:59 PST on Thursday 19 February. You can find out more and register here . - Student tickets Code BEAM SF offers a 50% discount to students. To learn more, email info@REDACTED from your academic email address to get a unique discount code. - Group tickets Bring the whole team along! Contact info@REDACTED for group discounts. - Sponsor If you want your company to join WhatsApp, Brex, and The RealReal and support the BEAM community, please do not hesitate to contact us about community support and sponsorship. We hope to see you at Code BEAM SF! Warm regards, Areege -- *Areege Al Zubaidi* *Conferences Marketing Executive* T: +44 (0) 2076550332 W: www.codesync.global A: Erlang Solutions Ltd, The Loom,14 Gower's Walk, London, E1 8PY *Code Sync & Erlang Solutions Conferences* Lambda Days - Krak?w - 21-22 February 2019 Code BEAM SF - San Francisco - 28 February - 1 March 2019 Code BEAM Lite - Bologna - 22 March 2019 ElixirConf EU - Prague - 8-9 April 2019 Code BEAM STO - Stockholm - 16-17 May 2019 Code Elixir LDN - London - 18 July 2019 Code Mesh LDN - London - 7-8 November 2019 *Erlang Solutions and Code Sync care about your data and privacy; please find all details about the basis for communicating with you and the way we process your data in our Privacy Policy . You can update your email preferences or opt-out from receiving marketing emails here .* -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Feb 6 16:50:06 2019 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 6 Feb 2019 07:50:06 -0800 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> Message-ID: On 2/6/19 2:53 AM, Frank Muller wrote: > Hello Michael and Fred > > Combing both solutions gave the best result. No need to even touch the > memory allocators at all and the system look stable for the last days. > > Michael: I?m passing a sub_binary to the child process created with > your sync_fun/2. Is it still a good idea to call binary:copy/1 on this > sub_bin inside the Fun? I think it?s useless because this process is > garbage collected ASAP anyway. It isn't necessary.? It should be better and simpler to only create a sub_binary inside the temporary process, but it may depend on your situation.? The reference counting will ensure all the binaries you need after the temporary process is done are kept and you don't need to do anything special for that to happen (only keep all the temporary binaries inside the temporary process). Best Regards, Michael From frank.muller.erl@REDACTED Wed Feb 6 18:46:30 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 6 Feb 2019 18:46:30 +0100 Subject: [erlang-questions] VM leaking memory In-Reply-To: References: <20190131225247.GH1915@ferdmbp.local> <602098a4-abd5-2f10-bcab-f17aba67cbdd@gmail.com> <9086f10c-6bb0-acb6-466c-d4ff3407ac8a@gmail.com> Message-ID: Hi Michael My sub_bin is built outside, but visible from inside the Fun?s scope. /Frank > On 2/6/19 2:53 AM, Frank Muller wrote: > > Hello Michael and Fred > > > > Combing both solutions gave the best result. No need to even touch the > > memory allocators at all and the system look stable for the last days. > > > > Michael: I?m passing a sub_binary to the child process created with > > your sync_fun/2. Is it still a good idea to call binary:copy/1 on this > > sub_bin inside the Fun? I think it?s useless because this process is > > garbage collected ASAP anyway. > > It isn't necessary. It should be better and simpler to only create a > sub_binary inside the temporary process, but it may depend on your > situation. The reference counting will ensure all the binaries you need > after the temporary process is done are kept and you don't need to do > anything special for that to happen (only keep all the temporary > binaries inside the temporary process). > > Best Regards, > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Thu Feb 7 15:35:43 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Thu, 7 Feb 2019 15:35:43 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data Message-ID: Hi, I would like to understand some things about ETS memory fragmentation after deleting data. My current (probably faulty) mental model of the issue looks like this: - For every object in an ETS table a block is allocated on a carrier (typically a multi-block carrier, unless the object is huge). - Besides the objects themselves, the ETS table obviously needs some additional blocks too to describe the hash table data structure. The size of this data shall be small compared to the object data however (since ETS is not terribly space-inefficient), so I won't think about them any more. - If I delete some objects from an ETS table, the corresponding blocks are deallocated. However, the rest of the objects remain in their original location, so the carriers cannot be deallocated (unless all of their objects get deleted). - This implies that deleting a lot of data from ETS tables would lead to memory fragmentation. - Since there's no way to force ETS to rearrange the objects it already stores, the memory remains fragmented until subsequent updates to ETS tables fill the gaps with new objects. I wrote a small test program (available here ) to verify my mental model. But it doesn't exactly behave as I expected. 1. I create an ETS table and populate it with 1M objects, where each object is 1027 words large. I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ 7835 MiB (the size of all other ETS tables on a newly started Erlang node is negligible). And indeed I see that the total block size is ~7881 MiB and the total carrier size is ~7885 MiB (99.95% utilisation). 2. I delete 75% of the objects randomly. I expect the block size to go down by ~75% and the carrier size with some smaller value. In practice however the block size goes down by 87%, while the carrier size drops by 48% (resulting in a disappointing 25% utilisation). 3. Finally, I try to defragment the memory by overwriting each object that was left in the table with itself. I expect this operation to have no effect on the block size, but close the gap between the block size and carrier size by compacting the blocks on fewer carriers. In practice however the block size goes up by 91%(!!!), while the carrier size comes down very close to this new block size (utilisation is back at 99.56%). All in all, compared to the initial state in step 1, both block and carrier size is down by 75%. So here's the list of things I don't understand or know based on this exercise: - How could the block size drop by 87% after deleting 75% of the data in step 2? - Why did overwriting each object with itself resulted in almost doubling the block size? - Would you consider running a select_replace to compact a table after deletions safe in production? E.g. doing it on a Mnesia table that's several GB-s in size and is actively used by Mnesia transactions. (I know the replace is atomic on each object, but how would a long running replace affect the execution time of other operations for example?) - Step 3 helped to reclaim unused memory, but it almost doubled the used memory (the block size). I don't know what caused this behaviour, but is there an operation that would achieve the opposite effect? That is, without altering the contents of the table reduce the block size by 45-50%? Thanks, Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Thu Feb 7 22:14:45 2019 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 7 Feb 2019 21:14:45 +0000 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: Message-ID: <1549574084.12116.11.camel@ericsson.com> Hi D?niel I looked at your test code and I think it can be the 'mbcs_pool' stats that are missing. They are returned as {mbcs_pool,[{blocks_size,0}]} without carriers_size for some reason by erlang:system_info({allocator_sizes,ets_alloc}). Use erlang:system_info({allocator,ets_alloc}) to get mbcs_pool with both block and carrier sizes. Another thing that might confuse is that all binaries larger than 64 bytes will be stored in binary_alloc. /Sverker On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: Hi, I would like to understand some things about ETS memory fragmentation after deleting data. My current (probably faulty) mental model of the issue looks like this: * For every object in an ETS table a block is allocated on a carrier (typically a multi-block carrier, unless the object is huge). * Besides the objects themselves, the ETS table obviously needs some additional blocks too to describe the hash table data structure. The size of this data shall be small compared to the object data however (since ETS is not terribly space-inefficient), so I won't think about them any more. * If I delete some objects from an ETS table, the corresponding blocks are deallocated. However, the rest of the objects remain in their original location, so the carriers cannot be deallocated (unless all of their objects get deleted). * This implies that deleting a lot of data from ETS tables would lead to memory fragmentation. * Since there's no way to force ETS to rearrange the objects it already stores, the memory remains fragmented until subsequent updates to ETS tables fill the gaps with new objects. I wrote a small test program (available here) to verify my mental model. But it doesn't exactly behave as I expected. 1. I create an ETS table and populate it with 1M objects, where each object is 1027 words large. I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ 7835 MiB (the size of all other ETS tables on a newly started Erlang node is negligible). And indeed I see that the total block size is ~7881 MiB and the total carrier size is ~7885 MiB (99.95% utilisation). 2. I delete 75% of the objects randomly. I expect the block size to go down by ~75% and the carrier size with some smaller value. In practice however the block size goes down by 87%, while the carrier size drops by 48% (resulting in a disappointing 25% utilisation). 3. Finally, I try to defragment the memory by overwriting each object that was left in the table with itself. I expect this operation to have no effect on the block size, but close the gap between the block size and carrier size by compacting the blocks on fewer carriers. In practice however the block size goes up by 91%(!!!), while the carrier size comes down very close to this new block size (utilisation is back at 99.56%). All in all, compared to the initial state in step 1, both block and carrier size is down by 75%. So here's the list of things I don't understand or know based on this exercise: * How could the block size drop by 87% after deleting 75% of the data in step 2? * Why did overwriting each object with itself resulted in almost doubling the block size? * Would you consider running a select_replace to compact a table after deletions safe in production? E.g. doing it on a Mnesia table that's several GB-s in size and is actively used by Mnesia transactions. (I know the replace is atomic on each object, but how would a long running replace affect the execution time of other operations for example?) * Step 3 helped to reclaim unused memory, but it almost doubled the used memory (the block size). I don't know what caused this behaviour, but is there an operation that would achieve the opposite effect? That is, without altering the contents of the table reduce the block size by 45-50%? Thanks, Daniel _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Thu Feb 7 22:21:06 2019 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 7 Feb 2019 22:21:06 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: Message-ID: <1549574084.12116.11.camel@ericsson.com> Hi D?niel? I looked at your test code and I think it can be the 'mbcs_pool' stats that are missing. They are returned as?{mbcs_pool,[{blocks_size,0}]} without carriers_size for some reason by?erlang:system_info({allocator_sizes,ets_alloc}). Use erlang:system_info({allocator,ets_alloc}) to get mbcs_pool with both block and carrier sizes. Another thing that might confuse is that all binaries larger than 64 bytes will be stored in binary_alloc. /Sverker On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: > Hi, > > I would like to understand some things about ETS memory fragmentation after > deleting data. My current (probably faulty) mental model of the issue looks > like this: > For every object in an ETS table a block is allocated on a carrier (typically > a multi-block carrier, unless the object is huge). > Besides the objects themselves, the ETS table obviously needs some additional > blocks too to describe the hash table data structure. The size of this data > shall be small compared to the object data however (since ETS is not terribly > space-inefficient), so I won't think about them any more. > If I delete some objects from an ETS table, the corresponding blocks are > deallocated. However, the rest of the objects remain in their original > location, so the carriers cannot be deallocated (unless all of their objects > get deleted). > This implies that deleting a lot of data from ETS tables would lead to memory > fragmentation. > Since there's no way to force ETS to rearrange the objects it already stores, > the memory remains fragmented until subsequent updates to ETS tables fill the > gaps with new objects. > I wrote a small test program (available here) to verify my mental model. But > it doesn't exactly behave as I expected. > I create an ETS table and populate it with 1M objects, where each object is > 1027 words large. > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ 7835 MiB > (the size of all other ETS tables on a newly started Erlang node is > negligible). > > And indeed I see that the total block size is ~7881 MiB and the total carrier > size is ~7885 MiB (99.95% utilisation). > I delete 75% of the objects randomly. > > I expect the block size to go down by ~75% and the carrier size with some > smaller value. > > In practice however the block size goes down by 87%, while the carrier size > drops by 48% (resulting in a disappointing 25% utilisation). > Finally, I try to defragment the memory by overwriting each object that was > left in the table with itself. > > I expect this operation to have no effect on the block size, but close the gap > between the block size and carrier size by compacting the blocks on fewer > carriers. > > In practice however the block size goes up by 91%(!!!), while the carrier size > comes down very close to this new block size (utilisation is back at 99.56%). > All in all, compared to the initial state in step 1, both block and carrier > size is down by 75%. > So here's the list of things I don't understand or know based on this > exercise: > How could the block size drop by 87% after deleting 75% of the data in step 2? > Why did overwriting each object with itself resulted in almost doubling the > block size? > Would you consider running a select_replace to compact a table after deletions > safe in production? E.g. doing it on a Mnesia table that's several GB-s in > size and is actively used by Mnesia transactions. (I know the replace is > atomic on each object, but how would a long running replace affect the > execution time of other operations for example?) > Step 3 helped to reclaim unused memory, but it almost doubled the used memory > (the block size). I don't know what caused this behaviour, but is there an > operation that would achieve the opposite effect? That is, without altering > the contents of the table reduce the block size by 45-50%? > Thanks, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Thu Feb 7 22:25:10 2019 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 7 Feb 2019 22:25:10 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: Message-ID: <1549574710.12116.12.camel@erlang.org> Hi D?niel? I looked at your test code and I think it can be the 'mbcs_pool' stats that are missing. They are returned as?{mbcs_pool,[{blocks_size,0}]} without carriers_size for some reason by?erlang:system_info({allocator_sizes,ets_alloc}). Use erlang:system_info({allocator,ets_alloc}) to get mbcs_pool with both block and carrier sizes. Another thing that might confuse is that all binaries larger than 64 bytes will be stored in binary_alloc. /Sverker On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: > Hi, > > I would like to understand some things about ETS memory fragmentation after > deleting data. My current (probably faulty) mental model of the issue looks > like this: > For every object in an ETS table a block is allocated on a carrier (typically > a multi-block carrier, unless the object is huge). > Besides the objects themselves, the ETS table obviously needs some additional > blocks too to describe the hash table data structure. The size of this data > shall be small compared to the object data however (since ETS is not terribly > space-inefficient), so I won't think about them any more. > If I delete some objects from an ETS table, the corresponding blocks are > deallocated. However, the rest of the objects remain in their original > location, so the carriers cannot be deallocated (unless all of their objects > get deleted). > This implies that deleting a lot of data from ETS tables would lead to memory > fragmentation. > Since there's no way to force ETS to rearrange the objects it already stores, > the memory remains fragmented until subsequent updates to ETS tables fill the > gaps with new objects. > I wrote a small test program (available here) to verify my mental model. But > it doesn't exactly behave as I expected. > I create an ETS table and populate it with 1M objects, where each object is > 1027 words large. > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ 7835 MiB > (the size of all other ETS tables on a newly started Erlang node is > negligible). > > And indeed I see that the total block size is ~7881 MiB and the total carrier > size is ~7885 MiB (99.95% utilisation). > I delete 75% of the objects randomly. > > I expect the block size to go down by ~75% and the carrier size with some > smaller value. > > In practice however the block size goes down by 87%, while the carrier size > drops by 48% (resulting in a disappointing 25% utilisation). > Finally, I try to defragment the memory by overwriting each object that was > left in the table with itself. > > I expect this operation to have no effect on the block size, but close the gap > between the block size and carrier size by compacting the blocks on fewer > carriers. > > In practice however the block size goes up by 91%(!!!), while the carrier size > comes down very close to this new block size (utilisation is back at 99.56%). > All in all, compared to the initial state in step 1, both block and carrier > size is down by 75%. > So here's the list of things I don't understand or know based on this > exercise: > How could the block size drop by 87% after deleting 75% of the data in step 2? > Why did overwriting each object with itself resulted in almost doubling the > block size? > Would you consider running a select_replace to compact a table after deletions > safe in production? E.g. doing it on a Mnesia table that's several GB-s in > size and is actively used by Mnesia transactions. (I know the replace is > atomic on each object, but how would a long running replace affect the > execution time of other operations for example?) > Step 3 helped to reclaim unused memory, but it almost doubled the used memory > (the block size). I don't know what caused this behaviour, but is there an > operation that would achieve the opposite effect? That is, without altering > the contents of the table reduce the block size by 45-50%? > Thanks, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivancarmenates@REDACTED Fri Feb 8 01:09:07 2019 From: ivancarmenates@REDACTED (=?UTF-8?Q?Iv=C3=A1n_Carmenates?=) Date: Thu, 7 Feb 2019 19:09:07 -0500 Subject: [erlang-questions] Looking for a Job in Erlang. Message-ID: Hi everybody I am currently and actively looking for a job in Erlang, can you give me one? Here I send you my resume and a test I already made to show my knowledge, it include OTP Principles, distribution, common test, covering, xref, dialyzer etc. https://github.com/StartSWest/erlang_chat_test https://www.linkedin.com/in/ivan-carmenates-garcia-9253b7b4/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Fri Feb 8 08:52:17 2019 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 8 Feb 2019 08:52:17 +0100 Subject: [erlang-questions] Looking for a Job in Erlang. In-Reply-To: References: Message-ID: Hello, Ivan. I have received several job offers pertaining to Erlang and elixir over LinkedIn without prompting. I'm based in Europe, however. I guess from your profile you're looking in the Americas? Then the recruiter contacts might not be helpful to you. Oliver? Gesendet:?Freitag, 08. Februar 2019 um 01:09 Uhr Von:?"Iv?n Carmenates" An:?erlang-questions Betreff:?[erlang-questions] Looking for a Job in Erlang. Hi everybody I am currently and actively looking for a job in Erlang, can you give me one? ? Here I send you my resume and a test I already made to show my knowledge, it include OTP Principles, distribution, common test, covering, xref, dialyzer etc. ? https://github.com/StartSWest/erlang_chat_test https://www.linkedin.com/in/ivan-carmenates-garcia-9253b7b4/_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From robert.harris@REDACTED Fri Feb 8 15:04:13 2019 From: robert.harris@REDACTED (Harris, Robert) Date: Fri, 8 Feb 2019 14:04:13 +0000 Subject: [erlang-questions] erl_nif: persist binary across calls Message-ID: <802DA8FC-8C0A-46E3-A28D-5999190B5598@alertlogic.com> Hi, I am writing a NIF library in which I would like to create persistent binary terms in my load() callback. The purpose of these terms is to act as ready-made keys for frequent calls to enif_get_map_value() in successive calls to other NIFs in the same library. My test case is > #include > #include > > ERL_NIF_TERM persistent; > > static ERL_NIF_TERM > create_binary(ErlNifEnv *env) > { > ErlNifBinary binary; > char *data = "abcdefghijklmnopqrstuvwxyz"; > size_t size = strlen(data); > > (void) enif_alloc_binary(size, &binary); > memcpy(binary.data, data, size); > binary.size = size; > return (enif_make_binary(env, &binary)); > } > > int > load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) > { > persistent = create_binary(env); > return (0); > } > > static ERL_NIF_TERM > world_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return (persistent); > } > > static ERL_NIF_TERM > hello_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > { > return (create_binary(env)); > } > > static ErlNifFunc nif_funcs[] = { > {"hello", 0, hello_nif}, > {"world", 0, world_nif}, > }; > > ERL_NIF_INIT(hello, nif_funcs, &load, NULL, NULL, NULL) An example of the behaviour I see is > Erlang/OTP 20 [erts-9.3.3.3] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V9.3.3.3 (abort with ^G) > 1> > 1> hello:hello(). > <<"abcdefghijklmnopqrstuvwxyz">> > 2> hello:world(). > eheap_alloc: Cannot allocate 2305843009213695256 bytes of memory (of type "heap_frag"). > > Crash dump is being written to: erl_crash.dump...done > make: *** [shell] Error 1 I'm guessing that what I'm trying is wrong but I'd appreciate a pointer as to why. I'm motivated in part by comments like the following extract from enif_make_new_binary(): > The drawbacks are that the binary cannot be kept between NIF calls and it cannot > be reallocated. I inferred from this that a binary generated by enif_make_binary() *can* be kept between NIF calls. I'm a newcomer to Erlang, if that's not obvious. Regards, Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3565 bytes Desc: not available URL: From sverker@REDACTED Fri Feb 8 16:49:44 2019 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 8 Feb 2019 16:49:44 +0100 Subject: [erlang-questions] erl_nif: persist binary across calls In-Reply-To: <802DA8FC-8C0A-46E3-A28D-5999190B5598@alertlogic.com> References: <802DA8FC-8C0A-46E3-A28D-5999190B5598@alertlogic.com> Message-ID: <1549640984.3905.10.camel@erlang.org> Hi Robert, The 'env' you get in 'load' is temporary. Terms created in it will disappear when 'load' returns. Use enif_alloc_env() to create terms that will survive between NIF calls: ErlNifEnv* persistent_env; int load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) { ? ? persistent_env = enif_alloc_env(); ? ? persistent = create_binary(persistent_env); ? ? return 0; } Use enif_make_copy() to copy terms between ErlNifEnv's: static ERL_NIF_TERM world_nif(ErlNifEnv* env, ...) { ? ? return enif_make_copy(persistent); } If the persistent binary is large (> 64 bytes) then only a small term header will be copied pointing to the binary data. /Sverker On fre, 2019-02-08 at 14:04 +0000, Harris, Robert wrote: > Hi, > > I am writing a NIF library in which I would like to create persistent binary > terms > in my load() callback.??The purpose of these terms is to act as ready-made > keys > for frequent calls to enif_get_map_value() in successive calls to other NIFs > in the > same library. > > My test case is > > > > > #include > > #include > > > > ERL_NIF_TERM persistent; > > > > static ERL_NIF_TERM > > create_binary(ErlNifEnv *env) > > { > > ????????ErlNifBinary binary; > > ????????char *data = "abcdefghijklmnopqrstuvwxyz"; > > ????????size_t size = strlen(data); > > > > ????????(void) enif_alloc_binary(size, &binary); > > ????????memcpy(binary.data, data, size); > > ????????binary.size = size; > > ????????return (enif_make_binary(env, &binary)); > > } > > > > int > > load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) > > { > > ????????persistent = create_binary(env); > > ????????return (0); > > } > > > > static ERL_NIF_TERM > > world_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > > { > > ?????????return (persistent); > > } > > > > static ERL_NIF_TERM > > hello_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) > > { > > ?????????return (create_binary(env)); > > } > > > > static ErlNifFunc nif_funcs[] = { > > ????????{"hello", 0, hello_nif}, > > ????????{"world", 0, world_nif}, > > }; > > > > ERL_NIF_INIT(hello, nif_funcs, &load, NULL, NULL, NULL) > An example of the behaviour I see is > > > > > Erlang/OTP 20 [erts-9.3.3.3] [source] [64-bit] [smp:12:12] [ds:12:12:10] > > [async-threads:10] [hipe] [kernel-poll:false] > > > > Eshell V9.3.3.3??(abort with ^G) > > 1> > > 1> hello:hello(). > > <<"abcdefghijklmnopqrstuvwxyz">> > > 2> hello:world(). > > eheap_alloc: Cannot allocate 2305843009213695256 bytes of memory (of type > > "heap_frag"). > > > > Crash dump is being written to: erl_crash.dump...done > > make: *** [shell] Error 1 > I'm guessing that what I'm trying is wrong but I'd appreciate a pointer as to > why.??I'm motivated in part by comments like the following extract from > enif_make_new_binary(): > > > > > The drawbacks are that the binary cannot be kept between NIF calls and it > > cannot > > be reallocated. > I inferred from this that a binary generated by enif_make_binary() *can* be > kept > between NIF calls.??I'm a newcomer to Erlang, if that's not obvious. > > Regards, > > Robert > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.harris@REDACTED Fri Feb 8 18:38:59 2019 From: robert.harris@REDACTED (Harris, Robert) Date: Fri, 8 Feb 2019 17:38:59 +0000 Subject: [erlang-questions] erl_nif: persist binary across calls In-Reply-To: <1549640984.3905.10.camel@erlang.org> References: <802DA8FC-8C0A-46E3-A28D-5999190B5598@alertlogic.com> <1549640984.3905.10.camel@erlang.org> Message-ID: <16E2C6D9-98A6-4301-BD3A-DCFD3320AFC0@alertlogic.com> Hi Sverker, That worked perfectly. Many thanks, Robert > On 8 Feb 2019, at 15:49, Sverker Eriksson wrote: > > Hi Robert, > > The 'env' you get in 'load' is temporary. Terms created in it will disappear > when 'load' returns. > Use enif_alloc_env() to create terms that will survive between NIF calls: > > ErlNifEnv* persistent_env; > > int load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) > { > persistent_env = enif_alloc_env(); > persistent = create_binary(persistent_env); > return 0; > } > > > Use enif_make_copy() to copy terms between ErlNifEnv's: > > static ERL_NIF_TERM world_nif(ErlNifEnv* env, ...) > { > return enif_make_copy(persistent); > } > > If the persistent binary is large (> 64 bytes) then only a small term header > will be copied pointing to the binary data. > > > /Sverker > > On fre, 2019-02-08 at 14:04 +0000, Harris, Robert wrote: >> Hi, >> >> I am writing a NIF library in which I would like to create persistent binary >> terms >> in my load() callback. The purpose of these terms is to act as ready-made >> keys >> for frequent calls to enif_get_map_value() in successive calls to other NIFs >> in the >> same library. >> >> My test case is >> >>> >>> #include >>> #include >>> >>> ERL_NIF_TERM persistent; >>> >>> static ERL_NIF_TERM >>> create_binary(ErlNifEnv *env) >>> { >>> ErlNifBinary binary; >>> char *data = "abcdefghijklmnopqrstuvwxyz"; >>> size_t size = strlen(data); >>> >>> (void) enif_alloc_binary(size, &binary); >>> memcpy(binary.data, data, size); >>> binary.size = size; >>> return (enif_make_binary(env, &binary)); >>> } >>> >>> int >>> load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) >>> { >>> persistent = create_binary(env); >>> return (0); >>> } >>> >>> static ERL_NIF_TERM >>> world_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> return (persistent); >>> } >>> >>> static ERL_NIF_TERM >>> hello_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) >>> { >>> return (create_binary(env)); >>> } >>> >>> static ErlNifFunc nif_funcs[] = { >>> {"hello", 0, hello_nif}, >>> {"world", 0, world_nif}, >>> }; >>> >>> ERL_NIF_INIT(hello, nif_funcs, &load, NULL, NULL, NULL) >> An example of the behaviour I see is >> >>> >>> Erlang/OTP 20 [erts-9.3.3.3] [source] [64-bit] [smp:12:12] [ds:12:12:10] >>> [async-threads:10] [hipe] [kernel-poll:false] >>> >>> Eshell V9.3.3.3 (abort with ^G) >>> 1> >>> 1> hello:hello(). >>> <<"abcdefghijklmnopqrstuvwxyz">> >>> 2> hello:world(). >>> eheap_alloc: Cannot allocate 2305843009213695256 bytes of memory (of type >>> "heap_frag"). >>> >>> Crash dump is being written to: erl_crash.dump...done >>> make: *** [shell] Error 1 >> I'm guessing that what I'm trying is wrong but I'd appreciate a pointer as to >> why. I'm motivated in part by comments like the following extract from >> enif_make_new_binary(): >> >>> >>> The drawbacks are that the binary cannot be kept between NIF calls and it >>> cannot >>> be reallocated. >> I inferred from this that a binary generated by enif_make_binary() *can* be >> kept >> between NIF calls. I'm a newcomer to Erlang, if that's not obvious. >> >> Regards, >> >> Robert >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwIGaQ&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=rgvWXjU_2tA1ZUQSslrg0TYOEP4F7UfuySlE3gZm1mQ&m=KpVT9ckjVGBzTRcLi8KyIRtHYE40kNnBV6KUJC9f2cc&s=FGyBxvciQtMGXgbPAmz4GbykgMAb2i_uqS1EuBTNLA8&e= > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > https://urldefense.proofpoint.com/v2/url?u=http-3A__erlang.org_mailman_listinfo_erlang-2Dquestions&d=DwIGaQ&c=L_h2OePR2UWWefmqrezxOsP9Uqw55rRfX5bRtw9S4KY&r=rgvWXjU_2tA1ZUQSslrg0TYOEP4F7UfuySlE3gZm1mQ&m=KpVT9ckjVGBzTRcLi8KyIRtHYE40kNnBV6KUJC9f2cc&s=FGyBxvciQtMGXgbPAmz4GbykgMAb2i_uqS1EuBTNLA8&e= -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3565 bytes Desc: not available URL: From dszoboszlay@REDACTED Sat Feb 9 00:30:43 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Sat, 9 Feb 2019 00:30:43 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: <1549574710.12116.12.camel@erlang.org> References: <1549574710.12116.12.camel@erlang.org> Message-ID: Hi Sverker, Thanks for the tip, I changed my code in the gist to use erlang:system_info({allocator, ets_alloc}) and the weirdest things disappeared. (Also, I intentionally avoided storing binaries in the ETS table in this test, so the binary_alloc couldn't play a role in the results.) But now I see different "problems": - Deleting from the ETS table cannot free up any of the carriers. :( After deleting 75% of the objects I could regain 0 memory for the OS and the utilisation is down to a disappointing 25%. - Overwriting every object once with itself sometimes have no effect at all on the carrier size either. In this case a second round of overwrites are needed to free up carriers. - My memory compaction trick can now only achieve 50% utilisation. So the memory is still fragmented. - I tried to repeat the overwrite step a few more times, but once it reaches 50% utilisation it cannot improve on it any more. My guess was that maybe carrier abandoning causes this problem. I tried playing with +MEacul 0, some different +MEas settings and even with +MEramv true, but neither of them helped. So my new questions are: - What may be preventing my overwrite-with-self-compactor to go above 50% carrier utilisation? - Is there any trick that would help me further reduce the fragmentation and get back to 90%+ utilisation after deleting a lot of objects from ETS? - Wouldn't ERTS benefit from some built-in memory defragmentator utility, at least for ets_alloc? (For example I don't think eheap_alloc would need it: the copying GC effectively performs defragmentation automatically. binary_alloc would also be a potential candidate, but it may be significantly harder to implement, and I guess most systems store less binary data than ETS data.) Thanks, Daniel On Thu, 7 Feb 2019 at 22:25 Sverker Eriksson wrote: > Hi D?niel > > I looked at your test code and I think it can be the 'mbcs_pool' stats > that are missing. > > They are returned as {mbcs_pool,[{blocks_size,0}]} without carriers_size > for some reason > by erlang:system_info({*allocator_sizes*,ets_alloc}). > > Use erlang:system_info({*allocator*,ets_alloc}) to get mbcs_pool with > both block and carrier sizes. > > Another thing that might confuse is that all binaries larger than 64 bytes > will be stored in binary_alloc. > > /Sverker > > > On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: > > Hi, > > I would like to understand some things about ETS memory fragmentation > after deleting data. My current (probably faulty) mental model of the issue > looks like this: > > - For every object in an ETS table a block is allocated on a carrier > (typically a multi-block carrier, unless the object is huge). > - Besides the objects themselves, the ETS table obviously needs some > additional blocks too to describe the hash table data structure. The size > of this data shall be small compared to the object data however (since ETS > is not terribly space-inefficient), so I won't think about them any more. > - If I delete some objects from an ETS table, the corresponding blocks > are deallocated. However, the rest of the objects remain in their original > location, so the carriers cannot be deallocated (unless all of their > objects get deleted). > - This implies that deleting a lot of data from ETS tables would lead > to memory fragmentation. > - Since there's no way to force ETS to rearrange the objects it > already stores, the memory remains fragmented until subsequent updates to > ETS tables fill the gaps with new objects. > > I wrote a small test program (available here > ) > to verify my mental model. But it doesn't exactly behave as I expected. > > 1. I create an ETS table and populate it with 1M objects, where each > object is 1027 words large. > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ > 7835 MiB (the size of all other ETS tables on a newly started Erlang node > is negligible). > > And indeed I see that the total block size is ~7881 MiB and the total > carrier size is ~7885 MiB (99.95% utilisation). > 2. I delete 75% of the objects randomly. > > I expect the block size to go down by ~75% and the carrier size with > some smaller value. > > In practice however the block size goes down by 87%, while the carrier > size drops by 48% (resulting in a disappointing 25% utilisation). > 3. Finally, I try to defragment the memory by overwriting each object > that was left in the table with itself. > > I expect this operation to have no effect on the block size, but close > the gap between the block size and carrier size by compacting the blocks on > fewer carriers. > > In practice however the block size goes up by 91%(!!!), while the > carrier size comes down very close to this new block size (utilisation is > back at 99.56%). All in all, compared to the initial state in step 1, both > block and carrier size is down by 75%. > > So here's the list of things I don't understand or know based on this > exercise: > > - How could the block size drop by 87% after deleting 75% of the data > in step 2? > - Why did overwriting each object with itself resulted in almost > doubling the block size? > - Would you consider running a select_replace to compact a table after > deletions safe in production? E.g. doing it on a Mnesia table that's > several GB-s in size and is actively used by Mnesia transactions. (I know > the replace is atomic on each object, but how would a long running replace > affect the execution time of other operations for example?) > - Step 3 helped to reclaim unused memory, but it almost doubled the > used memory (the block size). I don't know what caused this behaviour, but > is there an operation that would achieve the opposite effect? That is, > without altering the contents of the table reduce the block size by 45-50%? > > Thanks, > Daniel > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sun Feb 10 20:33:36 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 10 Feb 2019 14:33:36 -0500 Subject: [erlang-questions] Compound guard expression in case Message-ID: <4b90d1df-3a0c-01e6-bc61-b28028c4bb8b@aim.com> Newbie question regarding best practices (including a string of nested ifs): How do you code the following: case X of ??? 4???????????? -> something; ??? <2 or >6 -> something else; ??? _???????????? -> something else again end, and case X of ??? atom1 or atom2 or atom3 -> something; ??? atom4??????????????????????????????????? -> something else; ??? _ ->???????????????????????????????????????? -> something else again end, Thanks! Don From hugo@REDACTED Sun Feb 10 20:43:29 2019 From: hugo@REDACTED (Hugo Mills) Date: Sun, 10 Feb 2019 19:43:29 +0000 Subject: [erlang-questions] Compound guard expression in case In-Reply-To: <4b90d1df-3a0c-01e6-bc61-b28028c4bb8b@aim.com> References: <4b90d1df-3a0c-01e6-bc61-b28028c4bb8b@aim.com> Message-ID: <20190210194329.GC31067@carfax.org.uk> On Sun, Feb 10, 2019 at 02:33:36PM -0500, Donald Steven wrote: > Newbie question regarding best practices (including a string of nested ifs): > > How do you code the following: Guards: > case X of > ??? 4???????????? -> something; > ??? <2 or >6 -> something else; > ??? _???????????? -> something else again > end, case X of 4 -> something; Y when Y < 2; Y > 6 -> something else; _ -> something else again end, Note that >=2 and =<6 would be written with "when Y >= 2, Y =< 6" -- use comma for "and", semicolon for "or", with "or" being at the higher level: Y when Y >= 2, Y =< 6; Y >= 8, Y =< 10 -> Y is either between 2 and 6 or between 8 and 10 inclusive; > and > > case X of > ??? atom1 or atom2 or atom3 -> something; > ??? atom4??????????????????????????????????? -> something else; > ??? _ ->???????????????????????????????????????? -> something else again > end, case X of Y when Y =:= atom1; Y =:= atom2; Y =:= atom3 -> something; atom4 -> something else; _ -> something else again end, Hugo. -- Hugo Mills | I have a step-ladder. My real ladder left when I was hugo@REDACTED carfax.org.uk | a child. http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From raoknz@REDACTED Mon Feb 11 01:17:29 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Mon, 11 Feb 2019 13:17:29 +1300 Subject: [erlang-questions] Compound guard expression in case In-Reply-To: <20190210194329.GC31067@carfax.org.uk> References: <4b90d1df-3a0c-01e6-bc61-b28028c4bb8b@aim.com> <20190210194329.GC31067@carfax.org.uk> Message-ID: The first example would probably be better as if X =:= 4 -> first case ; X >= 2, X =< 6 -> third case ; true -> second case end The second example makes me unhappy. We are so far in the misty clouds of abstraction here that we can't see the ground with a telescope. And the thing that makes me unhappy is the "_" case. This is a maintenance bug waiting to happen, and it does not matter what your programming language is. This is the ML lesson. A concrete example would be a big help. On Mon, 11 Feb 2019 at 08:43, Hugo Mills wrote: > On Sun, Feb 10, 2019 at 02:33:36PM -0500, Donald Steven wrote: > > Newbie question regarding best practices (including a string of nested > ifs): > > > > How do you code the following: > > Guards: > > > case X of > > 4 -> something; > > <2 or >6 -> something else; > > _ -> something else again > > end, > > case X of > 4 -> > something; > Y when Y < 2; Y > 6 -> > something else; > _ -> > something else again > end, > > Note that >=2 and =<6 would be written with "when Y >= 2, Y =< 6" -- > use comma for "and", semicolon for "or", with "or" being at the higher > level: > > Y when Y >= 2, Y =< 6; Y >= 8, Y =< 10 -> > Y is either between 2 and 6 or between 8 and 10 inclusive; > > > and > > > > case X of > > atom1 or atom2 or atom3 -> something; > > atom4 -> something else; > > _ -> -> something else again > > end, > > case X of > Y when Y =:= atom1; Y =:= atom2; Y =:= atom3 -> > something; > atom4 -> > something else; > _ -> > something else again > end, > > Hugo. > > -- > Hugo Mills | I have a step-ladder. My real ladder left when I > was > hugo@REDACTED carfax.org.uk | a child. > http://carfax.org.uk/ | > PGP: E2AB1DE4 | > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Mon Feb 11 01:43:01 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 10 Feb 2019 19:43:01 -0500 Subject: [erlang-questions] Compound guard expression in case In-Reply-To: References: <4b90d1df-3a0c-01e6-bc61-b28028c4bb8b@aim.com> <20190210194329.GC31067@carfax.org.uk> Message-ID: <08b44e3e-4c9c-de21-3a49-508e3969b6ae@aim.com> Thanks, re the first example.? I had an if statement solution, but I was under the impression that ifs were frowned upon. Re the second example, each atom in fact represents a possible musical fork.? So, to create a bit of an odd but useful example: case SoundType of: ??? ostinato???????????????????????????????????????????? -> some action; ??? melisma or melody or ornament? -> some action common to all three atoms; ?? _?????????????????????????????????????????????????????????? -> error_message()??? % i.e., an unrecognized SoundType end, I was trying to understand how to do the 'or' part (the second guard statement) without repeating the code 3 times. On 2/10/2019 7.17 PM, Richard O'Keefe wrote: > The first example would probably be better as > ? ?if X =:= 4 -> first case > ? ? ; X >= 2, X =< 6 -> third case > ? ? ; true ?-> second case > ? ?end > > The second example makes me unhappy.? We are so far in the misty clouds of > abstraction here that we can't see the ground with a telescope.? And the > thing that makes me unhappy is the "_" case.? This is a maintenance bug > waiting to happen, and it does not matter what your programming > language is. > This is the ML lesson.? A concrete example would be a big help. > > On Mon, 11 Feb 2019 at 08:43, Hugo Mills > wrote: > > On Sun, Feb 10, 2019 at 02:33:36PM -0500, Donald Steven wrote: > > Newbie question regarding best practices (including a string of > nested ifs): > > > > How do you code the following: > > ? ?Guards: > > > case X of > > ??? 4???????????? -> something; > > ??? <2 or >6 -> something else; > > ??? _???????????? -> something else again > > end, > > case X of > ? ? 4 -> > ? ? ? ? something; > ? ? Y when Y < 2; Y > 6 -> > ? ? ? ? something else; > ? ? _ -> > ? ? ? ? something else again > end, > > Note that >=2 and =<6 would be written with "when Y >= 2, Y =< 6" -- > use comma for "and", semicolon for "or", with "or" being at the higher > level: > > ? ?Y when Y >= 2, Y =< 6; Y >= 8, Y =< 10 -> > ? ? ? ?Y is either between 2 and 6 or between 8 and 10 inclusive; > > > and > > > > case X of > > ??? atom1 or atom2 or atom3 -> something; > > ??? atom4??????????????????????????????????? -> something else; > > ??? _ ->???????????????????????????????????????? -> something > else again > > end, > > case X of > ? ? Y when Y =:= atom1; Y =:= atom2; Y =:= atom3 -> > ? ? ? ? something; > ? ? atom4 -> > ? ? ? ? something else; > ? ? _ -> > ? ? ? ? something else again > end, > > ? ?Hugo. > > -- > Hugo Mills? ? ? ? ? ? ?| I have a step-ladder. My real ladder left > when I was > hugo@REDACTED carfax.org.uk | a child. > http://carfax.org.uk/ | > PGP: E2AB1DE4? ? ? ? ? | > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sperber@REDACTED Tue Feb 12 09:58:17 2019 From: sperber@REDACTED (Michael Sperber) Date: Tue, 12 Feb 2019 09:58:17 +0100 Subject: [erlang-questions] 2nd Call for Participation: BOB 2019 (March 22, Berlin) Message-ID: ================================================================================ BOB 2019 Conference ?What happens if we simply use what?s best?? March 22, 2019, Berlin http://bobkonf.de/2019/ Program: http://bobkonf.de/2019/en/program.html Registration: http://bobkonf.de/2019/en/registration.html ================================================================================ BOB is the conference for developers, architects and decision-makers to explore technologies beyond the mainstream in software development, and to find the best tools available to software developers today. Our goal is for all participants of BOB to return home with new insights that enable them to improve their own software development experiences. The program features 14 talks and 8 tutorials on current topics: http://bobkonf.de/2019/en/program.html The subject range of talks includes functional programming, formal methods, event sourcing, music, advanced SQL, logic, and feelings. The tutorials feature introductions to Racket, Clojure, Functional Programming, TypeScript, type-level programming, SQL indexing, probabilistic programming, and hardware. Gabriele Keller will give the keynote talk. Registration is open online: http://bobkonf.de/2019/en/registration.html NOTE: The early-bird rates expire on February 19, 2019! BOB cooperates with the RacketFest conference on the following day: https://racketfest.com/ From Dinislam.Salikhov@REDACTED Tue Feb 12 17:44:00 2019 From: Dinislam.Salikhov@REDACTED (Salikhov Dinislam) Date: Tue, 12 Feb 2019 19:44:00 +0300 Subject: [erlang-questions] high memory consumption in crash.dump Message-ID: <351db645-2d19-b233-2e4c-d313aeb79dc5@kaspersky.com> Hello. My VM (OTP-20.2) crashed with out of memory error. I noticed that some processes have unexpectedly high memory consumption. One of them is a long living process: =proc:<0.483.0> State: Waiting Name: periodic_checker Spawned as: proc_lib:init_p/5 Spawned by: <0.284.0> Started: Fri Jan 18 20:37:49 2019 Message queue length: 0 Number of heap fragments: 0 Heap fragment data: 0 Link list: [<0.284.0>] Reductions: 7015535411 Stack+heap: 1199557 OldHeap: 0 Heap unused: 313695 OldHeap unused: 0 Memory: 9597520 Program counter: 0x000000086de82df8 (gen_server:loop/7 + 632) CP: 0x0000000000000000 (invalid) arity = 0 Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ON_HEAP_MSGQ =proc_dictionary:<0.483.0> HED872D078 HED872D090 =proc_stack:<0.483.0> 0x0000000ed9053df8:SReturn addr 0x4368ACB0 (proc_lib:init_p_do_apply/3 + 72) y0:N y1:A8:infinity y2:I600000 y3:A10:periodic_checker y4:N y5:A10:periodic_checker y6:P<0.284.0> 0x0000000ed9053e38:SReturn addr 0xA2E048 () y0:N y1:SCatch 0x4368ACD0 (proc_lib:init_p_do_apply/3 + 104) =proc_heap:<0.483.0> ED872D078:t2:AD:$initial_call,HED872D0D0 ED872D0D0:t3:A10:periodic_checker,A4:init,I1 ED872D090:t2:AA:$ancestors,HED872D0F0 ED872D0F0:lAC:ejabberd_sup|HED872D110 ED872D110:lP<0.203.0>|N The process is a gen_server that once per 10 minutes searches a process with the longest message queue: init(_) -> ??? ets:new(periodic_checker, [named_table, {read_concurrency, true}]), ??? ets:insert(periodic_checker, {max_process_mq_len, get_self_mq_len()}), ??? {ok, [], 0}. handle_call(_, _, State) -> ??? {reply, undefined, State, 600000}. handle_cast(_, State) -> ??? {noreply, State, 600000}. handle_info(timeout, State) -> ??? update_max_process_mq_len(), ??? {noreply, State, 600000}; handle_info(_, State) -> ??? {noreply, State, 600000}. get_self_mq_len() -> ??? Pid = self(), ??? {_, MQ_len} = process_info(Pid, message_queue_len), ??? {Pid, MQ_len}. update_max_process_mq_len() -> ??? MaxMQ_len = lists:foldl(fun(Pid, Acc = {_, AccSize}) -> ??????????? case process_info(Pid, message_queue_len) of ??????????????? undefined -> Acc; ??????????????? {_, MQ_len} -> ??????????????????? if MQ_len > AccSize -> ??????????????????????? {Pid, MQ_len}; ??????????????????? true -> ??????????????????????? Acc ??????????????????? end ??????????? end ??? end, get_self_mq_len(), processes()), ??? ets:insert(periodic_checker, {max_process_mq_len, MaxMQ_len}). The thing that looks very suspicious is that the process consumed 9Mb of memory. For instance, on working node the process takes only ~88Kb. There are more such processes in crash.dump, this one is the simplest. Is there a way to find out what led to such high memory consumption? Any ideas are appreciated. Thanks in advance, Dinislam Salikhov From sverker@REDACTED Tue Feb 12 18:48:18 2019 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 12 Feb 2019 18:48:18 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: <1549574710.12116.12.camel@erlang.org> Message-ID: <1549993698.19868.20.camel@erlang.org> I think the table segments are whats keeping the carriers ?alive. When a set,bag or duplicate_bag grows new table segments are allocated. Each new segment contains 2048 hash buckets and the load limit for growth is 100%. This means for every 2048 object you insert a new segments i allocated. The load limit for shrinking is 50%, so after inserting 1 miljon objects you have to delete 0.5 miljon before the table starts to shrink and segments are deallocated. Increasing the shrink limit will reduce carrier fragmentation in your case, but it may also cost in performance from more frequent rehashing when number of objects fluctuates. The shrink limit is controlled by? #define SHRINK_LIMIT(NACTIVE) ((NACTIVE) / 2) in erts/emulator/beam/erl_db_hash.c /Sverker On l?r, 2019-02-09 at 00:30 +0100, D?niel Szoboszlay wrote: > Hi Sverker, > > Thanks for the tip, I changed my code in the gist to > use?erlang:system_info({allocator, ets_alloc}) and the weirdest things > disappeared. (Also, I intentionally avoided storing binaries in the ETS table > in this test, so the binary_alloc couldn't play a role in the results.) > > But now I see different "problems": > Deleting from the ETS table cannot free up any of the carriers. :( > After deleting 75% of the objects I could regain 0 memory for the OS and the > utilisation is down to a disappointing 25%. > Overwriting every object once with itself sometimes have no effect at all on > the carrier size either. In this case a second round of overwrites are needed > to free up carriers. > My memory compaction trick can now only achieve 50% utilisation. So the memory > is still fragmented. > I tried to repeat the overwrite step a few more times, but once it reaches 50% > utilisation it cannot improve on it any more. > My guess was that maybe carrier abandoning causes this problem. I tried > playing with?+MEacul 0, some different +MEas settings and even with?+MEramv > true, but neither of them helped. > > So my new questions are: > What may be preventing my overwrite-with-self-compactor to go above 50% > carrier utilisation? > Is there any trick that would help me further reduce the fragmentation and get > back to 90%+ utilisation after deleting a lot of objects from ETS? > Wouldn't ERTS benefit from some built-in memory defragmentator utility, at > least for ets_alloc? (For example I don't think eheap_alloc would need it: the > copying GC effectively performs defragmentation automatically. binary_alloc > would also be a potential candidate, but it may be significantly harder to > implement, and I guess most systems store less binary data than ETS data.) > Thanks, > Daniel > > On Thu, 7 Feb 2019 at 22:25 Sverker Eriksson wrote: > > Hi D?niel? > > > > I looked at your test code and I think it can be the 'mbcs_pool' stats that > > are missing. > > > > They are returned as?{mbcs_pool,[{blocks_size,0}]} without carriers_size for > > some reason > > by?erlang:system_info({allocator_sizes,ets_alloc}). > > > > Use erlang:system_info({allocator,ets_alloc}) to get mbcs_pool with both > > block and carrier sizes. > > > > Another thing that might confuse is that all binaries larger than 64 bytes > > will be stored in binary_alloc. > > > > /Sverker > > > > > > On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: > > > Hi, > > > > > > I would like to understand some things about ETS memory fragmentation > > > after deleting data. My current (probably faulty) mental model of the > > > issue looks like this: > > > For every object in an ETS table a block is allocated on a carrier > > > (typically a multi-block carrier, unless the object is huge). > > > Besides the objects themselves, the ETS table obviously needs some > > > additional blocks too to describe the hash table data structure. The size > > > of this data shall be small compared to the object data however (since ETS > > > is not terribly space-inefficient), so I won't think about them any more. > > > If I delete some objects from an ETS table, the corresponding blocks are > > > deallocated. However, the rest of the objects remain in their original > > > location, so the carriers cannot be deallocated (unless all of their > > > objects get deleted). > > > This implies that deleting a lot of data from ETS tables would lead to > > > memory fragmentation. > > > Since there's no way to force ETS to rearrange the objects it already > > > stores, the memory remains fragmented until subsequent updates to ETS > > > tables fill the gaps with new objects. > > > I wrote a small test program (available here) to verify my mental model. > > > But it doesn't exactly behave as I expected. > > > I create an ETS table and populate it with 1M objects, where each object > > > is 1027 words large. > > > > > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ 7835 > > > MiB (the size of all other ETS tables on a newly started Erlang node is > > > negligible). > > > > > > And indeed I see that the total block size is ~7881 MiB and the total > > > carrier size is ~7885 MiB (99.95% utilisation). > > > I delete 75% of the objects randomly. > > > > > > I expect the block size to go down by ~75% and the carrier size with some > > > smaller value. > > > > > > In practice however the block size goes down by 87%, while the carrier > > > size drops by 48% (resulting in a disappointing 25% utilisation). > > > Finally, I try to defragment the memory by overwriting each object that > > > was left in the table with itself. > > > > > > I expect this operation to have no effect on the block size, but close the > > > gap between the block size and carrier size by compacting the blocks on > > > fewer carriers. > > > > > > In practice however the block size goes up by 91%(!!!), while the carrier > > > size comes down very close to this new block size (utilisation is back at > > > 99.56%). All in all, compared to the initial state in step 1, both block > > > and carrier size is down by 75%. > > > So here's the list of things I don't understand or know based on this > > > exercise: > > > How could the block size drop by 87% after deleting 75% of the data in > > > step 2? > > > Why did overwriting each object with itself resulted in almost doubling > > > the block size? > > > Would you consider running a select_replace to compact a table after > > > deletions safe in production? E.g. doing it on a Mnesia table that's > > > several GB-s in size and is actively used by Mnesia transactions. (I know > > > the replace is atomic on each object, but how would a long running replace > > > affect the execution time of other operations for example?) > > > Step 3 helped to reclaim unused memory, but it almost doubled the used > > > memory (the block size). I don't know what caused this behaviour, but is > > > there an operation that would achieve the opposite effect? That is, > > > without altering the contents of the table reduce the block size by 45- > > > 50%? > > > Thanks, > > > Daniel > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jstimpson@REDACTED Tue Feb 12 20:21:28 2019 From: jstimpson@REDACTED (Jesse Stimpson) Date: Tue, 12 Feb 2019 14:21:28 -0500 Subject: [erlang-questions] Error report printed to stdout when using remsh to a tls dist cluster Message-ID: Hello, I've set up a distribution of 2 Erlang nodes, using "-proto_dist inet_tls", and when I connect to one of the nodes with remsh, I receive the following error report: Erlang/OTP 21 [erts-10.2.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] Eshell V10.2.3 (abort with ^G) (ssl_test2@REDACTED)1> =ERROR REPORT==== 12-Feb-2019::14:17:25.653872 === Error in process <0.68.0> on node 'ssl_test_remsh@REDACTED' with exit value: {{badmatch,{error,closed}}, [{inet_tls_dist,accept_loop,4,[{file,"inet_tls_dist.erl"},{line,253}]}]} The error appears to be benign; I don't observe any undesirable behavior from the Erlang nodes so far. It's also inconsistently seen; it takes a handful of new remsh attempts to produce the error. I've pinpointed the 21.2 release as the culprit, but there are many commits between 21.1.4 and 21.2, so it's difficult for me to isolate. To reproduce: set up 2 nodes with proto_dist inet_tls (call them A and B). Have B issue a ping to A. Then set up a remsh to B. Close and repeat the remsh until the error appears. From what I have seen this is the minimal set of steps to reproduce. When just using 1 node with a remsh, I've not seen the error. Thanks, Jesse Stimpson -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Wed Feb 13 13:40:59 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 13 Feb 2019 07:40:59 -0500 Subject: [erlang-questions] Best practices -- conditional statements Message-ID: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> The code excerpt below shows two alternate ways of branching. I'd be grateful for opinions on which represents best practices. Thanks. Don makePanPositionL(Notes, Mode, L), ??? CurrentPanPosition = hd(L), ??? case Mode of ??????? left2right -> ??????????? if ??????????????? Notes < 6 -> PanPosition = CurrentPanPosition + 8; ??????????????? Notes < 12 -> PanPosition = CurrentPanPosition + 4; ??????????????? true -> PanPosition = CurrentPanPosition + 2 ??????????? end; ??????? left2right -> ??????????? case Notes of ??????????????? Few? when Few? < 6 -> PanPosition = CurrentPanPosition + 8; ??????????????? More when More < 12 -> PanPosition = CurrentPanPosition + 4; ??????????????? _Lots -> PanPosition = CurrentPanPosition + 2 ??????????? end; From ivan@REDACTED Wed Feb 13 13:51:30 2019 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 13 Feb 2019 12:51:30 +0000 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> Message-ID: This might be cheating but I would avoid the branching altogether if poss., eg: ??? PanPosition = CurrentPanPosition + notes_to_pad(Notes), ??? ... ??? notes_to_pad(N) where N 8; ??? notes_to_pad(N) where N < 12 -> 4; ??? notes_to_pad(_)????????????? -> 2. Also, I would avoid assignment inside a conditional, preferring eg: ??? PanPosition = ??????? case Notes of Few when Few < 6 -> CurrentPanPosition + 8; ??????????? ... Best wishes Ivan On 13/02/2019 12:40, Donald Steven wrote: > The code excerpt below shows two alternate ways of branching. I'd be > grateful for opinions on which represents best practices. Thanks. > > > Don > > > makePanPositionL(Notes, Mode, L), > > ??? CurrentPanPosition = hd(L), > > ??? case Mode of > > ??????? left2right -> > ??????????? if > ??????????????? Notes < 6 -> PanPosition = CurrentPanPosition + 8; > ??????????????? Notes < 12 -> PanPosition = CurrentPanPosition + 4; > ??????????????? true -> PanPosition = CurrentPanPosition + 2 > ??????????? end; > > ??????? left2right -> > ??????????? case Notes of > ??????????????? Few? when Few? < 6 -> PanPosition = CurrentPanPosition > + 8; > ??????????????? More when More < 12 -> PanPosition = > CurrentPanPosition + 4; > ??????????????? _Lots -> PanPosition = CurrentPanPosition + 2 > ??????????? end; > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Ymchwil a Datblygu Technoleg Lleferydd Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From t6sn7gt@REDACTED Wed Feb 13 14:35:29 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 13 Feb 2019 08:35:29 -0500 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> Message-ID: <9d48ed71-8cbb-2b02-e992-b3656caf191f@aim.com> Thanks!? I'm gonna try that. My multilingual (using elements of Rust and c), heretical fantasy (with apologies) is something like: makePanPositionL(Notes, Mode, L), ??? mut CurrentPanPosition = hd(L), ??? case Mode of ??????? left2right -> ??????????? case Notes of ??????????????? < 6?? -> CurrentPanPosition += 8; ??????????????? < 12? -> CurrentPanPosition += 4; ??????????????? _else -> CurrentPanPosition += 2 ??????????? end; ??????? right2left -> ??? ??? ??? ... On 2/13/2019 7.51 AM, Ivan Uemlianin wrote: > This might be cheating but I would avoid the branching altogether if > poss., eg: > > ??? PanPosition = CurrentPanPosition + notes_to_pad(Notes), > ??? ... > > ??? notes_to_pad(N) where N 8; > ??? notes_to_pad(N) where N < 12 -> 4; > ??? notes_to_pad(_)????????????? -> 2. > > Also, I would avoid assignment inside a conditional, preferring eg: > > ??? PanPosition = > ??????? case Notes of > Few when Few < 6 -> CurrentPanPosition + 8; > ??????????? ... > > Best wishes > > Ivan > > > On 13/02/2019 12:40, Donald Steven wrote: >> The code excerpt below shows two alternate ways of branching. I'd be >> grateful for opinions on which represents best practices. Thanks. >> >> >> Don >> >> >> makePanPositionL(Notes, Mode, L), >> >> ??? CurrentPanPosition = hd(L), >> >> ??? case Mode of >> >> ??????? left2right -> >> ??????????? if >> ??????????????? Notes < 6 -> PanPosition = CurrentPanPosition + 8; >> ??????????????? Notes < 12 -> PanPosition = CurrentPanPosition + 4; >> ??????????????? true -> PanPosition = CurrentPanPosition + 2 >> ??????????? end; >> >> ??????? left2right -> >> ??????????? case Notes of >> ??????????????? Few? when Few? < 6 -> PanPosition = >> CurrentPanPosition + 8; >> ??????????????? More when More < 12 -> PanPosition = >> CurrentPanPosition + 4; >> ??????????????? _Lots -> PanPosition = CurrentPanPosition + 2 >> ??????????? end; >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Wed Feb 13 14:35:28 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Feb 2019 14:35:28 +0100 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> Message-ID: On Wed, Feb 13, 2019 at 1:51 PM Ivan Uemlianin wrote: > This might be cheating but I would avoid the branching altogether if > poss., eg: > > PanPosition = CurrentPanPosition + notes_to_pad(Notes), > ... > > notes_to_pad(N) where N < 6 -> 8; > notes_to_pad(N) where N < 12 -> 4; > notes_to_pad(_) -> 2. > > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6 -> Cur + 8; makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> Cur + 4; makePanPositionL(Notes, left2right, [Cur|_]) -> Cur + 2. You can just lift everything into a pattern match for this piece of code, which is what I think I'd do. It makes the result far more tabular, which in some cases is easier to read. There may be other reasons in the code to not do this, but as it was written, I think I'd pick the above as the solution. YMMV of course. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Wed Feb 13 14:45:49 2019 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 13 Feb 2019 13:45:49 +0000 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: <9d48ed71-8cbb-2b02-e992-b3656caf191f@aim.com> References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <9d48ed71-8cbb-2b02-e992-b3656caf191f@aim.com> Message-ID: <0b7915f9-641c-0b76-6c0e-ee99f1514bc3@llaisdy.com> Jesper's solution might be even better if you can handle that style.? Your code base becomes more like a set of rules rather than procedures.? You especially avoid the nested cases (which I didn't notice in your first example).? Nested cases really seem to enforce a procedural view of what's going on. Ivan On 13/02/2019 13:35, Donald Steven wrote: > Thanks! I'm gonna try that. > > My multilingual (using elements of Rust and c), heretical fantasy > (with apologies) is something like: > > makePanPositionL(Notes, Mode, L), > > ??? mut CurrentPanPosition = hd(L), > > ??? case Mode of > ??????? left2right -> > ??????????? case Notes of > ??????????????? < 6?? -> CurrentPanPosition += 8; > ??????????????? < 12? -> CurrentPanPosition += 4; > ??????????????? _else -> CurrentPanPosition += 2 > ??????????? end; > ??????? right2left -> > ??? ??? ??? ... > > On 2/13/2019 7.51 AM, Ivan Uemlianin wrote: >> This might be cheating but I would avoid the branching altogether if >> poss., eg: >> >> ??? PanPosition = CurrentPanPosition + notes_to_pad(Notes), >> ??? ... >> >> ??? notes_to_pad(N) where N 8; >> ??? notes_to_pad(N) where N < 12 -> 4; >> ??? notes_to_pad(_)????????????? -> 2. >> >> Also, I would avoid assignment inside a conditional, preferring eg: >> >> ??? PanPosition = >> ??????? case Notes of >> Few when Few < 6 -> CurrentPanPosition + 8; >> ??????????? ... >> >> Best wishes >> >> Ivan >> >> >> On 13/02/2019 12:40, Donald Steven wrote: >>> The code excerpt below shows two alternate ways of branching. I'd be >>> grateful for opinions on which represents best practices. Thanks. >>> >>> >>> Don >>> >>> >>> makePanPositionL(Notes, Mode, L), >>> >>> ??? CurrentPanPosition = hd(L), >>> >>> ??? case Mode of >>> >>> ??????? left2right -> >>> ??????????? if >>> ??????????????? Notes < 6 -> PanPosition = CurrentPanPosition + 8; >>> ??????????????? Notes < 12 -> PanPosition = CurrentPanPosition + 4; >>> ??????????????? true -> PanPosition = CurrentPanPosition + 2 >>> ??????????? end; >>> >>> ??????? left2right -> >>> ??????????? case Notes of >>> ??????????????? Few? when Few? < 6 -> PanPosition = >>> CurrentPanPosition + 8; >>> ??????????????? More when More < 12 -> PanPosition = >>> CurrentPanPosition + 4; >>> ??????????????? _Lots -> PanPosition = CurrentPanPosition + 2 >>> ??????????? end; >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Ymchwil a Datblygu Technoleg Lleferydd Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From t6sn7gt@REDACTED Wed Feb 13 16:02:07 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 13 Feb 2019 10:02:07 -0500 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> Message-ID: <35a6da28-6e22-783a-f4b0-828cfd13ce63@aim.com> Alas, that creates a different form of code sprawl, as in: makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<6->Cur+8; makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<12->Cur+4; makePanPositionL(Notes, left2right, [Cur|_]) ->Cur+2. makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<6->Cur-8; makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<12->Cur-4; makePanPositionL(Notes, right2left, [Cur|_]) ->Cur-2. makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) whenNotes<6->Cur+16; makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) whenNotes<12->Cur+8; makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) ->Cur+4. makePanPositionL(Notes, rapidlyright2left, [Cur|_]) whenNotes<6->Cur-16; makePanPositionL(Notes, rapidlyright2left, [Cur|_]) whenNotes<12->Cur-8; makePanPositionL(Notes, rapidlyright2left, [Cur|_]) ->Cur-4. On 2/13/2019 8.35 AM, Jesper Louis Andersen wrote: > On Wed, Feb 13, 2019 at 1:51 PM Ivan Uemlianin > wrote: > > This might be cheating but I would avoid the branching altogether if > poss., eg: > > ???? PanPosition = CurrentPanPosition + notes_to_pad(Notes), > ???? ... > > ???? notes_to_pad(N) where N 8; > ???? notes_to_pad(N) where N < 12 -> 4; > ???? notes_to_pad(_)????????????? -> 2. > > > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6? -> Cur + 8; > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> Cur + 4; > makePanPositionL(Notes, left2right, [Cur|_])???????????????? -> Cur + 2. > > You can just lift everything into a pattern match for this piece of > code, which is what I think I'd do. It makes the result far more > tabular, which in some cases is easier to read. There may be other > reasons in the code to not do this, but as it was written, I think I'd > pick the above as the solution. YMMV of course. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Wed Feb 13 16:06:36 2019 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 13 Feb 2019 15:06:36 +0000 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: <35a6da28-6e22-783a-f4b0-828cfd13ce63@aim.com> References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <35a6da28-6e22-783a-f4b0-828cfd13ce63@aim.com> Message-ID: Some people like it like that.? The repitition on the RHS makes me think there's room for more streamlining. On 13/02/2019 15:02, Donald Steven wrote: > Alas, that creates a different form of code sprawl, as in: > > makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<6->Cur+8; > makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<12->Cur+4; > makePanPositionL(Notes, left2right, [Cur|_]) ->Cur+2. > makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<6->Cur-8; > makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<12->Cur-4; > makePanPositionL(Notes, right2left, [Cur|_]) ->Cur-2. > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) whenNotes<6->Cur+16; > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) whenNotes<12->Cur+8; > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) ->Cur+4. > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) whenNotes<6->Cur-16; > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) whenNotes<12->Cur-8; > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) ->Cur-4. > > > On 2/13/2019 8.35 AM, Jesper Louis Andersen wrote: >> On Wed, Feb 13, 2019 at 1:51 PM Ivan Uemlianin > > wrote: >> >> This might be cheating but I would avoid the branching altogether if >> poss., eg: >> >> ???? PanPosition = CurrentPanPosition + notes_to_pad(Notes), >> ???? ... >> >> ???? notes_to_pad(N) where N 8; >> ???? notes_to_pad(N) where N < 12 -> 4; >> ???? notes_to_pad(_)????????????? -> 2. >> >> >> makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6? -> Cur + 8; >> makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> Cur + 4; >> makePanPositionL(Notes, left2right, [Cur|_])???????????????? -> Cur + 2. >> >> You can just lift everything into a pattern match for this piece of >> code, which is what I think I'd do. It makes the result far more >> tabular, which in some cases is easier to read. There may be other >> reasons in the code to not do this, but as it was written, I think >> I'd pick the above as the solution. YMMV of course. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Ymchwil a Datblygu Technoleg Lleferydd Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Feb 13 16:25:45 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Feb 2019 16:25:45 +0100 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <35a6da28-6e22-783a-f4b0-828cfd13ce63@aim.com> Message-ID: You can remove some of that repetition as well: notes(N) when N < 6 -> 8; notes(N) when N < 12 -> 4; notes(_N) -> 2. makePanPositionL(Notes, rapidlyleft2right, L) -> makePanPositionL(Notes, {rapidly, left2right}, L); makePanPositionL(Notes, rapidlyright2left, L) -> makePanPositionL(Notes, {rapidly, right2left}, L); makePanPositionL(Notes, {rapidly, Mode}, L) -> 2 * makePanPositionL(Notes, Mode, L); makePanPositionL(Notes, left2right, [Cur|_]) -> Cur + notes(Notes); makePanPositionL(Notes, right2left, [Cur|_]) -> Cur - notes(Notes). Though this is golfing, so go for the solution you think is most easy to read. One advantage of the above model is that it might be better if you are toying with altering the scaling. The scaling parameters are in one place in this code, so alteration is easier. Also, you can simplify by using an erlang term rather than an atom via the {rapidly, Mode} trick. Or perhaps just {scale, 2, Mode} and then interpret this as mpp(Notes, {scale, K, Mode}, L) -> K * mpp(Notes, Mode, L). On Wed, Feb 13, 2019 at 4:06 PM Ivan Uemlianin wrote: > Some people like it like that. The repitition on the RHS makes me think > there's room for more streamlining. > > On 13/02/2019 15:02, Donald Steven wrote: > > Alas, that creates a different form of code sprawl, as in: > > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6 -> Cur + 8; > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> Cur + 4; > makePanPositionL(Notes, left2right, [Cur|_]) -> Cur + 2. > makePanPositionL(Notes, right2left, [Cur|_]) when Notes < 6 -> Cur - 8; > makePanPositionL(Notes, right2left, [Cur|_]) when Notes < 12 -> Cur - 4; > makePanPositionL(Notes, right2left, [Cur|_]) -> Cur - 2. > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) when Notes < 6 -> Cur > + 16; > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) when Notes < 12 -> Cur > + 8; > makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) -> Cur + 4. > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) when Notes < 6 -> Cur > - 16; > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) when Notes < 12 -> Cur > - 8; > makePanPositionL(Notes, rapidlyright2left, [Cur|_]) -> Cur - 4. > > > On 2/13/2019 8.35 AM, Jesper Louis Andersen wrote: > > On Wed, Feb 13, 2019 at 1:51 PM Ivan Uemlianin wrote: > >> This might be cheating but I would avoid the branching altogether if >> poss., eg: >> >> PanPosition = CurrentPanPosition + notes_to_pad(Notes), >> ... >> >> notes_to_pad(N) where N < 6 -> 8; >> notes_to_pad(N) where N < 12 -> 4; >> notes_to_pad(_) -> 2. >> >> > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6 -> Cur + 8; > makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> Cur + 4; > makePanPositionL(Notes, left2right, [Cur|_]) -> Cur + 2. > > You can just lift everything into a pattern match for this piece of code, > which is what I think I'd do. It makes the result far more tabular, which > in some cases is easier to read. There may be other reasons in the code to > not do this, but as it was written, I think I'd pick the above as the > solution. YMMV of course. > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > > Ymchwil a Datblygu Technoleg Lleferydd > Speech Technology Research and Development > > ivan@REDACTED > @llaisdy > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Wed Feb 13 16:29:07 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 13 Feb 2019 10:29:07 -0500 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <35a6da28-6e22-783a-f4b0-828cfd13ce63@aim.com> Message-ID: Aha!!!!!!!!!? Thanks Jesper and Ivan, much appreciated. Don On 2/13/2019 10.25 AM, Jesper Louis Andersen wrote: > You can remove some of that repetition as well: > > notes(N) when N < 6? -> 8; > notes(N) when N < 12 -> 4; > notes(_N)??????????? -> 2. > > makePanPositionL(Notes, rapidlyleft2right, L) -> > makePanPositionL(Notes, {rapidly, left2right}, L); > makePanPositionL(Notes, rapidlyright2left, L) -> > makePanPositionL(Notes, {rapidly, right2left}, L); > makePanPositionL(Notes, {rapidly, Mode}, L)?? -> 2 * > makePanPositionL(Notes, Mode, L); > makePanPositionL(Notes, left2right, [Cur|_])? -> Cur + notes(Notes); > makePanPositionL(Notes, right2left, [Cur|_])? -> Cur - notes(Notes). > > Though this is golfing, so go for the solution you think is most easy > to read. One advantage of the above model is that it might be better > if you are toying with altering the scaling. The scaling parameters > are in one place in this code, so alteration is easier. Also, you can > simplify by using an erlang term rather than an atom via the {rapidly, > Mode} trick. Or perhaps just {scale, 2, Mode} and then interpret this as > > mpp(Notes, {scale, K, Mode}, L) -> > ? K * mpp(Notes, Mode, L). > > > On Wed, Feb 13, 2019 at 4:06 PM Ivan Uemlianin > wrote: > > Some people like it like that.? The repitition on the RHS makes me > think there's room for more streamlining. > > On 13/02/2019 15:02, Donald Steven wrote: >> Alas, that creates a different form of code sprawl, as in: >> >> makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<6->Cur+8; >> makePanPositionL(Notes, left2right, [Cur|_]) whenNotes<12->Cur+4; >> makePanPositionL(Notes, left2right, [Cur|_]) ->Cur+2. >> makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<6->Cur-8; >> makePanPositionL(Notes, right2left, [Cur|_]) whenNotes<12->Cur-4; >> makePanPositionL(Notes, right2left, [Cur|_]) ->Cur-2. >> makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) >> whenNotes<6->Cur+16; >> makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) >> whenNotes<12->Cur+8; >> makePanPositionL(Notes, rapidlyleft2right, [Cur|_]) ->Cur+4. >> makePanPositionL(Notes, rapidlyright2left, [Cur|_]) >> whenNotes<6->Cur-16; >> makePanPositionL(Notes, rapidlyright2left, [Cur|_]) >> whenNotes<12->Cur-8; >> makePanPositionL(Notes, rapidlyright2left, [Cur|_]) ->Cur-4. >> >> >> On 2/13/2019 8.35 AM, Jesper Louis Andersen wrote: >>> On Wed, Feb 13, 2019 at 1:51 PM Ivan Uemlianin >> > wrote: >>> >>> This might be cheating but I would avoid the branching >>> altogether if >>> poss., eg: >>> >>> ???? PanPosition = CurrentPanPosition + notes_to_pad(Notes), >>> ???? ... >>> >>> ???? notes_to_pad(N) where N 8; >>> ???? notes_to_pad(N) where N < 12 -> 4; >>> ???? notes_to_pad(_)????????????? -> 2. >>> >>> >>> makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 6? -> >>> Cur + 8; >>> makePanPositionL(Notes, left2right, [Cur|_]) when Notes < 12 -> >>> Cur + 4; >>> makePanPositionL(Notes, left2right, [Cur|_])???????????????? -> >>> Cur + 2. >>> >>> You can just lift everything into a pattern match for this piece >>> of code, which is what I think I'd do. It makes the result far >>> more tabular, which in some cases is easier to read. There may >>> be other reasons in the code to not do this, but as it was >>> written, I think I'd pick the above as the solution. YMMV of course. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > > Ymchwil a Datblygu Technoleg Lleferydd > Speech Technology Research and Development > > ivan@REDACTED > @llaisdy > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > > > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From empro2@REDACTED Wed Feb 13 17:04:52 2019 From: empro2@REDACTED (empro2@REDACTED) Date: Wed, 13 Feb 2019 17:04:52 +0100 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: <0b7915f9-641c-0b76-6c0e-ee99f1514bc3@llaisdy.com> References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <9d48ed71-8cbb-2b02-e992-b3656caf191f@aim.com> <0b7915f9-641c-0b76-6c0e-ee99f1514bc3@llaisdy.com> Message-ID: <20190213170452.4e2e95a0@raspy> On Wed, 13 Feb 2019 13:45:49 +0000 Ivan Uemlianin wrote: > example).? Nested cases really seem to enforce a > procedural view of what's going on. Is it not the other way around: > On 13/02/2019 13:35, Donald Steven wrote: > > My multilingual (using elements of Rust and c), > > heretical fantasy (with apologies) is something like: the procedural thinking causing the nesting and distracting from, even preventing the calculation of values? > > makePanPositionL(Notes, Mode, L), Why "make" it and not calculate pan_position(...)? Just as it is simply math:pi/0, not math:make_pi, get_pi, approximate_pi, .../0 I am still "forgetting" the "naming conventions" of procedural code - and that includes the "hidden procedures" of object orientation. I even avoid the Smalltalk instead of Algol identifiers (especially as fonts tend to have short capitals (MlKkWf0Od) that are good for all-capital words in writing but do not really stand out enough for noInterWordSpaceWordSeparation). In the last few years more and more Smalltalk identifiers appeared in the docs (and the code), and though I am most certainly a worse programmer (in general, not Erlang alone) than anyone who has contributed to Erlang I would have s/maps:get/maps:value/ (or maps:val, or in that context perhaps even maps:v) and probably s/maps:new/maps:empty/, to solely describe an empty map and avoiding the hint at the making of a new one. I am hoping the value description instead of action description helps me switch from procedural and OO to more functional thinking. The "branching" in the original post immediately made me think of branching mnemonics (BCS, BCC, ...) My! am I old :-) Michael -- Normality is merely a question of quantity, not of quality. From ivan@REDACTED Wed Feb 13 17:11:10 2019 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 13 Feb 2019 16:11:10 +0000 Subject: [erlang-questions] Best practices -- conditional statements In-Reply-To: <20190213170452.4e2e95a0@raspy> References: <9dd5e96c-d3fc-ba65-251e-12f12d4327ed@aim.com> <9d48ed71-8cbb-2b02-e992-b3656caf191f@aim.com> <0b7915f9-641c-0b76-6c0e-ee99f1514bc3@llaisdy.com> <20190213170452.4e2e95a0@raspy> Message-ID: <3dc8f86d-a2a8-b6f6-bf8c-166a4df5a025@llaisdy.com> Certainly code/language<->thinking is a vicious/virtuous circle.? Interesting point about get & set. On 13/02/2019 16:04, empro2@REDACTED wrote: > On Wed, 13 Feb 2019 13:45:49 +0000 > Ivan Uemlianin wrote: > >> example).? Nested cases really seem to enforce a >> procedural view of what's going on. > Is it not the other way around: > >> On 13/02/2019 13:35, Donald Steven wrote: >>> My multilingual (using elements of Rust and c), >>> heretical fantasy (with apologies) is something like: > the procedural thinking causing the nesting and > distracting from, even preventing the calculation of values? > >>> makePanPositionL(Notes, Mode, L), > Why "make" it and not calculate pan_position(...)? > Just as it is simply math:pi/0, not math:make_pi, get_pi, > approximate_pi, .../0 > > I am still "forgetting" the "naming conventions" of > procedural code - and that includes the "hidden procedures" > of object orientation. I even avoid the Smalltalk instead > of Algol identifiers (especially as fonts tend to have > short capitals (MlKkWf0Od) that are good for all-capital > words in writing but do not really stand out enough for > noInterWordSpaceWordSeparation). > > In the last few years more and more Smalltalk identifiers > appeared in the docs (and the code), and though I am most > certainly a worse programmer (in general, not Erlang alone) > than anyone who has contributed to Erlang I would have > s/maps:get/maps:value/ > (or maps:val, or in that context perhaps even maps:v) > and probably s/maps:new/maps:empty/, to solely describe an > empty map and avoiding the hint at the making of a new one. > > I am hoping the value description instead of action > description helps me switch from procedural and OO to more > functional thinking. > > The "branching" in the original post immediately made me > think of branching mnemonics (BCS, BCC, ...) > > My! am I old :-) > > Michael > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Ymchwil a Datblygu Technoleg Lleferydd Speech Technology Research and Development ivan@REDACTED @llaisdy llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From icfp.publicity@REDACTED Wed Feb 13 19:14:53 2019 From: icfp.publicity@REDACTED (Sam Tobin-Hochstadt) Date: Wed, 13 Feb 2019 13:14:53 -0500 Subject: [erlang-questions] Third Call for Papers: PACMPL issue ICFP 2019 Message-ID: <5c645e9d66518_1b92abe6f8c25b43382f@hermes.mail> PACMPL Volume 3, Issue ICFP 2019 Call for Papers accepted papers to be invited for presentation at The 24th ACM SIGPLAN International Conference on Functional Programming Berlin, Germany http://icfp19.sigplan.org/ ### Important dates Submissions due: 1 March 2019 (Friday) Anywhere on Earth https://icfp19.hotcrp.com Author response: 16 April (Tuesday) - 18 Apri (Friday) 14:00 UTC Notification: 3 May (Friday) Final copy due: 22 June (Saturday) Conference: 18 August (Sunday) - 23 August (Friday) ### About PACMPL Proceedings of the ACM on Programming Languages (PACMPL ) is a Gold Open Access journal publishing research on all aspects of programming languages, from design to implementation and from mathematical formalisms to empirical studies. Each issue of the journal is devoted to a particular subject area within programming languages and will be announced through publicized Calls for Papers, like this one. ### Scope [PACMPL](https://pacmpl.acm.org/) issue ICFP 2019 seeks original papers on the art and science of functional programming. Submissions are invited on all topics from principles to practice, from foundations to features, and from abstraction to application. The scope includes all languages that encourage functional programming, including both purely applicative and imperative languages, as well as languages with objects, concurrency, or parallelism. Topics of interest include (but are not limited to): * *Language Design*: concurrency, parallelism, and distribution; modules; components and composition; metaprogramming; type systems; interoperability; domain-specific languages; and relations to imperative, object-oriented, or logic programming. * *Implementation*: abstract machines; virtual machines; interpretation; compilation; compile-time and run-time optimization; garbage collection and memory management; multi-threading; exploiting parallel hardware; interfaces to foreign functions, services, components, or low-level machine resources. * *Software-Development Techniques*: algorithms and data structures; design patterns; specification; verification; validation; proof assistants; debugging; testing; tracing; profiling. * *Foundations*: formal semantics; lambda calculus; rewriting; type theory; monads; continuations; control; state; effects; program verification; dependent types. * *Analysis and Transformation*: control-flow; data-flow; abstract interpretation; partial evaluation; program calculation. * *Applications*: symbolic computing; formal-methods tools; artificial intelligence; systems programming; distributed-systems and web programming; hardware design; databases; XML processing; scientific and numerical computing; graphical user interfaces; multimedia and 3D graphics programming; scripting; system administration; security. * *Education*: teaching introductory programming; parallel programming; mathematical proof; algebra. Submissions will be evaluated according to their relevance, correctness, significance, originality, and clarity. Each submission should explain its contributions in both general and technical terms, clearly identifying what has been accomplished, explaining why it is significant, and comparing it with previous work. The technical content should be accessible to a broad audience. PACMPL issue ICFP 2019 also welcomes submissions in two separate categories — Functional Pearls and Experience Reports — that must be marked as such at the time of submission and that need not report original research results. Detailed guidelines on both categories are given at the end of this call. Please contact the principal editor if you have questions or are concerned about the appropriateness of a topic. ### Preparation of submissions **Deadline**: The deadline for submissions is **Friday, March 1, 2019**, Anywhere on Earth (). This deadline will be strictly enforced. **Formatting**: Submissions must be in PDF format, printable in black and white on US Letter sized paper, and interpretable by common PDF tools. All submissions must adhere to the "ACM Small" template that is available (in both LaTeX and Word formats) from . For authors using LaTeX, a lighter-weight package, including only the essential files, is available from . There is a limit of **25 pages for a full paper or Functional Pearl** and **12 pages for an Experience Report**; in either case, the bibliography will not be counted against these limits. Submissions that exceed the page limits or, for other reasons, do not meet the requirements for formatting, will be summarily rejected. Supplementary material can and should be **separately** submitted (see below). See also PACMPL's Information and Guidelines for Authors at . **Submission**: Submissions will be accepted at Improved versions of a paper may be submitted at any point before the submission deadline using the same web interface. **Author Response Period**: Authors will have a 72-hour period, starting at 14:00 UTC on **Tuesday, April 16, 2019**, to read reviews and respond to them. **Supplementary Material**: Authors have the option to attach supplementary material to a submission, on the understanding that reviewers may choose not to look at it. This supplementary material should **not** be submitted as part of the main document; instead, it should be uploaded as a **separate** PDF document or tarball. Supplementary material should be uploaded **at submission time**, not by providing a URL in the paper that points to an external repository. Authors are free to upload both anonymized and non-anonymized supplementary material. Anonymized supplementary material will be visible to reviewers immediately; non-anonymized supplementary material will be revealed to reviewers only after they have submitted their review of the paper and learned the identity of the author(s). **Authorship Policies**: All submissions are expected to comply with the ACM Policies for Authorship that are detailed at . **Republication Policies**: Each submission must adhere to SIGPLAN's republication policy, as explained on the web at . **Resubmitted Papers**: Authors who submit a revised version of a paper that has previously been rejected by another conference have the option to attach an annotated copy of the reviews of their previous submission(s), explaining how they have addressed these previous reviews in the present submission. If a reviewer identifies him/herself as a reviewer of this previous submission and wishes to see how his/her comments have been addressed, the principal editor will communicate to this reviewer the annotated copy of his/her previous review. Otherwise, no reviewer will read the annotated copies of the previous reviews. ### Review Process This section outlines the two-stage process with lightweight double-blind reviewing that will be used to select papers for PACMPL issue ICFP 2019. We anticipate that there will be a need to clarify and expand on this process, and we will maintain a list of frequently asked questions and answers on the conference website to address common concerns. **PACMPL issue ICFP 2019 will employ a two-stage review process.** The first stage in the review process will assess submitted papers using the criteria stated above and will allow for feedback and input on initial reviews through the author response period mentioned previously. At the review meeting, a set of papers will be conditionally accepted and all other papers will be rejected. Authors will be notified of these decisions on **May 3, 2019**. Authors of conditionally accepted papers will be provided with committee reviews (just as in previous conferences) along with a set of mandatory revisions. After four weeks (May 31, 2019), the authors will provide a second submission. The second and final reviewing phase assesses whether the mandatory revisions have been adequately addressed by the authors and thereby determines the final accept/reject status of the paper. The intent and expectation is that the mandatory revisions can be addressed within four weeks and hence that conditionally accepted papers will in general be accepted in the second phase. The second submission should clearly identify how the mandatory revisions were addressed. To that end, the second submission must be accompanied by a cover letter mapping each mandatory revision request to specific parts of the paper. The cover letter will facilitate a quick second review, allowing for confirmation of final acceptance within two weeks. Conversely, the absence of a cover letter will be grounds for the paper?s rejection. **PACMPL issue ICFP 2019 will employ a lightweight double-blind reviewing process.** To facilitate this, submitted papers must adhere to two rules: 1. **author names and institutions must be omitted**, and 2. **references to authors' own related work should be in the third person** (e.g., not "We build on our previous work ..." but rather "We build on the work of ..."). The purpose of this process is to help the reviewers come to an initial judgement about the paper without bias, not to make it impossible for them to discover the authors if they were to try. Nothing should be done in the name of anonymity that weakens the submission or makes the job of reviewing the paper more difficult (e.g., important background references should not be omitted or anonymized). In addition, authors should feel free to disseminate their ideas or draft versions of their paper as they normally would. For instance, authors may post drafts of their papers on the web or give talks on their research ideas. ### Information for Authors of Accepted Papers * As a condition of acceptance, final versions of all papers must adhere to the new ACM Small format. The page limit for the final versions of papers will be increased by two pages to help authors respond to reviewer comments and mandatory revisions: **27 pages plus bibliography for a regular paper or Functional Pearl, 14 pages plus bibliography for an Experience Report**. * Authors of accepted submissions will be required to agree to one of the three ACM licensing options: open access on payment of a fee (**recommended**, and SIGPLAN can cover the cost as described next); copyright transfer to ACM; or retaining copyright but granting ACM exclusive publication rights. Further information about ACM author rights is available from . * PACMPL is a Gold Open Access journal. It will be archived in ACM?s Digital Library, but no membership or fee is required for access. Gold Open Access has been made possible by generous funding through ACM SIGPLAN, which will cover all open access costs in the event authors cannot. Authors who can cover the costs may do so by paying an Article Processing Charge (APC). PACMPL, SIGPLAN, and ACM Headquarters are committed to exploring routes to making Gold Open Access publication both affordable and sustainable. * ACM offers authors a range of copyright options, one of which is Creative Commons CC-BY publication; this is the option recommended by the PACMPL editorial board. A reasoned argument in favour of this option can be found in the article [Why CC-BY?](https://oaspa.org/why-cc-by/) published by OASPA, the Open Access Scholarly Publishers Association. * We intend that the papers will be freely available for download from the ACM Digital Library in perpetuity via the OpenTOC mechanism. * ACM Author-Izer is a unique service that enables ACM authors to generate and post links on either their home page or institutional repository for visitors to download the definitive version of their articles from the ACM Digital Library at no charge. Downloads through Author-Izer links are captured in official ACM statistics, improving the accuracy of usage and impact measurements. Consistently linking to the definitive version of an ACM article should reduce user confusion over article versioning. After an article has been published and assigned to the appropriate ACM Author Profile pages, authors should visit to learn how to create links for free downloads from the ACM DL. * At least one author of each accepted submissions will be expected to attend and present their paper at the conference. The schedule for presentations will be determined and shared with authors after the full program has been selected. Presentations will be videotaped and released online if the presenter consents. * The official publication date is the date the papers are made available in the ACM Digital Library. This date may be up to *two weeks prior* to the first day of the conference. The official publication date affects the deadline for any patent filings related to published work. ### Artifact Evaluation Authors of papers that are conditionally accepted in the first phase of the review process will be encouraged (but not required) to submit supporting materials for Artifact Evaluation. These items will then be reviewed by an Artifact Evaluation Committee, separate from the paper Review Committee, whose task is to assess how the artifacts support the work described in the associated paper. Papers that go through the Artifact Evaluation process successfully will receive a seal of approval printed on the papers themselves. Authors of accepted papers will be encouraged to make the supporting materials publicly available upon publication of the papers, for example, by including them as "source materials" in the ACM Digital Library. An additional seal will mark papers whose artifacts are made available, as outlined in the ACM guidelines for artifact badging. Participation in Artifact Evaluation is voluntary and will not influence the final decision regarding paper acceptance. ### Special categories of papers In addition to research papers, PACMPL issue ICFP solicits two kinds of papers that do not require original research contributions: Functional Pearls, which are full papers, and Experience Reports, which are limited to half the length of a full paper. Authors submitting such papers should consider the following guidelines. #### Functional Pearls A Functional Pearl is an elegant essay about something related to functional programming. Examples include, but are not limited to: * a new and thought-provoking way of looking at an old idea * an instructive example of program calculation or proof * a nifty presentation of an old or new data structure * an interesting application of functional programming techniques * a novel use or exposition of functional programming in the classroom While pearls often demonstrate an idea through the development of a short program, there is no requirement or expectation that they do so. Thus, they encompass the notions of theoretical and educational pearls. Functional Pearls are valued as highly and judged as rigorously as ordinary papers, but using somewhat different criteria. In particular, a pearl is not required to report original research, but, it should be concise, instructive, and entertaining. A pearl is likely to be rejected if its readers get bored, if the material gets too complicated, if too much specialized knowledge is needed, or if the writing is inelegant. The key to writing a good pearl is polishing. A submission that is intended to be treated as a pearl must be marked as such on the submission web page, and should contain the words "Functional Pearl" somewhere in its title or subtitle. These steps will alert reviewers to use the appropriate evaluation criteria. Pearls will be combined with ordinary papers, however, for the purpose of computing the conference's acceptance rate. #### Experience Reports The purpose of an Experience Report is to help create a body of published, refereed, citable evidence that functional programming really works — or to describe what obstacles prevent it from working. Possible topics for an Experience Report include, but are not limited to: * insights gained from real-world projects using functional programming * comparison of functional programming with conventional programming in the context of an industrial project or a university curriculum * project-management, business, or legal issues encountered when using functional programming in a real-world project * curricular issues encountered when using functional programming in education * real-world constraints that created special challenges for an implementation of a functional language or for functional programming in general An Experience Report is distinguished from a normal PACMPL issue ICFP paper by its title, by its length, and by the criteria used to evaluate it. * Both in the papers and in any citations, the title of each accepted Experience Report must end with the words "(Experience Report)" in parentheses. The acceptance rate for Experience Reports will be computed and reported separately from the rate for ordinary papers. * Experience Report submissions can be at most 12 pages long, excluding bibliography. * Each accepted Experience Report will be presented at the conference, but depending on the number of Experience Reports and regular papers accepted, authors of Experience reports may be asked to give shorter talks. * Because the purpose of Experience Reports is to enable our community to accumulate a body of evidence about the efficacy of functional programming, an acceptable Experience Report need not add to the body of knowledge of the functional-programming community by presenting novel results or conclusions. It is sufficient if the Report states a clear thesis and provides supporting evidence. The thesis must be relevant to ICFP, but it need not be novel. The review committee will accept or reject Experience Reports based on whether they judge the evidence to be convincing. Anecdotal evidence will be acceptable provided it is well argued and the author explains what efforts were made to gather as much evidence as possible. Typically, more convincing evidence is obtained from papers which show how functional programming was used than from papers which only say that functional programming was used. The most convincing evidence often includes comparisons of situations before and after the introduction or discontinuation of functional programming. Evidence drawn from a single person's experience may be sufficient, but more weight will be given to evidence drawn from the experience of groups of people. An Experience Report should be short and to the point: it should make a claim about how well functional programming worked on a particular project and why, and produce evidence to substantiate this claim. If functional programming worked in this case in the same ways it has worked for others, the paper need only summarize the results — the main part of the paper should discuss how well it worked and in what context. Most readers will not want to know all the details of the project and its implementation, but the paper should characterize the project and its context well enough so that readers can judge to what degree this experience is relevant to their own projects. The paper should take care to highlight any unusual aspects of the project. Specifics about the project are more valuable than generalities about functional programming; for example, it is more valuable to say that the team delivered its software a month ahead of schedule than it is to say that functional programming made the team more productive. If the paper not only describes experience but also presents new technical results, or if the experience refutes cherished beliefs of the functional-programming community, it may be better to submit it as a full paper, which will be judged by the usual criteria of novelty, originality, and relevance. The principal editor will be happy to advise on any concerns about which category to submit to. ### ICFP Organizers General Chair: Derek Dreyer (MPI-SWS, Germany) Artifact Evaluation Co-Chairs: Simon Marlow (Facebook, UK) Industrial Relations Chair: Alan Jeffrey (Mozilla Research, USA) Programming Contest Organiser: Ilya Sergey (Yale-NUS College, Singapore) Publicity and Web Chair: Sam Tobin-Hochstadt (Indiana University, USA) Student Research Competition Chair: William J. Bowman (University of British Columbia, Canada) Workshops Co-Chair: Christophe Scholliers (Universiteit Gent, Belgium) Jennifer Hackett (University of Nottingham, UK) Conference Manager: Annabel Satin (P.C.K.) ### PACMPL Volume 3, Issue ICFP 2019 Principal Editor: Fran?ois Pottier (Inria, France) Review Committee: Lennart Beringer (Princeton University, United States) Joachim Breitner (DFINITY Foundation, Germany) Laura M. Castro (University of A Coru?a, Spain) Ezgi ?i?ek (Facebook London, United Kingdom) Pierre-Evariste Dagand (LIP6/CNRS, France) Christos Dimoulas (Northwestern University, United States) Jacques-Henri Jourdan (CNRS, LRI, Universit? Paris-Sud, France) Andrew Kennedy (Facebook London, United Kingdom) Daan Leijen (Microsoft Research, United States) Kazutaka Matsuda (Tohoku University, Japan) Bruno C. d. S. Oliveira (University of Hong Kong, China) Klaus Ostermann (University of T?bingen, Germany) Jennifer Paykin (Galois, United States) Frank Pfenning (Carnegie Mellon University, USA) Mike Rainey (Indiana University, USA) Chung-chieh Shan (Indiana University, USA) Sam Staton (University of Oxford, UK) Pierre-Yves Strub (Ecole Polytechnique, France) German Vidal (Universitat Politecnica de Valencia, Spain) External Review Committee: Michael D. Adams (University of Utah, USA) Robert Atkey (University of Strathclyde, IK) Sheng Chen (University of Louisiana at Lafayette, USA) James Cheney (University of Edinburgh, UK) Adam Chlipala (Massachusetts Institute of Technology, USA) Evelyne Contejean (LRI, Universit? Paris-Sud, France) Germ?n Andr?s Delbianco (IRIF, Universit? Paris Diderot, France) Dominique Devriese (Vrije Universiteit Brussel, Belgium) Richard A. Eisenberg (Bryn Mawr College, USA) Conal Elliott (Target, USA) Sebastian Erdweg (Delft University of Technology, Netherlands) Michael Greenberg (Pomona College, USA) Adrien Guatto (IRIF, Universit? Paris Diderot, France) Jennifer Hackett (University of Nottingham, UK) Troels Henriksen (University of Copenhagen, Denmark) Chung-Kil Hur (Seoul National University, Republic of Korea) Roberto Ierusalimschy (PUC-Rio, Brazil) Ranjit Jhala (University of California, San Diego, USA) Ralf Jung (MPI-SWS, Germany) Ohad Kammar (University of Oxford, UK) Oleg Kiselyov (Tohoku University, Japan) Hsiang-Shang ?Josh? Ko (National Institute of Informatics, Japan) Ond?ej Lhot?k (University of Waterloo, Canada) Dan Licata (Wesleyan University, USA) Geoffrey Mainland (Drexel University, USA) Simon Marlow (Facebook, UK) Akimasa Morihata (University of Tokyo, Japan) Shin-Cheng Mu (Academia Sinica, Taiwan) Guillaume Munch-Maccagnoni (Inria, France) Kim Nguy?n (University of Paris-Sud, France) Ulf Norell (Gothenburg University, Sweden) Atsushi Ohori (Tohoku University, Japan) Rex Page (University of Oklahoma, USA) Zoe Paraskevopoulou (Princeton University, USA) Nadia Polikarpova (University of California, San Diego, USA) Jonathan Protzenko (Microsoft Research, USA) Tiark Rompf (Purdue University, USA) Andreas Rossberg (Dfinity, Germany) KC Sivaramakrishnan (University of Cambridge, UI) Nicholas Smallbone (Chalmers University of Technology, Sweden) Matthieu Sozeau (Inria, France) Sandro Stucki (Chalmers | University of Gothenburg, Sweden) Don Syme (Microsoft, UK) Zachary Tatlock (University of Washington, USA) Sam Tobin-Hochstadt (Indiana University, USA) Takeshi Tsukada (University of Tokyo, Japan) Tarmo Uustalu (Reykjavik University, Iceland) Benoit Valiron (LRI, CentraleSupelec, Univ. Paris Saclay, France) Daniel Winograd-Cort (University of Pennsylvania, USA) Nicolas Wu (University of Bristol, UK) From icfp.publicity@REDACTED Wed Feb 13 19:29:15 2019 From: icfp.publicity@REDACTED (Sam Tobin-Hochstadt) Date: Wed, 13 Feb 2019 13:29:15 -0500 Subject: [erlang-questions] Call for Submissions: ICFP Student Research Competition Message-ID: <5c6461fbc08ba_77d2b0585e545c49988@hermes.mail> ICFP 2019 Student Research Competition Call for Submissions ICFP invites students to participate in the Student Research Competition in order to present their research and get feedback from prominent members of the programming language research community. Please submit your extended abstracts through the submission website. ### Important dates Submissions due: 14 Jun 2019 (Friday) https://icfp19src.hotcrp.com Notification: 28 Jun 2019 (Friday) Conference: 18 August (Sunday) - 23 August (Friday) Each submission (referred to as "abstract" below) should include the student author?s name and e-mail address; institutional affiliation; research advisor?s name; ACM student member number; category (undergraduate or graduate); research title; and an extended abstract addressing the following: * Problem and Motivation: Clearly state the problem being addressed and explain the reasons for seeking a solution to this problem. * Background and Related Work: Describe the specialized (but pertinent) background necessary to appreciate the work in the context of ICFP areas of interest. Include references to the literature where appropriate, and briefly explain where your work departs from that done by others. * Approach and Uniqueness: Describe your approach in addressing the problem and clearly state how your approach is novel. * Results and Contributions: Clearly show how the results of your work contribute to programming language design and implementation in particular and to computer science in general; explain the significance of those results. * Submissions must be original research that is not already published at ICFP or another conference or journal. One of the goals of the SRC is to give students feedback on ongoing, unpublished work. Furthermore, the abstract must be authored solely by the student. If the work is collaborative with others and*or part of a larger group project, the abstract should make clear what the student?s role was and should focus on that portion of the work. * Formatting: Submissions must be in PDF format, printable in black and white on US Letter sized paper, and interpretable by common PDF tools. All submissions must adhere to the "ACM Small" template that is available (in both LaTeX and Word formats) from https://www.acm.org/publications/authors/submissions. For authors using LaTeX, a lighter-weight package, including only the essential files, is available from http://sigplan.org/Resources/Author/#acmart-format. The submission must not exceed 3 pages in PDF format. Reference lists do not count towards the 3-page limit. Further information is available at the ICFP SRC website: https://icfp19.sigplan.org/track/icfp-2019-Student-Research-Competition ICFP Student Research Competition Chair: William J. Bowman (University of British Columbia) From block.rxckin.beats@REDACTED Thu Feb 14 06:17:09 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 14 Feb 2019 14:17:09 +0900 Subject: [erlang-questions] terminate supervisor from it's child process Message-ID: Hi I'm writing udp server with supervision tree like below. ``` (s1for1) (1for1) server_sup - udp_sup - |- udp_worker |- dtls_worker |- stun_worker |- srtp_worker ``` 1. app opening UDP socket 2. server_sup spawn udp_sup process tree with that udp socket 3. udp_worker receives every packet from socket 4. udp_woker passing packet to other worker based on first byte demultiplexing Question: when timeout in udp_worker happen, I wanna terminate all processes in udp_sup (including udp_sup) so I tried shutdown udp_sup with calling `server_sup:terminate_child(pid)` that's terminates but rises SUPERVISOR REPORT with shutdown_error. I also tried one_for_all for udp_sup's childspec but I don't want to restart children because UDP socket already timeouted. is there any way to terminate supervisor from it's child process ? or any other recommended process architecture for UDP server ? thanks Jxck -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Thu Feb 14 07:53:09 2019 From: vances@REDACTED (Vance Shipley) Date: Thu, 14 Feb 2019 12:23:09 +0530 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: On Thu, Feb 14, 2019 at 11:52 AM block.rxckin.beats@REDACTED wrote: > I'm writing udp server with supervision tree like below. ... > (s1for1) (1for1) > server_sup - udp_sup - |- udp_worker > |- dtls_worker > |- stun_worker > |- srtp_worker > is there any way to terminate supervisor from it's child process ? Yes, exit the child and have the supervisor's restart strategy do it's job. > or any other recommended process architecture for UDP server ? You could set the restart strategy of the above udp_sup to be 'rest_for_one' which would kill all the other workers when it died. However that would have the side effect of kill stun_worker and srtp_worker when dtls_worker exited. The right way is to have a seperate supervisor for [dtls|stun|srtp]_worker as a sibling to udp_worker. -- -Vance From vances@REDACTED Thu Feb 14 09:59:45 2019 From: vances@REDACTED (Vance Shipley) Date: Thu, 14 Feb 2019 14:29:45 +0530 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: On Thu, Feb 14, 2019 at 1:47 PM block.rxckin.beats@REDACTED wrote: > I think you mean like this. > > udp_sup - udp_worker > |- dtls_sup - dtls_woker > .... (same for stun,srtp) > > if udp_worker be temporary, udp_sup will ignore, > if udp_wokrer be permanent and intensity = 0, period = 1 > udp_sup will terminate with error because of restart error (even if I don't intent to restart) > > > if I misreading something, please mention me. No, I meant: udp_sup -> udp_worker -> proto_sup -> dtls_woker -> stun_worker -> srtp_worker -- -Vance From block.rxckin.beats@REDACTED Thu Feb 14 09:16:51 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 14 Feb 2019 17:16:51 +0900 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: Hi Vance thanks for your advice ! > However that would have the side effect of kill stun_worker and srtp_worker when dtls_worker exited. that's expected, because they share same socket which timeouted. > Yes, exit the child and have the supervisor's restart strategy do it's job. > You could set the restart strategy of the above udp_sup to be 'rest_for_one' which would kill all the other workers when it died. I don't wanna restart child workers, so make it `restart => temporary` in childspec. but in that case, if udp_woker terminates, udp_sup doesn't terminate. only igore it. make if make `restart => permanent` and sup flag like below ``` SupFlags = #{ strategy => rest_for_one, # same for one_for_all intensity => 0, period => 1 }, ``` that's terminates supervisor with ErrorReport with reached_max_restart_intensity. it's terminates all supervision tree include udp_sup as I expected but I don't intent to restart, only shutdown like temporary :( > The right way is to have a seperate supervisor for [dtls|stun|srtp]_worker as a sibling to udp_worker. I think you mean like this. udp_sup - udp_worker |- dtls_sup - dtls_woker .... (same for stun,srtp) if udp_worker be temporary, udp_sup will ignore, if udp_wokrer be permanent and intensity = 0, period = 1 udp_sup will terminate with error because of restart error (even if I don't intent to restart) if I misreading something, please mention me. thanks Jxck 2019?2?14?(?) 15:53 Vance Shipley : > On Thu, Feb 14, 2019 at 11:52 AM block.rxckin.beats@REDACTED > wrote: > > I'm writing udp server with supervision tree like below. > ... > > (s1for1) (1for1) > > server_sup - udp_sup - |- udp_worker > > |- dtls_worker > > |- stun_worker > > |- srtp_worker > > > is there any way to terminate supervisor from it's child process ? > > Yes, exit the child and have the supervisor's restart strategy do it's job. > > > or any other recommended process architecture for UDP server ? > > You could set the restart strategy of the above udp_sup to be > 'rest_for_one' which would kill all the other workers when it died. > However that would have the side effect of kill stun_worker and > srtp_worker when dtls_worker exited. > > The right way is to have a seperate supervisor for > [dtls|stun|srtp]_worker as a sibling to udp_worker. > > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From block.rxckin.beats@REDACTED Thu Feb 14 10:09:53 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 14 Feb 2019 18:09:53 +0900 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: > No, I meant: > udp_sup > -> udp_worker > -> proto_sup > -> dtls_woker > -> stun_worker > -> srtp_worker in that case, udp_worker can terminate proto_sup but who terminate udp_sup ? here is my PoC https://github.com/Jxck/udp_server 2019?2?14?(?) 17:59 Vance Shipley : > On Thu, Feb 14, 2019 at 1:47 PM block.rxckin.beats@REDACTED > wrote: > > I think you mean like this. > > > > udp_sup - udp_worker > > |- dtls_sup - dtls_woker > > .... (same for stun,srtp) > > > > if udp_worker be temporary, udp_sup will ignore, > > if udp_wokrer be permanent and intensity = 0, period = 1 > > udp_sup will terminate with error because of restart error (even if I > don't intent to restart) > > > > > > if I misreading something, please mention me. > > No, I meant: > > udp_sup > -> udp_worker > -> proto_sup > -> dtls_woker > -> stun_worker > -> srtp_worker > > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Feb 14 11:59:42 2019 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 14 Feb 2019 11:59:42 +0100 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: <20170414130016.GB61497@ferdmbp.local> References: <20170414130016.GB61497@ferdmbp.local> Message-ID: For existing beam code that relies on tuple calls (parameterized modules), there is now an upgrade path that allows you to bring the old code up on OTP 21 before you recompile with the +tuple_calls compiler option (which does not exist in OTP 20 and earlier): https://github.com/erlang/otp/pull/2113 For current users of OTP 21, this changes nothing (i.e., there is no tuple dispatch) unless you explicitly pass the new '+ztma true' flag. (The flag will not be available in OTP 22, so this is just for the transition.) /Richard Den fre 14 apr. 2017 kl 15:00 skrev Fred Hebert : > On 04/14, Jos? Valim wrote: > >This behaviour is considered by most in the community to be undesired > >and confusing, as it obfuscates the meaning of the code and adds > >indirection. > > > >Please let me know if there are other options available, > > > >I will be glad to send patches and implement the required parse-transforms > >if this is accepted by the OTP team. > > > > One thing to consider is that this code is a remnant of parametrized > modules, which have been removed back in 2012. See > https://web-beta.archive.org/web/20121019180935/http://www.erlang.org/news/35 > > > > To allow for source code compatibility, the board decides to only > > remove the syntactic support for parameterized modules. This means: > > > > * Apply with tuple modules will be retained, supported and properly > documented. > > > > [...] > > > > * The parser will accept the syntax for implementation of > > parameterized modules [...] > > > > * A parse_transform handling source code implementing a parameterized > > module will be published in the public domain, free to use and > > include in any package needing it. It will however not be part of OTP. > > As it uses supported and documented functionality, it will however > > be expected to work for forthcoming releases as well, or at least > > to be possible to adopt to forthcoming releases. > > This is to say the OTP team made a commitment to supporting tuple calls > as a way to remove parametrized modules and keep the code working. > > Removing tuple calls must at the very least ensure a seamless transition > to keep working with the parametrized module parse transforms, and > moreover have the guarantee that it will "be expected to work for > forthcoming releases" because it "uses supported and documented > functionality". > > Regards, > Fred. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Thu Feb 14 12:22:02 2019 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 14 Feb 2019 12:22:02 +0100 Subject: [erlang-questions] Proposal to remove tuple dispatches from Erlang In-Reply-To: References: <20170414130016.GB61497@ferdmbp.local> Message-ID: On Thu, Feb 14, 2019 at 11:59 AM Richard Carlsson wrote: > > For existing beam code that relies on tuple calls (parameterized modules), there is now an upgrade path that allows you to bring the old code up on OTP 21 before you recompile with the +tuple_calls compiler option (which does not exist in OTP 20 and earlier): > https://github.com/erlang/otp/pull/2113 > > For current users of OTP 21, this changes nothing (i.e., there is no tuple dispatch) unless you explicitly pass the new '+ztma true' flag. > (The flag will not be available in OTP 22, so this is just for the transition.) Here is the pull request that removes the emulator option in OTP 22: https://github.com/erlang/otp/pull/2142 /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From g.santomaggio@REDACTED Thu Feb 14 13:02:56 2019 From: g.santomaggio@REDACTED (Gabriele Santomaggio) Date: Thu, 14 Feb 2019 13:02:56 +0100 Subject: [erlang-questions] Detect the Erlang cluster connection (ssl/tcp). Message-ID: I am looking an easy way to detect the cluster connection (ssl/tcp). Someone suggested using net_kernel:nodes_info(). but not totally sure, because of: - nodes_info is undocumented (http://erlang.org/doc/man/net_kernel.html) so I suppose that should not be used. - mostly works, but in servers, with multiple network interfaces I can have an output like this: $ sudo rabbitmqctl eval "net_kernel:nodes_info()." {ok,[{'rabbit@REDACTED', [{owner,<11202.610.0>}, {state,up}, {address, {net_address, {{127,0,0,1},52568}, "yyyyyyy",proxy,inet}}, {type,normal}, {in,5160486}, {out,3360784}]}, {'rabbit@REDACTED, [{owner,<11202.402.0>}, {state,up}, {address, {net_address, {{127,0,0,1},52568}, "yyyyyyyyyyyy",proxy,inet}}, {type,normal}, {in,2956758}, {out,5019215}]}, so "proxy" "inet" as family address and it is not clear what kind on the cluster connection is. In this case, for example, is a ssl cluster sudo rabbitmqctl eval "init:get_arguments()." ... {pa,["$ERL_SSL_PATH"]}, {proto_dist,["inet_tls"]}, Is there another way? should I call init:get_arguments() for each node? Thank you -- Gabriele Santomaggio -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Thu Feb 14 13:18:08 2019 From: otp@REDACTED (Erlang/OTP) Date: Thu, 14 Feb 2019 13:18:08 +0100 Subject: [erlang-questions] Patch Package OTP 20.3.8.19 Released Message-ID: <20190214121808.GA665@erix.ericsson.se> Patch Package: OTP 20.3.8.19 Git Tag: OTP-20.3.8.19 Date: 2019-02-14 Trouble Report Id: OTP-15430, OTP-15569 Seq num: ERIERL-237, ERIERL-302 System: OTP Release: 20 Application: diameter-2.1.4.1, erts-9.3.3.9 Predecessor: OTP 20.3.8.18 Check out the git tag OTP-20.3.8.19, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- diameter-2.1.4.1 ------------------------------------------------ --------------------------------------------------------------------- The diameter-2.1.4.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-15569 Application(s): diameter Related Id(s): ERIERL-302 Fix failure of incoming answer message with faulty Experimental-Result-Code. Failure to decode the AVP resulted in an uncaught exception, with no no handle_answer/error callback as a consequence. Full runtime dependencies of diameter-2.1.4.1: erts-6.4, kernel-3.2, ssl-6.0, stdlib-2.4 --------------------------------------------------------------------- --- erts-9.3.3.9 ---------------------------------------------------- --------------------------------------------------------------------- The erts-9.3.3.9 application can be applied independently of other applications on a full OTP 20 installation. --- Improvements and New Features --- OTP-15430 Application(s): erts Related Id(s): ERIERL-237 Added an optional ./configure flag to compile the emulator with spectre mitigation: --with-spectre-mitigation Note that this requires a recent version of GCC with support for spectre mitigation and the --mindirect-branch=thunk flag, such as 8.1. Full runtime dependencies of erts-9.3.3.9: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From vances@REDACTED Thu Feb 14 18:09:14 2019 From: vances@REDACTED (Vance Shipley) Date: Thu, 14 Feb 2019 22:39:14 +0530 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: On Thu, Feb 14, 2019 at 2:40 PM block.rxckin.beats@REDACTED wrote: > > No, I meant: > > udp_sup > > -> udp_worker > > -> proto_sup > > -> dtls_woker > > -> stun_worker > > -> srtp_worker > > in that case, > udp_worker can terminate proto_sup > but who terminate udp_sup ? No, udp_sup terminates proto_sup. Use a one_for_all strategy on udp_sup. When udp_worker exits all the others are terminated by the supervisors. -- -Vance From block.rxckin.beats@REDACTED Fri Feb 15 00:23:08 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Fri, 15 Feb 2019 08:23:08 +0900 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: References: Message-ID: <9DB2B7F7-3E7A-44B1-AC06-8C0B0DB0D3B2@gmail.com> in that case, is child of udp_sup "permanent" or "temporary" ? if temporary, udp_sup seems ignore udp_worker's terminate, and proto_sup still alive. if permanent, udp_sup try to restart child which I don't expected. and if *failed to restart* udp_sup will terminate. so, you mean, "fail to restart permanent udp_worker(which unexpected) will terminate udp_sup eventually in one_for_all" is correct way to terminate supervisor from its child? thanks Jxck 2019/02/15 2:09?Vance Shipley ????: > On Thu, Feb 14, 2019 at 2:40 PM block.rxckin.beats@REDACTED > wrote: >>> No, I meant: >>> udp_sup >>> -> udp_worker >>> -> proto_sup >>> -> dtls_woker >>> -> stun_worker >>> -> srtp_worker >> >> in that case, >> udp_worker can terminate proto_sup >> but who terminate udp_sup ? > > No, udp_sup terminates proto_sup. Use a one_for_all strategy on > udp_sup. When udp_worker exits all the others are terminated by the > supervisors. > > -- > -Vance From by@REDACTED Fri Feb 15 08:11:54 2019 From: by@REDACTED (by) Date: Fri, 15 Feb 2019 15:11:54 +0800 Subject: [erlang-questions] Build Erlang from source failed with GitHub tag OTP-21.2.5 Message-ID: <12FCF2C6-688E-4001-A95C-34E5DECECC00@meetlost.com> Hello list, I decide to build Erlang from source via Official GitHub repo. [ on macOS 10.12.6 ] I check out the tag OTP-21.2.5 into local branch, and try to build it from script below: ==== File ==== #!/bin/sh LANG=C ./otp_build autoconf ./configure --prefix=/Users/by/local/pkgs/erlang/erlang21.2.5 \ --with-ssl=/usr/local/Cellar/openssl/1.0.2q make #make install #make install-docs ==== File ==== The build failed with below error (if more detailed error is needed, I can provide later): === Error Log ==== === Entering application stdlib make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. === Leaving application stdlib === Entering application sasl make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. === Leaving application sasl === Entering application kernel make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. === Leaving application kernel === Entering application compiler make[3]: Nothing to be done for `opt'. make[3]: Nothing to be done for `opt'. === Leaving application compiler === Entering application tools MAKE opt make[4]: Nothing to be done for `all'. ERLC ../ebin/xref_parser.beam compile: warnings being treated as errors /Users/by/projects/otp/bootstrap/lib/parsetools/include/yeccpre.hrl:60: erlang:get_stacktrace/0: deprecated; use the new try/catch syntax for retrieving the stack backtrace make[3]: *** [../ebin/xref_parser.beam] Error 1 make[2]: *** [opt] Error 2 make[1]: *** [opt] Error 2 make: *** [libs] Error 2 === Error Log ==== But when I build from otp_src_21.0.tar.gz, with the same script, it just works. Can anyone help me on this? Best Regards, Yao Bao -------------- next part -------------- An HTML attachment was scrubbed... URL: From starbelly@REDACTED Fri Feb 15 08:43:12 2019 From: starbelly@REDACTED (Bryan Paxton) Date: Fri, 15 Feb 2019 01:43:12 -0600 Subject: [erlang-questions] Build Erlang from source failed with GitHub tag OTP-21.2.5 In-Reply-To: <12FCF2C6-688E-4001-A95C-34E5DECECC00@meetlost.com> References: <12FCF2C6-688E-4001-A95C-34E5DECECC00@meetlost.com> Message-ID: <1e1502bd-defc-ae29-b6f4-7650871506db@pobox.com> ?I have run into this issue when I had artifacts left over from previous builds in my repo. Try to executing the following (after ensuring you save any changes you might have made) : git clean -xfd Then run ./otp_build autoconf again -- Bryan On 2/15/19 1:11 AM, by wrote: > Hello list, > > I decide to build Erlang from source via Official GitHub repo. [ on > macOS 10.12.6?] > > I check out the tag?OTP-21.2.5 into local branch, and try to build it > from script below: > ==== File ==== > #!/bin/sh > > LANG=C ./otp_build autoconf > ./configure --prefix=/Users/by/local/pkgs/erlang/erlang21.2.5 \ > --with-ssl=/usr/local/Cellar/openssl/1.0.2q > make > #make install > #make install-docs > ==== File ==== > > The build failed with below error (if more detailed error is needed, I > can provide later): > === Error Log ==== > === Entering application stdlib > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > === Leaving application stdlib > === Entering application sasl > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > === Leaving application sasl > === Entering application kernel > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > === Leaving application kernel > === Entering application compiler > make[3]: Nothing to be done for `opt'. > make[3]: Nothing to be done for `opt'. > === Leaving application compiler > === Entering application tools > ?MAKEopt > make[4]: Nothing to be done for `all'. > ?ERLC../ebin/xref_parser.beam > compile: warnings being treated as errors > /Users/by/projects/otp/bootstrap/lib/parsetools/include/yeccpre.hrl:60: > erlang:get_stacktrace/0: deprecated; use the new try/catch syntax for > retrieving the stack backtrace > make[3]: *** [../ebin/xref_parser.beam] Error 1 > make[2]: *** [opt] Error 2 > make[1]: *** [opt] Error 2 > make: *** [libs] Error 2 > === Error Log ==== > > But when I build from?otp_src_21.0.tar.gz, with the same script, it > just works. > > Can anyone help me on this? > > Best Regards, > Yao Bao > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From block.rxckin.beats@REDACTED Fri Feb 15 09:39:15 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Fri, 15 Feb 2019 17:39:15 +0900 Subject: [erlang-questions] terminate supervisor from it's child process In-Reply-To: <9DB2B7F7-3E7A-44B1-AC06-8C0B0DB0D3B2@gmail.com> References: <9DB2B7F7-3E7A-44B1-AC06-8C0B0DB0D3B2@gmail.com> Message-ID: I solved this problem by using linking workers. s_1_f_1 udp_sup ----- udp_worker_sup - udp_worker |- dtls_worker_sup - dtls_worker |- srtp_worker_sup - srtp_worker |- stun_worker_sup - stun_worker start udp_worker for single UDP socket, and start dtls/srtp/stun workers in init() of udp_worker, and udp_worker link with ever other workers. if any worker terminates, ever other worker handling same UDP socket will terminate. no need to supervisor for single UDP socket and terminate supervisor for socket closing. thanks Jxck 2019?2?15?(?) 8:23 : > in that case, is child of udp_sup "permanent" or "temporary" ? > > if temporary, udp_sup seems ignore udp_worker's terminate, and proto_sup > still alive. > > if permanent, udp_sup try to restart child which I don't expected. and if > *failed to restart* udp_sup will terminate. > > so, you mean, "fail to restart permanent udp_worker(which unexpected) > will terminate udp_sup eventually in one_for_all" is correct way to > terminate supervisor from its child? > > thanks > Jxck > > 2019/02/15 2:09?Vance Shipley ????: > > > On Thu, Feb 14, 2019 at 2:40 PM block.rxckin.beats@REDACTED > > wrote: > >>> No, I meant: > >>> udp_sup > >>> -> udp_worker > >>> -> proto_sup > >>> -> dtls_woker > >>> -> stun_worker > >>> -> srtp_worker > >> > >> in that case, > >> udp_worker can terminate proto_sup > >> but who terminate udp_sup ? > > > > No, udp_sup terminates proto_sup. Use a one_for_all strategy on > > udp_sup. When udp_worker exits all the others are terminated by the > > supervisors. > > > > -- > > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From by@REDACTED Fri Feb 15 09:42:04 2019 From: by@REDACTED (by) Date: Fri, 15 Feb 2019 16:42:04 +0800 Subject: [erlang-questions] Build Erlang from source failed with GitHub tag OTP-21.2.5 In-Reply-To: <1e1502bd-defc-ae29-b6f4-7650871506db@pobox.com> References: <12FCF2C6-688E-4001-A95C-34E5DECECC00@meetlost.com> <1e1502bd-defc-ae29-b6f4-7650871506db@pobox.com> Message-ID: You are right. I run the command you mentioned, and the build just works. Thank you! > ? 2019?2?15??15:43?Bryan Paxton ??? > > I have run into this issue when I had artifacts left over from previous builds in my repo. Try to executing the following (after ensuring you save any changes you might have made) : > > git clean -xfd > > > Then run ./otp_build autoconf again > > > -- > > Bryan > > > On 2/15/19 1:11 AM, by wrote: >> Hello list, >> >> I decide to build Erlang from source via Official GitHub repo. [ on macOS 10.12.6 ] >> >> I check out the tag OTP-21.2.5 into local branch, and try to build it from script below: >> ==== File ==== >> #!/bin/sh >> >> LANG=C ./otp_build autoconf >> ./configure --prefix=/Users/by/local/pkgs/erlang/erlang21.2.5 \ >> --with-ssl=/usr/local/Cellar/openssl/1.0.2q >> make >> #make install >> #make install-docs >> ==== File ==== >> >> The build failed with below error (if more detailed error is needed, I can provide later): >> === Error Log ==== >> === Entering application stdlib >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> === Leaving application stdlib >> === Entering application sasl >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> === Leaving application sasl >> === Entering application kernel >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> === Leaving application kernel >> === Entering application compiler >> make[3]: Nothing to be done for `opt'. >> make[3]: Nothing to be done for `opt'. >> === Leaving application compiler >> === Entering application tools >> MAKE opt >> make[4]: Nothing to be done for `all'. >> ERLC ../ebin/xref_parser.beam >> compile: warnings being treated as errors >> /Users/by/projects/otp/bootstrap/lib/parsetools/include/yeccpre.hrl:60: erlang:get_stacktrace/0: deprecated; use the new try/catch syntax for retrieving the stack backtrace >> make[3]: *** [../ebin/xref_parser.beam] Error 1 >> make[2]: *** [opt] Error 2 >> make[1]: *** [opt] Error 2 >> make: *** [libs] Error 2 >> === Error Log ==== >> >> But when I build from otp_src_21.0.tar.gz, with the same script, it just works. >> >> Can anyone help me on this? >> >> Best Regards, >> Yao Bao >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From aperri@REDACTED Fri Feb 15 08:44:55 2019 From: aperri@REDACTED (Alberto Perri) Date: Fri, 15 Feb 2019 08:44:55 +0100 Subject: [erlang-questions] Profinet Message-ID: <12d9f372-2e64-f858-34e2-19846965c3de@operamail.com> I'm looking for some information on how to interface Erlang with Profient. *"Profinet*Technology.*PROFINET*is the standard for industrial networking in automation. It connects devices, systems, and cells, facilitating faster, safer, less costly and higher quality manufacturing. It easily integrates existing systems and equipment while bringing the richness of Ethernet down to the factory floor." -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Fri Feb 15 15:46:47 2019 From: otp@REDACTED (Erlang/OTP) Date: Fri, 15 Feb 2019 15:46:47 +0100 Subject: [erlang-questions] Patch Package OTP 20.3.8.20 Released Message-ID: <20190215144647.GA11433@erix.ericsson.se> Patch Package: OTP 20.3.8.20 Git Tag: OTP-20.3.8.20 Date: 2019-02-15 Trouble Report Id: OTP-15584 Seq num: ERIERL-282 System: OTP Release: 20 Application: common_test-1.15.4.1 Predecessor: OTP 20.3.8.19 Check out the git tag OTP-20.3.8.20, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.15.4.1 -------------------------------------------- --------------------------------------------------------------------- The common_test-1.15.4.1 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-15584 Application(s): common_test Related Id(s): ERIERL-282 The status of a test case which failed with timetrap timeout in end_per_testcase could not be modified by returning {fail,Reason} from a post_end_per_testcase hook function. This is now corrected. Full runtime dependencies of common_test-1.15.4.1: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-3.4, syntax_tools-1.7, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From marc@REDACTED Fri Feb 15 16:06:49 2019 From: marc@REDACTED (Marc Worrell) Date: Fri, 15 Feb 2019 16:06:49 +0100 Subject: [erlang-questions] [ANN] Zotonic release 0.46.0 Message-ID: <8F643125-2095-4399-8C3A-91FDB1264909@worrell.nl> Hi, Zotonic is the Erlang Content Management System and Framework. We have just released Zotonic version 0.46.0. This is a maintenance release, the main changes are: * Added ``mod_auth2fa`` for two factor authentication support * Added ``mod_clamav`` for virus scanning of media items * Newly entered passwords can now be checked against a regex, configured in ``mod_admin_identity.password_regex`` * New versions of tinyMCE, jQuery, and Bootstrap. Full release notes are at: http://docs.zotonic.com/en/latest/developer-guide/releasenotes/rel_0.46.0.html Kind regards, The Zotonic core team. From vfordos@REDACTED Fri Feb 15 17:11:24 2019 From: vfordos@REDACTED (Viktoria Fordos (vfordos)) Date: Fri, 15 Feb 2019 16:11:24 +0000 Subject: [erlang-questions] 1ST CALL FOR PAPERS: Erlang Workshop 2019 Message-ID: Technical, practice, and application papers related to Erlang, BEAM, Elixir, Scala/Akka, CloudHaskell, Lisp Flavoured Erlang, OCaml, and functional programming are welcome and encouraged. 1ST CALL FOR PAPERS Eighteenth ACM SIGPLAN Erlang Workshop https://icfp19.sigplan.org/home/erlang-2019 Berlin, Germany, 18 August 2019 Satellite event of the 24th ACM SIGPLAN International Conference on Functional Programming (ICFP 2019) 18 - 23 August, 2019 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite two types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). Submission related to Erlang, Elixir, Scala/Akka, CloudHaskell, Lisp Flavoured Erlang, OCaml, and functional programming are welcome and encouraged. The maximum length for technical papers is restricted to 12 pages, but short papers (max. 6 pages) are welcomed as well. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages, but short papers (max. 6 pages) are welcomed as well. Workshop Co-Chairs Adrian Francalanza, University of Malta, Malta Vikt?ria F?rd?s, Cisco Systems, Sweden Program Committee (Note: the Workshop Co-Chairs are also committee members) Annette Bieniusa, University of Kaiserslautern, Germany Christopher S. Meiklejohn, Carnegie Mellon University, USA Clara Benac Earle, Universidad Politecnica de Madrid, Spain Claudio Antares Mezzina, IMT Lucca, Italy Emilio Tuosto, University of Leicester, UK Felix Mulder, Klarna AB, Sweden Francesco Cesarini, Erlang Solutions Ltd, UK Julien Lange, University of Kent, UK Kenji Rikitake, KRPEO, Japan Melinda T?th, ELTE E?tv?s Lor?nd University, Hungary Natalia Chechina, Bournemouth University, UK Rumyana Neykova, Brunel University, UK Scott Lystig Fritchie, Wallaroo, USA Thomas Arts, Quviq AB, Sweden Torben Hoffmann, Alert Logic, Denmark Important Dates Submission deadline: Fri May 10, 2019 Author notification: Fri June 7, 2019 Final submission for the publisher: Sun June 30, 2019 Workshop date: Sun August 18, 2019 Instructions to authors Papers must be submitted online via HotCRP (via the "Erlang2019" event). The submission page is https://erlang19.hotcrp.com Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. The proceedings will be freely available for download from the ACM Digital Library from one week before the start of the conference until two weeks after the conference. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details For registration, please see the ICFP 2019 web site at: http://icfp19.sigplan.org/ Related Links ICFP 2019 web site: http://icfp19.sigplan.org/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ HotCRP submission site: https://erlang19.hotcrp.com Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Attendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/CodeOfConduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Fri Feb 15 17:41:09 2019 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 15 Feb 2019 17:41:09 +0100 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: <20160401.130823.1006060931061872335.mbj@tail-f.com> Message-ID: If anyone is interested (hello Martin and Siri!), we have slightly modernized our rotating cleartext logger handler to keep it compatible with recent OTP versions, and split it up into three separate modules, one as api and handler-supervisor, and one for the formatting of messages. Available here, as a diff on top of the old version if anyone still wants that: https://github.com/richcarl/otp/commits/disk_log_h /Richard Den fre 1 apr. 2016 kl 13:34 skrev Richard Carlsson < carlsson.richard@REDACTED>: > Thanks for pointing out that old mail, Martin! These days though, it's > probably worthwhile to rethink/rewrite these handlers from scratch rather > than import disk_log_h into OTP as it is. > > > /Richard > > 2016-04-01 13:08 GMT+02:00 Martin Bjorklund : > >> Hi, >> >> We are not using log_mf_h, but instead we're using disk_log_h (see >> http://erlang.org/pipermail/erlang-patches/2012-January/002583.html) >> >> We use this handler to get log rotation, but we log all error logger >> messages in clear text (this should IMO be the default). >> >> If you're interested in adding disk_log_h to OTP (search for >> disk_log_h in disk_log.erl...), the latest version >> (updated as recently as 2006 :) can be found here: >> >> https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl >> >> I support the deprecation initiative! >> >> >> /martin >> >> >> >> Siri Hansen wrote: >> > Hello list! >> > >> > log_mf_h is an error_logger event handler which logs events to disk and >> > does log rotation. rb (report browser) is the tool for reading and >> > formatting the logs. The implementation of both modules is quite old and >> > outdated. The fact that there is a size field for the events of only 16 >> > bits, which we haven't got any complaints for, has made us believe that >> > there might not be many users of these tools. I'm writing to the list to >> > find out if this is correct... >> > >> > Are you using log_mf_h (e.g. by setting environment variables >> > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log >> rotation or >> > the binary logging (or both) that you are really after? Have you >> considered >> > alternative tools? >> > >> > Kind Regards >> > /siri@REDACTED >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Fri Feb 15 22:09:38 2019 From: andrew@REDACTED (Andrew Thompson) Date: Fri, 15 Feb 2019 16:09:38 -0500 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: <20160405084953.GA32163@seth> Message-ID: <20190215210935.GK1783@thecloud.hijacked.us> On Fri, Apr 08, 2016 at 02:36:37PM +0200, Siri Hansen wrote: > Thanks for the feedback, everyone! > > We have now decided not to remove log_mf_h and rb. Instead there will be > some work done to modernize these tools. The big problem we had with these at Basho (RIP) was that customers had no idea how the log files worked because it's very different to traditional UNIX logging. Usually you have things like app.log, app.log.0, ... app.log.N and the number indicates their age. log_mf has a weird index file where the active log file is encoded as a decimal value (not ASCII, IIRC). This is very confusing for everyone who is not an Erlang user (maybe even for them judging by this thread). The main thing I'd like to see improve is the ergonomics. Make it work more like a sysadmin would expect, have the newest log file be .0 and dump the index. Every time you overflow move the files around to reflect that. Andrew From carlsson.richard@REDACTED Fri Feb 15 23:45:56 2019 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 15 Feb 2019 23:45:56 +0100 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: <20190215210935.GK1783@thecloud.hijacked.us> References: <20160405084953.GA32163@seth> <20190215210935.GK1783@thecloud.hijacked.us> Message-ID: Yeah, the main thing the rotation mechanism is missing in our handler as well is renumbering of existing files; it also simply increments the number as it goes, and you have to sort on timestamps to see which one is the latest. Hence, I can't recommend it as is unless you don't mind that sort of bother. On the other hand, clear text logs are nice. Somehow, there's always been something more important to do than improve the log numbering, but that's how it is. /Richard Den fre 15 feb. 2019 kl 22:09 skrev Andrew Thompson : > On Fri, Apr 08, 2016 at 02:36:37PM +0200, Siri Hansen wrote: > > Thanks for the feedback, everyone! > > > > We have now decided not to remove log_mf_h and rb. Instead there will be > > some work done to modernize these tools. > > The big problem we had with these at Basho (RIP) was that customers had > no idea how the log files worked because it's very different to > traditional UNIX logging. Usually you have things like app.log, > app.log.0, ... app.log.N and the number indicates their age. log_mf has > a weird index file where the active log file is encoded as a decimal > value (not ASCII, IIRC). This is very confusing for everyone who is not > an Erlang user (maybe even for them judging by this thread). > > The main thing I'd like to see improve is the ergonomics. Make it work > more like a sysadmin would expect, have the newest log file be .0 and > dump the index. Every time you overflow move the files around to reflect > that. > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Sat Feb 16 06:00:39 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Sat, 16 Feb 2019 16:00:39 +1100 Subject: [erlang-questions] meaning of Mod ? Message-ID: <24cb898e223813e10273c635c3824a83769d4517@webmail.iinet.net.au> Dear Erlangers, In module server1 at the top of Joe's book on page 363 is; start(Name, Mod)? -> In this context, what is the meaning of 'Mod' ? Thank you in advance,peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From by@REDACTED Sat Feb 16 07:00:52 2019 From: by@REDACTED (by) Date: Sat, 16 Feb 2019 14:00:52 +0800 Subject: [erlang-questions] meaning of Mod ? In-Reply-To: <24cb898e223813e10273c635c3824a83769d4517@webmail.iinet.net.au> References: <24cb898e223813e10273c635c3824a83769d4517@webmail.iinet.net.au> Message-ID: <95046141-3630-46AF-9EB2-937E3D42F8F2@meetlost.com> I believe the ?Mod? means the real server(module) the program trying to start. You can see a module name_server(.erl) on this page. And the practice runs with: 1> server1:start(name_server, name_server). Best Regards, Yao Bao > ? 2019?2?16??13:00?Peter J Etheridge ??? > > Dear Erlangers, > In module server1 at the top of Joe's book on page 363 is; > > start(Name, Mod) -> > > In this context, what is the meaning of 'Mod' ? > > Thank you in advance, > peter > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sat Feb 16 13:15:13 2019 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 16 Feb 2019 13:15:13 +0100 Subject: [erlang-questions] meaning of Mod ? In-Reply-To: <24cb898e223813e10273c635c3824a83769d4517@webmail.iinet.net.au> References: <24cb898e223813e10273c635c3824a83769d4517@webmail.iinet.net.au> Message-ID: Mod is an atom containing the name of a module. It gets used twice in server3.erl in the lines Mod:init() and Mod:handle(Request, OldState) Have fun /Joe On Sat, Feb 16, 2019 at 6:00 AM Peter J Etheridge wrote: > > Dear Erlangers, > In module server1 at the top of Joe's book on page 363 is; > > start(Name, Mod) -> > > In this context, what is the meaning of 'Mod' ? > > Thank you in advance, > peter > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sat Feb 16 13:35:35 2019 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 16 Feb 2019 13:35:35 +0100 Subject: [erlang-questions] Best choice for complex data structure In-Reply-To: <1779429.TEnnqv59f6@takoyaki> References: <23fc155e-93ec-690f-eeec-58232dfdf517@aim.com> <31599511.N10xD8Cjzr@takoyaki> <1779429.TEnnqv59f6@takoyaki> Message-ID: Musical representation is *very* complex - I've also written several libraries to manipulate midi and turn note representations into SVG - had you thought about basing your representation on musicXML - sooner or later you might want to talk to a program that speaks musicXML. I have drivers for coreaudio that run on a mac that interface erlang to any midi device connected to a mac - if anybody is interested I can make these available. Cheers /Joe On Thu, Jan 24, 2019 at 7:01 PM wrote: > > On 2019?1?24???? 11?08?09? JST you wrote: > > Hi Craig, > > > > Thanks so much for sharing this. You're very generous. > > > > I have a fully functional MIDI module with a tested and effective set of > > low level IO. I'm working now at the meta level (as a composer), > > looking to translate some impressionistic higher level ideas through > > intermediary functions to MIDI output. > > > > I'm also just learning Erlang, so I'm sure some of my questions are very > > naive. > > > > What is: > > > > Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data, > > RepeatMeta}] > > > > Is Score a variable -- or a type that is a list of tuples? > > Nice! Welcome to Erlang! > Once your eyes adjust I think you'll thoroughly enjoy it around here. :-D > > And yes, you are spot on, `Score` above is a type specification. > You'll find this helpful: > http://erlang.org/doc/reference_manual/typespec.html > > It is the required notation when talking to Dialyzer (obviously) but also useful when communicating amongst Erlangers. > I gave quite a few type specification examples in the original message, so that link will help you decipher them. > > Everything capitalized is a possible type (a type variable that I left undefined -- use your imagination about what type of values might be useful in those places) and everything that is lower-case is an actual definite type. Parens after a lower-case string are a defined type, an atom with no parens is just the atom. A pipe (like `|`) is an "or". > > To get more specific... > > > Score :: [{InsertTime, Title, SegmentStart, SegmentStop, MIDI_Data, RepeatMeta}] > where InsertTime :: time_unit(), > Title :: string(), > SegmentStart :: time_unit(), > SegmentStop :: time_unit(), > MIDI_Data :: binary(), > RepeatMeta :: none | repeat_meta(). > > -type time_unit() :: non_neg_integer() % an integer value in milliseconds representing absolute time > | {time_signature(), Beats :: non_neg_integer()}. > > > The above isn't 100% proper type notation (the way Score is defined looks like a -spec, not a -type, but it is a useful notation amongst erlangers, anyway), and I never defined what `repeat_meta()` should be, but should convey some general idea what I intended. > > The point in expressing things this way is that a list of MIDI segments (which already carry their own musical data, relative to themselves) can be included as data into a higher-level program with the addition of metadata that is meaningful to that higher-level program, and the resulting list of MIDI segment points (some composed locally, some referenced from a library -- the origin not being important) can be synthesized by a program to produce a new MIDI output readable by a musical notation output program that reads MIDI, or by a player. > > That may or may not be what you're looking for (maybe you want to write a musical synthesis program that focuses specifically on musical architecture, in which case your real focus is the segment meta and arrangements among the segments, not the actual MIDI data), but hopefully this gives you some ideas for thinking of data as more than just nested data or OOP-style "objects". > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From viktor@REDACTED Sat Feb 16 19:24:38 2019 From: viktor@REDACTED (=?UTF-8?Q?Viktor_S=c3=b6derqvist?=) Date: Sat, 16 Feb 2019 19:24:38 +0100 Subject: [erlang-questions] if-elseif-else trick Message-ID: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> Hey list! Last night I was thinking that it might be possible to implement if-elseif-else syntax using macros and a parse transform, so you can write like this: -include_lib("elseif/include/elseif.hrl"). f(X) -> ?'if'(X > 0) -> pos ?elseif(X < 0) -> neg ?else -> zero end. It's possible. Here is how: https://github.com/zuiderkwast/elseif Disclaimer: This may be considered macro and parse-transform abuse. You may get strange syntax error messages if e.g. using ?elseif without ?'if' or if leaving out the mandatory ?else part. Viktor From soverdor@REDACTED Sun Feb 17 02:07:50 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Sat, 16 Feb 2019 17:07:50 -0800 Subject: [erlang-questions] add custom paths for my programs Message-ID: How do I add a custom-path for my application location on all of the systems that I support on a daily basis? Thanks, Sam From mikpelinux@REDACTED Sun Feb 17 10:02:02 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 17 Feb 2019 10:02:02 +0100 Subject: [erlang-questions] add custom paths for my programs In-Reply-To: References: Message-ID: On Sun, Feb 17, 2019 at 1:59 AM Sam Overdorf wrote: > > How do I add a custom-path for my application location on all of the > systems that I support on a daily basis? See "erl -pa " docs and the things (code module, ERL_LIBS env var) it refers to. From cchalasani@REDACTED Sun Feb 17 11:16:00 2019 From: cchalasani@REDACTED (Chaitanya Chalasani) Date: Sun, 17 Feb 2019 15:46:00 +0530 Subject: [erlang-questions] Mnesia transform_table doesn't work Message-ID: <9D2CD72B-33A5-4729-A916-C5826810157D@me.com> Hi, When I run mnesia:transform_table/4 on a table to add a new field, the function returns {atomic, ok} but neither the schema is updated nor the records are transformed. The transform function is written as - transform_fields(Tname, FieldsNew) -> FieldsOld = mnesia:table_info(Tname, attributes), RecordName = mnesia:table_info(Tname, record_name), if FieldsNew == FieldsOld -> not_needed; true -> Transformer = make_transformer(RecordName, FieldsOld, FieldsNew), mnesia:transform_table(Tname, Transformer, FieldsNew, RecordName), end. make_transformer(RecordName, OldFields, NewFields) -> fun(OldRec) -> list_to_tuple([RecordName|lists:map( fun(NewField) -> case string:str(OldFields, [NewField]) of 0 -> undefined; Pos -> element(Pos + 1, OldRec) end end, NewFields )]) end. The table I was running is - calls : with 2917589 records occupying 2056606274 words of mem I am running - Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] /Chaitanya From hugo@REDACTED Sun Feb 17 19:47:56 2019 From: hugo@REDACTED (Hugo Mills) Date: Sun, 17 Feb 2019 18:47:56 +0000 Subject: [erlang-questions] Odd behaviour of code:priv_dir/1 Message-ID: <20190217184756.GC317@carfax.org.uk> I've been trying to set up a basic CI system for a few projects. The problem is, I'm consistently getting test failures when running tests through the CI, where those tests pass perfectly well on a manual run. The test failure is: === Ended at 2019-02-17 15:36:20 === Location: [{filename,join,446}, {stdlib_SUITE,run_suite,37}, {test_server,ts_tc,1545}, {test_server,run_test_case_eval1,1063}, {test_server,run_test_case_eval,995}] === Reason: no function clause matching filename:join({error,bad_name},"stdlib") (filename.erl, line 446) in function stdlib_SUITE:run_suite/3 (test/stdlib_SUITE.erl, line 37) in call from test_server:ts_tc/3 (test_server.erl, line 1545) in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1063) in call from test_server:run_test_case_eval/9 (test_server.erl, line 995) which comes from this line of code in my CT text fixture: BaseContext = #{"$module_path" => [filename:join(code:priv_dir(childe), "stdlib")], "$data_path" => [?config(data_dir, Conf)]}, I can run this on my main dev machines ("make ct" -- I'm using erlang.mk) quite happily, and it passes the test. I can run it manually on the CI worker machine and it passes the test. If I run it through a CI system, it fails with the above error. There's clearly _something_ odd about the environment that the CI system is running in, but I'm damned if I can tell what it is. It's not specific to the CI system, either: I've seen this with Buildbot, yesterday, and now GoCD today. Why would code:priv_dir/1 be failing in the CI when it's working in an interactive shell? My dev machines are running 21.2.2 from Debian; the CI is running 21.2.5 from Erlang Solutions, but that's not the issue, because I can do a manual run on the CI box, and it works. Thanks for any ideas, Hugo. -- Hugo Mills | I used to live in hope, but I got evicted. hugo@REDACTED carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From petergi@REDACTED Mon Feb 18 04:11:56 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Mon, 18 Feb 2019 14:11:56 +1100 Subject: [erlang-questions] meaning of Mod ? In-Reply-To: Message-ID: <7570b445da997ea05f52788e7ce2eb0f88bf57bb@webmail.iinet.net.au> Thank you for the clarification, Joe. happy coding, peter ----- Original Message ----- From: "Joe Armstrong" To:"Peter J Etheridge" Cc:"Erlang" Sent:Sat, 16 Feb 2019 13:15:13 +0100 Subject:Re: [erlang-questions] meaning of Mod ? Mod is an atom containing the name of a module. It gets used twice in server3.erl in the lines Mod:init() and Mod:handle(Request, OldState) Have fun /Joe On Sat, Feb 16, 2019 at 6:00 AM Peter J Etheridge wrote: > > Dear Erlangers, > In module server1 at the top of Joe's book on page 363 is; > > start(Name, Mod) -> > > In this context, what is the meaning of 'Mod' ? > > Thank you in advance, > peter > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.schultz@REDACTED Mon Feb 18 09:02:11 2019 From: andreas.schultz@REDACTED (Andreas Schultz) Date: Mon, 18 Feb 2019 09:02:11 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> Message-ID: Hi Viktor, Viktor S?derqvist schrieb am Sa., 16. Feb. 2019 um 19:24 Uhr: > Hey list! > > Last night I was thinking that it might be possible to implement > if-elseif-else syntax using macros and a parse transform, so you can > write like this: > > -include_lib("elseif/include/elseif.hrl"). > > f(X) -> > ?'if'(X > 0) -> pos > ?elseif(X < 0) -> neg > ?else -> zero > end. > but why would you use such a thing when you can simply write: if X > 0 -> pos; X < 0 -> neg; true -> zero end. Andreas > It's possible. Here is how: https://github.com/zuiderkwast/elseif > > Disclaimer: This may be considered macro and parse-transform abuse. You > may get strange syntax error messages if e.g. using ?elseif without > ?'if' or if leaving out the mandatory ?else part. > > Viktor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- -- Dipl.-Inform. Andreas Schultz ----------------------- enabling your networks ---------------------- Travelping GmbH Phone: +49-391-81 90 99 0 Roentgenstr. 13 Fax: +49-391-81 90 99 299 39108 Magdeburg Email: info@REDACTED GERMANY Web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann VAT ID No.: DE236673780 --------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Mon Feb 18 10:34:11 2019 From: zxq9@REDACTED (zxq9@REDACTED) Date: Mon, 18 Feb 2019 18:34:11 +0900 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> Message-ID: <2471909.jqRKXF7QSM@takoyaki> On 2019?2?18???? 9?02?11? JST Andreas Schultz wrote: > Hi Viktor, > > Viktor S?derqvist schrieb am Sa., 16. Feb. 2019 um > 19:24 Uhr: > > > Hey list! > > > > Last night I was thinking that it might be possible to implement > > if-elseif-else syntax using macros and a parse transform, so you can > > write like this: > > > > -include_lib("elseif/include/elseif.hrl"). > > > > f(X) -> > > ?'if'(X > 0) -> pos > > ?elseif(X < 0) -> neg > > ?else -> zero > > end. > > > > but why would you use such a thing when you can simply write: > > if X > 0 -> pos; > X < 0 -> neg; > true -> zero > end. `if ... true` is a code stink. Be explicit in your range tests -- that is what `if` is actually good at: if X > 0 -> pos; X < 0 -> neg; X == 0 -> zero end If you find yourself *needing* a `true` catchall, you should be writing a function head or `case`. That said... The original premise of this thread is based on a misunderstanding of `if` and probably inexperience with `case`. -Craig From otp@REDACTED Mon Feb 18 13:51:14 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 18 Feb 2019 13:51:14 +0100 Subject: [erlang-questions] Patch Package OTP 21.2.6 Released Message-ID: <20190218125114.GA47444@erix.ericsson.se> Patch Package: OTP 21.2.6 Git Tag: OTP-21.2.6 Date: 2019-02-18 Trouble Report Id: OTP-14728, OTP-15573 Seq num: ERIERL-303, ERIERL-306 System: OTP Release: 21 Application: erts-10.2.4, stdlib-3.7.1 Predecessor: OTP 21.2.5 Check out the git tag OTP-21.2.6, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.2.4 ----------------------------------------------------- --------------------------------------------------------------------- Note! The erts-10.2.4 application can *not* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependencies have to be satisfied: -- kernel-6.1 (first satisfied in OTP 21.1) -- sasl-3.3 (first satisfied in OTP 21.2) --- Fixed Bugs and Malfunctions --- OTP-14728 Application(s): erts Related Id(s): ERIERL-303 When using the {linger,{true,T}} option; gen_tcp:listen/2 used the full linger time before returning for example eaddrinuse. This bug has now been corrected. Full runtime dependencies of erts-10.2.4: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- stdlib-3.7.1 ---------------------------------------------------- --------------------------------------------------------------------- The stdlib-3.7.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15573 Application(s): stdlib Related Id(s): ERIERL-306 Optimize pretty printing of terms. The slower behaviour was introduced in Erlang/OTP 20. Full runtime dependencies of stdlib-3.7.1: compiler-5.0, crypto-3.3, erts-10.0, kernel-6.0, sasl-3.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From dszoboszlay@REDACTED Mon Feb 18 14:06:44 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Mon, 18 Feb 2019 14:06:44 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: <1549993698.19868.20.camel@erlang.org> References: <1549574710.12116.12.camel@erlang.org> <1549993698.19868.20.camel@erlang.org> Message-ID: Table segments could indeed be responsible for keeping some carriers alive. I deleted 750k objects out of the initial 1M, so I had to hit the shrink limit at one point, but probably not for a second time. And overwriting the contents of the table with an equal number of objects cannot defragment table segment memory, since there's no need to reallocate those. I wish there was a function to force the compaction of an ETS table and maybe even defragment ETS allocators a bit. But I guess it wouldn't be useful for enough people to justify the development cost. It may be easier to simply restart the Erlang node after deleting a lot of data. Thanks again for the insight, Sverker! Daniel On Tue, 12 Feb 2019 at 18:48 Sverker Eriksson wrote: > I think the table segments are whats keeping the carriers alive. > > When a set,bag or duplicate_bag grows new table segments are allocated. > Each new segment contains 2048 hash buckets and the load limit for growth > is 100%. > This means for every 2048 object you insert a new segments i allocated. > The load limit for shrinking is 50%, so after inserting 1 miljon objects > you have to delete 0.5 miljon before the table starts to shrink and > segments are deallocated. > > Increasing the shrink limit will reduce carrier fragmentation in your case, > but it may also cost in performance from more frequent rehashing when > number of objects fluctuates. > > The shrink limit is controlled by > #define SHRINK_LIMIT(NACTIVE) ((NACTIVE) / 2) > in erts/emulator/beam/erl_db_hash.c > > /Sverker > > On l?r, 2019-02-09 at 00:30 +0100, D?niel Szoboszlay wrote: > > Hi Sverker, > > Thanks for the tip, I changed my code in the gist to use erlang:system_info({allocator, > ets_alloc}) and the weirdest things disappeared. (Also, I intentionally > avoided storing binaries in the ETS table in this test, so the > binary_alloc couldn't play a role in the results.) > > But now I see different "problems": > > - Deleting from the ETS table cannot free up any of the carriers. :( > After deleting 75% of the objects I could regain 0 memory for the OS > and the utilisation is down to a disappointing 25%. > - Overwriting every object once with itself sometimes have no effect > at all on the carrier size either. In this case a second round of > overwrites are needed to free up carriers. > - My memory compaction trick can now only achieve 50% utilisation. So > the memory is still fragmented. > - I tried to repeat the overwrite step a few more times, but once it > reaches 50% utilisation it cannot improve on it any more. > > My guess was that maybe carrier abandoning causes this problem. I tried > playing with +MEacul 0, some different +MEas settings and even with +MEramv > true, but neither of them helped. > > So my new questions are: > > - What may be preventing my overwrite-with-self-compactor to go above > 50% carrier utilisation? > - Is there any trick that would help me further reduce the > fragmentation and get back to 90%+ utilisation after deleting a lot of > objects from ETS? > - Wouldn't ERTS benefit from some built-in memory defragmentator > utility, at least for ets_alloc? (For example I don't think eheap_alloc > would need it: the copying GC effectively performs defragmentation > automatically. binary_alloc would also be a potential candidate, but it may > be significantly harder to implement, and I guess most systems store less > binary data than ETS data.) > > Thanks, > Daniel > > On Thu, 7 Feb 2019 at 22:25 Sverker Eriksson wrote: > > Hi D?niel > > I looked at your test code and I think it can be the 'mbcs_pool' stats > that are missing. > > They are returned as {mbcs_pool,[{blocks_size,0}]} without carriers_size > for some reason > by erlang:system_info({*allocator_sizes*,ets_alloc}). > > Use erlang:system_info({*allocator*,ets_alloc}) to get mbcs_pool with > both block and carrier sizes. > > Another thing that might confuse is that all binaries larger than 64 bytes > will be stored in binary_alloc. > > /Sverker > > > On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: > > Hi, > > I would like to understand some things about ETS memory fragmentation > after deleting data. My current (probably faulty) mental model of the issue > looks like this: > > - For every object in an ETS table a block is allocated on a carrier > (typically a multi-block carrier, unless the object is huge). > - Besides the objects themselves, the ETS table obviously needs some > additional blocks too to describe the hash table data structure. The size > of this data shall be small compared to the object data however (since ETS > is not terribly space-inefficient), so I won't think about them any more. > - If I delete some objects from an ETS table, the corresponding blocks > are deallocated. However, the rest of the objects remain in their original > location, so the carriers cannot be deallocated (unless all of their > objects get deleted). > - This implies that deleting a lot of data from ETS tables would lead > to memory fragmentation. > - Since there's no way to force ETS to rearrange the objects it > already stores, the memory remains fragmented until subsequent updates to > ETS tables fill the gaps with new objects. > > I wrote a small test program (available here > ) > to verify my mental model. But it doesn't exactly behave as I expected. > > 1. I create an ETS table and populate it with 1M objects, where each > object is 1027 words large. > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ > 7835 MiB (the size of all other ETS tables on a newly started Erlang node > is negligible). > > And indeed I see that the total block size is ~7881 MiB and the total > carrier size is ~7885 MiB (99.95% utilisation). > 2. I delete 75% of the objects randomly. > > I expect the block size to go down by ~75% and the carrier size with > some smaller value. > > In practice however the block size goes down by 87%, while the carrier > size drops by 48% (resulting in a disappointing 25% utilisation). > 3. Finally, I try to defragment the memory by overwriting each object > that was left in the table with itself. > > I expect this operation to have no effect on the block size, but close > the gap between the block size and carrier size by compacting the blocks on > fewer carriers. > > In practice however the block size goes up by 91%(!!!), while the > carrier size comes down very close to this new block size (utilisation is > back at 99.56%). All in all, compared to the initial state in step 1, both > block and carrier size is down by 75%. > > So here's the list of things I don't understand or know based on this > exercise: > > - How could the block size drop by 87% after deleting 75% of the data > in step 2? > - Why did overwriting each object with itself resulted in almost > doubling the block size? > - Would you consider running a select_replace to compact a table after > deletions safe in production? E.g. doing it on a Mnesia table that's > several GB-s in size and is actively used by Mnesia transactions. (I know > the replace is atomic on each object, but how would a long running replace > affect the execution time of other operations for example?) > - Step 3 helped to reclaim unused memory, but it almost doubled the > used memory (the block size). I don't know what caused this behaviour, but > is there an operation that would achieve the opposite effect? That is, > without altering the contents of the table reduce the block size by 45-50%? > > Thanks, > Daniel > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Feb 18 14:15:15 2019 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 18 Feb 2019 14:15:15 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: <1549574710.12116.12.camel@erlang.org> <1549993698.19868.20.camel@erlang.org> Message-ID: On Mon, Feb 18, 2019 at 2:07 PM D?niel Szoboszlay wrote: > Table segments could indeed be responsible for keeping some carriers > alive. I deleted 750k objects out of the initial 1M, so I had to hit the > shrink limit at one point, but probably not for a second time. And > overwriting the contents of the table with an equal number of objects > cannot defragment table segment memory, since there's no need to reallocate > those. > > I wish there was a function to force the compaction of an ETS table and > maybe even defragment ETS allocators a bit. But I guess it wouldn't be > useful for enough people to justify the development cost. It may be easier > to simply restart the Erlang node after deleting a lot of data. > No need to restart the node, the same effect should be possible to achieve by deleting the table and re-creating it. > > Thanks again for the insight, Sverker! > > Daniel > > On Tue, 12 Feb 2019 at 18:48 Sverker Eriksson wrote: > >> I think the table segments are whats keeping the carriers alive. >> >> When a set,bag or duplicate_bag grows new table segments are allocated. >> Each new segment contains 2048 hash buckets and the load limit for growth >> is 100%. >> This means for every 2048 object you insert a new segments i allocated. >> The load limit for shrinking is 50%, so after inserting 1 miljon objects >> you have to delete 0.5 miljon before the table starts to shrink and >> segments are deallocated. >> >> Increasing the shrink limit will reduce carrier fragmentation in your >> case, >> but it may also cost in performance from more frequent rehashing when >> number of objects fluctuates. >> >> The shrink limit is controlled by >> #define SHRINK_LIMIT(NACTIVE) ((NACTIVE) / 2) >> in erts/emulator/beam/erl_db_hash.c >> >> /Sverker >> >> On l?r, 2019-02-09 at 00:30 +0100, D?niel Szoboszlay wrote: >> >> Hi Sverker, >> >> Thanks for the tip, I changed my code in the gist to use erlang:system_info({allocator, >> ets_alloc}) and the weirdest things disappeared. (Also, I intentionally >> avoided storing binaries in the ETS table in this test, so the >> binary_alloc couldn't play a role in the results.) >> >> But now I see different "problems": >> >> - Deleting from the ETS table cannot free up any of the carriers. :( >> After deleting 75% of the objects I could regain 0 memory for the OS >> and the utilisation is down to a disappointing 25%. >> - Overwriting every object once with itself sometimes have no effect >> at all on the carrier size either. In this case a second round of >> overwrites are needed to free up carriers. >> - My memory compaction trick can now only achieve 50% utilisation. So >> the memory is still fragmented. >> - I tried to repeat the overwrite step a few more times, but once it >> reaches 50% utilisation it cannot improve on it any more. >> >> My guess was that maybe carrier abandoning causes this problem. I tried >> playing with +MEacul 0, some different +MEas settings and even with +MEramv >> true, but neither of them helped. >> >> So my new questions are: >> >> - What may be preventing my overwrite-with-self-compactor to go above >> 50% carrier utilisation? >> - Is there any trick that would help me further reduce the >> fragmentation and get back to 90%+ utilisation after deleting a lot of >> objects from ETS? >> - Wouldn't ERTS benefit from some built-in memory defragmentator >> utility, at least for ets_alloc? (For example I don't think eheap_alloc >> would need it: the copying GC effectively performs defragmentation >> automatically. binary_alloc would also be a potential candidate, but it may >> be significantly harder to implement, and I guess most systems store less >> binary data than ETS data.) >> >> Thanks, >> Daniel >> >> On Thu, 7 Feb 2019 at 22:25 Sverker Eriksson wrote: >> >> Hi D?niel >> >> I looked at your test code and I think it can be the 'mbcs_pool' stats >> that are missing. >> >> They are returned as {mbcs_pool,[{blocks_size,0}]} without carriers_size >> for some reason >> by erlang:system_info({*allocator_sizes*,ets_alloc}). >> >> Use erlang:system_info({*allocator*,ets_alloc}) to get mbcs_pool with >> both block and carrier sizes. >> >> Another thing that might confuse is that all binaries larger than 64 >> bytes will be stored in binary_alloc. >> >> /Sverker >> >> >> On tor, 2019-02-07 at 15:35 +0100, D?niel Szoboszlay wrote: >> >> Hi, >> >> I would like to understand some things about ETS memory fragmentation >> after deleting data. My current (probably faulty) mental model of the issue >> looks like this: >> >> - For every object in an ETS table a block is allocated on a carrier >> (typically a multi-block carrier, unless the object is huge). >> - Besides the objects themselves, the ETS table obviously needs some >> additional blocks too to describe the hash table data structure. The size >> of this data shall be small compared to the object data however (since ETS >> is not terribly space-inefficient), so I won't think about them any more. >> - If I delete some objects from an ETS table, the corresponding >> blocks are deallocated. However, the rest of the objects remain in their >> original location, so the carriers cannot be deallocated (unless all of >> their objects get deleted). >> - This implies that deleting a lot of data from ETS tables would lead >> to memory fragmentation. >> - Since there's no way to force ETS to rearrange the objects it >> already stores, the memory remains fragmented until subsequent updates to >> ETS tables fill the gaps with new objects. >> >> I wrote a small test program (available here >> ) >> to verify my mental model. But it doesn't exactly behave as I expected. >> >> 1. I create an ETS table and populate it with 1M objects, where each >> object is 1027 words large. >> >> I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ >> 7835 MiB (the size of all other ETS tables on a newly started Erlang node >> is negligible). >> >> And indeed I see that the total block size is ~7881 MiB and the total >> carrier size is ~7885 MiB (99.95% utilisation). >> 2. I delete 75% of the objects randomly. >> >> I expect the block size to go down by ~75% and the carrier size with >> some smaller value. >> >> In practice however the block size goes down by 87%, while the >> carrier size drops by 48% (resulting in a disappointing 25% utilisation). >> 3. Finally, I try to defragment the memory by overwriting each object >> that was left in the table with itself. >> >> I expect this operation to have no effect on the block size, but >> close the gap between the block size and carrier size by compacting the >> blocks on fewer carriers. >> >> In practice however the block size goes up by 91%(!!!), while the >> carrier size comes down very close to this new block size (utilisation is >> back at 99.56%). All in all, compared to the initial state in step 1, both >> block and carrier size is down by 75%. >> >> So here's the list of things I don't understand or know based on this >> exercise: >> >> - How could the block size drop by 87% after deleting 75% of the data >> in step 2? >> - Why did overwriting each object with itself resulted in almost >> doubling the block size? >> - Would you consider running a select_replace to compact a table >> after deletions safe in production? E.g. doing it on a Mnesia table that's >> several GB-s in size and is actively used by Mnesia transactions. (I know >> the replace is atomic on each object, but how would a long running replace >> affect the execution time of other operations for example?) >> - Step 3 helped to reclaim unused memory, but it almost doubled the >> used memory (the block size). I don't know what caused this behaviour, but >> is there an operation that would achieve the opposite effect? That is, >> without altering the contents of the table reduce the block size by 45-50%? >> >> Thanks, >> Daniel >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ledest@REDACTED Mon Feb 18 14:41:59 2019 From: ledest@REDACTED (Led) Date: Mon, 18 Feb 2019 15:41:59 +0200 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: Message-ID: Use binary:copy/1 to fill ets tables with fragments of binary stream. ??, 7 ???. 2019 ? 16:36 D?niel Szoboszlay ????: > Hi, > > I would like to understand some things about ETS memory fragmentation > after deleting data. My current (probably faulty) mental model of the issue > looks like this: > > - For every object in an ETS table a block is allocated on a carrier > (typically a multi-block carrier, unless the object is huge). > - Besides the objects themselves, the ETS table obviously needs some > additional blocks too to describe the hash table data structure. The size > of this data shall be small compared to the object data however (since ETS > is not terribly space-inefficient), so I won't think about them any more. > - If I delete some objects from an ETS table, the corresponding blocks > are deallocated. However, the rest of the objects remain in their original > location, so the carriers cannot be deallocated (unless all of their > objects get deleted). > - This implies that deleting a lot of data from ETS tables would lead > to memory fragmentation. > - Since there's no way to force ETS to rearrange the objects it > already stores, the memory remains fragmented until subsequent updates to > ETS tables fill the gaps with new objects. > > I wrote a small test program (available here > ) > to verify my mental model. But it doesn't exactly behave as I expected. > > 1. I create an ETS table and populate it with 1M objects, where each > object is 1027 words large. > > I expect the total ETS memory use to be around 1M * 1027 * 8 bytes ~ > 7835 MiB (the size of all other ETS tables on a newly started Erlang node > is negligible). > > And indeed I see that the total block size is ~7881 MiB and the total > carrier size is ~7885 MiB (99.95% utilisation). > 2. I delete 75% of the objects randomly. > > I expect the block size to go down by ~75% and the carrier size with > some smaller value. > > In practice however the block size goes down by 87%, while the carrier > size drops by 48% (resulting in a disappointing 25% utilisation). > 3. Finally, I try to defragment the memory by overwriting each object > that was left in the table with itself. > > I expect this operation to have no effect on the block size, but close > the gap between the block size and carrier size by compacting the blocks on > fewer carriers. > > In practice however the block size goes up by 91%(!!!), while the > carrier size comes down very close to this new block size (utilisation is > back at 99.56%). All in all, compared to the initial state in step 1, both > block and carrier size is down by 75%. > > So here's the list of things I don't understand or know based on this > exercise: > > - How could the block size drop by 87% after deleting 75% of the data > in step 2? > - Why did overwriting each object with itself resulted in almost > doubling the block size? > - Would you consider running a select_replace to compact a table after > deletions safe in production? E.g. doing it on a Mnesia table that's > several GB-s in size and is actively used by Mnesia transactions. (I know > the replace is atomic on each object, but how would a long running replace > affect the execution time of other operations for example?) > - Step 3 helped to reclaim unused memory, but it almost doubled the > used memory (the block size). I don't know what caused this behaviour, but > is there an operation that would achieve the opposite effect? That is, > without altering the contents of the table reduce the block size by 45-50%? > > Thanks, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Led. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Mon Feb 18 15:06:03 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Mon, 18 Feb 2019 15:06:03 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: <1549574710.12116.12.camel@erlang.org> <1549993698.19868.20.camel@erlang.org> Message-ID: On Mon, 18 Feb 2019 at 14:15 Lukas Larsson wrote: > No need to restart the node, the same effect should be possible to achieve > by deleting the table and re-creating it. > Yes, that's true. I was just thinking that restarting the node is more practical, because that's a scenario all applications shall support. On the other hand re-creating a single ETS table from scratch on a running node is a feature that would have to be developed and tested a lot before doing it in production. Daniel -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon Feb 18 15:16:49 2019 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 18 Feb 2019 15:16:49 +0100 Subject: [erlang-questions] ETS memory fragmentation after deleting data In-Reply-To: References: <1549574710.12116.12.camel@erlang.org> <1549993698.19868.20.camel@erlang.org> Message-ID: On Mon, Feb 18, 2019 at 3:06 PM D?niel Szoboszlay wrote: > On Mon, 18 Feb 2019 at 14:15 Lukas Larsson wrote: > >> No need to restart the node, the same effect should be possible to >> achieve by deleting the table and re-creating it. >> > > Yes, that's true. I was just thinking that restarting the node is more > practical, because that's a scenario all applications shall support. On the > other hand re-creating a single ETS table from scratch on a running node is > a feature that would have to be developed and tested a lot before doing it > in production. > Indeed it does add a bit of testing. We have talked about adding an ets:rename variant that replaces the new name atomically for this type of operations. Though I suppose the new modern world you could implement that yourself by using persistent term. Just be careful with the inherent cost of replacing values in the pts (persistent term store). Though if the alternative is a node restart, it may be worth doing a global GC once a day in a live system. > > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Feb 18 15:21:32 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 18 Feb 2019 15:21:32 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <2471909.jqRKXF7QSM@takoyaki> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> Message-ID: <20190218142132.GA91859@erix.ericsson.se> On Mon, Feb 18, 2019 at 06:34:11PM +0900, zxq9@REDACTED wrote: > On 2019?2?18???? 9?02?11? JST Andreas Schultz wrote: > > Hi Viktor, > > : : > > but why would you use such a thing when you can simply write: > > > > if X > 0 -> pos; > > X < 0 -> neg; > > true -> zero > > end. > > > `if ... true` is a code stink. That is a personal opinion. if ... end selects the clause of the first succesful guard expression, and 'true' is simply a succesful guard expression. > Be explicit in your range tests -- that is what `if` is actually good at: > > if > X > 0 -> pos; > X < 0 -> neg; > X == 0 -> zero > end That is a pretty example that is obviois about which cases it handles, (note that it also handles non-numbers since all terms have an order) but sometimes you instead want it to be obvious that you handle all cases. if is_integer(X), 0 =< X -> non_negative; true -> erlang:error(badarg) end > > If you find yourself *needing* a `true` catchall, you should be writing a function head or `case`. I have no problem using 'if' for that. The 'true' catchall in 'if' is just as much a catchall as the '_' catchall in 'case'. > > > That said... > The original premise of this thread is based on a misunderstanding of `if` and probably inexperience with `case`. Agreed > > > -Craig -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From t6sn7gt@REDACTED Mon Feb 18 16:55:35 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 18 Feb 2019 10:55:35 -0500 Subject: [erlang-questions] Completing the thought (a question about receive) Message-ID: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> In a receive block, will the code 'thought' be completed before things goes 'back' to the beginning of the block, or does another incoming message received before the completion of that thought 'interrupt' the thought.? If I haven't expressed that clearly, consider: ??? receive ??? ??? {_, ComposerList, StartTime, XCoordinate, _, _, Distance, notdone} -> ??????????? some code I want executed ??????????? some code I want executed ??????????? some code I want executed; ??? ??? {Base, _, StartTime, _, _, _, done}?? -> ??????????? some code I want executed ??? end. ============================= SO, will the three lines marked "some code I want executed" be executed always, or does a closely following message interrup it and send things back to the 'receive' (leaving some code perhaps not executed)? (...trying to find a problem...) Thanks. Don From mononcqc@REDACTED Tue Feb 19 02:07:11 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 18 Feb 2019 20:07:11 -0500 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> Message-ID: <20190219010710.GD1196@ferdmbp.local> On 02/18, Donald Steven wrote: >SO, will the three lines marked "some code I want executed" be executed >always, or does a closely following message interrup it and send things >back to the 'receive' (leaving some code perhaps not executed)? > They will execute. A message is only handled when `receive' gets to be able to grab one. From luke@REDACTED Tue Feb 19 02:12:54 2019 From: luke@REDACTED (Luke Bakken) Date: Mon, 18 Feb 2019 17:12:54 -0800 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> Message-ID: Hi Donald, The short answer is yes, the code you want executed is always run. Messages don't preempt already running statements in a process. I was looking through the docs to see where this is explained and this does a decent job -http://erlang.org/doc/getting_started/conc_prog.html On Mon, Feb 18, 2019 at 7:55 AM Donald Steven wrote: > > In a receive block, will the code 'thought' be completed before things > goes 'back' to the beginning of the block, or does another incoming > message received before the completion of that thought 'interrupt' the > thought. If I haven't expressed that clearly, consider: > > receive > > {_, ComposerList, StartTime, XCoordinate, _, _, Distance, > notdone} -> > > some code I want executed > some code I want executed > some code I want executed; > > {Base, _, StartTime, _, _, _, done} -> > > some code I want executed > > end. > > ============================= > > SO, will the three lines marked "some code I want executed" be executed > always, or does a closely following message interrup it and send things > back to the 'receive' (leaving some code perhaps not executed)? > > (...trying to find a problem...) > > Thanks. > > Don > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Mon Feb 18 19:45:49 2019 From: hugo@REDACTED (Hugo Mills) Date: Mon, 18 Feb 2019 18:45:49 +0000 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> Message-ID: <20190218184549.GD317@carfax.org.uk> On Mon, Feb 18, 2019 at 10:55:35AM -0500, Donald Steven wrote: > In a receive block, will the code 'thought' be completed before > things goes 'back' to the beginning of the block, or does another > incoming message received before the completion of that thought > 'interrupt' the thought. Messages are processed singly and sequentially by any one process. Each process has a message queue, and the only time that a message comes off that queue is when receive is executed. The receive takes the first matching message off the queue and then runs the code for the branch matched. Only when another receive is encountered will the queue be looked at again. So, basically, there's no pre-emption of the code within a process -- each process runs its code linearly and sequentially without interruption. HTH, Hugo. >? If I haven't expressed that clearly, > consider: > > ??? receive > > ??? ??? {_, ComposerList, StartTime, XCoordinate, _, _, Distance, > notdone} -> > > ??????????? some code I want executed > ??????????? some code I want executed > ??????????? some code I want executed; > > ??? ??? {Base, _, StartTime, _, _, _, done}?? -> > > ??????????? some code I want executed > > ??? end. > > ============================= > > SO, will the three lines marked "some code I want executed" be > executed always, or does a closely following message interrup it and > send things back to the 'receive' (leaving some code perhaps not > executed)? > > (...trying to find a problem...) > > Thanks. > > Don > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | We demand rigidly defined areas of doubt and hugo@REDACTED carfax.org.uk | uncertainty! http://carfax.org.uk/ | Vroomfondel PGP: E2AB1DE4 | The Hitch-Hikers Guide to the Galaxy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From viktor@REDACTED Mon Feb 18 21:08:44 2019 From: viktor@REDACTED (=?UTF-8?Q?Viktor_S=c3=b6derqvist?=) Date: Mon, 18 Feb 2019 21:08:44 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <20190218142132.GA91859@erix.ericsson.se> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> Message-ID: <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> Hi! My example proved silly. Of course, normal 'if' is fine for guard expressions and catch-all is to be avoided in general. I agree. What I failed to mention was that you can use non-guard expressions with this trick. (The trick's actually rewriting the elseifs to nested case.) When using if with non-guard expressions, you need to evaluate them in advance, which you may not want if these are side-effectful or just slow: f() -> Cond1 = g1(), Cond2 = g2(), Cond3 = g3(), if Cond1 -> a; Cond2 -> b; Cond3 -> c; true -> d end. Therefore, you often see nested case expressions instead: f() -> case g1() of true -> a; false -> case g2() of true -> b; false -> case g3() of true -> c; false -> d end end end. ... or broken down into multiple functions: f() -> case g1() of true -> a; false -> f2() end. f2() -> %% You know There's the throw style too, a slightly silly but a flat style: f() -> try g1() andalso throw(a); g2() andalso throw(b); g3() andalso throw(c); d catch X -> X end. The point of the ?elseif syntax trick is that it lets you write the nested case above as the flat structure: f() -> ?'if'(g1()) -> a; ?elseif(g2()) -> b; ?elseif(g3()) -> c; ?else -> d. You can even bind variables in the conditions if you would want that, just to avoid deeply nested code. I would look more like perl than erlang though, so don't do that. >> That said... >> The original premise of this thread is based on a misunderstanding of `if` and probably inexperience with `case`. > > Agreed > I agree too. :-) Thanks for clarifying! Btw, I'm sorry for posting without having a real question. I just wanted to show what can be done using a non-trivial parse transform. Viktor From t6sn7gt@REDACTED Mon Feb 18 23:38:15 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 18 Feb 2019 17:38:15 -0500 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <20190218184549.GD317@carfax.org.uk> References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> <20190218184549.GD317@carfax.org.uk> Message-ID: Got it!? Thanks Hugo, much appreciated. Don On 2/18/2019 1.45 PM, Hugo Mills wrote: > On Mon, Feb 18, 2019 at 10:55:35AM -0500, Donald Steven wrote: >> In a receive block, will the code 'thought' be completed before >> things goes 'back' to the beginning of the block, or does another >> incoming message received before the completion of that thought >> 'interrupt' the thought. > Messages are processed singly and sequentially by any one > process. Each process has a message queue, and the only time that a > message comes off that queue is when receive is executed. The receive > takes the first matching message off the queue and then runs the code > for the branch matched. Only when another receive is encountered will > the queue be looked at again. > > So, basically, there's no pre-emption of the code within a process > -- each process runs its code linearly and sequentially without > interruption. > > HTH, > Hugo. > >> ? If I haven't expressed that clearly, >> consider: >> >> ??? receive >> >> ??? ??? {_, ComposerList, StartTime, XCoordinate, _, _, Distance, >> notdone} -> >> >> ??????????? some code I want executed >> ??????????? some code I want executed >> ??????????? some code I want executed; >> >> ??? ??? {Base, _, StartTime, _, _, _, done}?? -> >> >> ??????????? some code I want executed >> >> ??? end. >> >> ============================= >> >> SO, will the three lines marked "some code I want executed" be >> executed always, or does a closely following message interrup it and >> send things back to the 'receive' (leaving some code perhaps not >> executed)? >> >> (...trying to find a problem...) >> >> Thanks. >> >> Don >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From sdl.web@REDACTED Tue Feb 19 10:45:32 2019 From: sdl.web@REDACTED (Leo Liu) Date: Tue, 19 Feb 2019 17:45:32 +0800 Subject: [erlang-questions] how to stop rebar3 from recursively compile some directories? Message-ID: Hi there, I have a common test data directory that I would like to opt out of this recursion. Based on https://github.com/erlang/rebar3/pull/1382 I have tried {profiles, [{test, [ {extra_src_dirs, [{"test", [{recursive, false}]}]} ]}]}. and {extra_src_dirs, [{"test", [{recursive, false}]}]}. neither worked. Any suggestions? Thanks. -- Leo From gomoripeti@REDACTED Tue Feb 19 10:48:22 2019 From: gomoripeti@REDACTED (=?UTF-8?B?UGV0aSBHw7Ztw7ZyaQ==?=) Date: Tue, 19 Feb 2019 10:48:22 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> Message-ID: This makes much more sense now :) Just noting that Elixir has the `cond` expression for this use case. On Tue, Feb 19, 2019 at 8:37 AM Viktor S?derqvist wrote: > Hi! > > My example proved silly. Of course, normal 'if' is fine for guard > expressions and catch-all is to be avoided in general. I agree. > > What I failed to mention was that you can use non-guard expressions with > this trick. (The trick's actually rewriting the elseifs to nested case.) > > When using if with non-guard expressions, you need to evaluate them in > advance, which you may not want if these are side-effectful or just slow: > > f() -> > Cond1 = g1(), > Cond2 = g2(), > Cond3 = g3(), > if Cond1 -> a; > Cond2 -> b; > Cond3 -> c; > true -> d > end. > > Therefore, you often see nested case expressions instead: > > f() -> > case g1() of > true -> a; > false -> > case g2() of > true -> b; > false -> > case g3() of > true -> c; > false -> d > end > end > end. > > ... or broken down into multiple functions: > > f() -> > case g1() of > true -> a; > false -> f2() > end. > f2() -> > %% You know > > There's the throw style too, a slightly silly but a flat style: > > f() -> > try > g1() andalso throw(a); > g2() andalso throw(b); > g3() andalso throw(c); > d > catch > X -> X > end. > > The point of the ?elseif syntax trick is that it lets you write the > nested case above as the flat structure: > > f() -> > ?'if'(g1()) -> a; > ?elseif(g2()) -> b; > ?elseif(g3()) -> c; > ?else -> d. > > You can even bind variables in the conditions if you would want that, > just to avoid deeply nested code. I would look more like perl than > erlang though, so don't do that. > > >> That said... > >> The original premise of this thread is based on a misunderstanding of > `if` and probably inexperience with `case`. > > > > Agreed > > > > I agree too. :-) Thanks for clarifying! > > Btw, I'm sorry for posting without having a real question. I just wanted > to show what can be done using a non-trivial parse transform. > > Viktor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Feb 19 10:51:24 2019 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 19 Feb 2019 11:51:24 +0200 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> Message-ID: <427C2A91-7A67-4692-9BA7-75097C8D4656@gmail.com> Hello Viktor, You might look into my solution where this concept is generally solved under the abstraction of category pattern. https://github.com/fogfish/datum/blob/master/doc/category.md Best Regards, Dmitry > On 19 Feb 2019, at 11.48, Peti G?m?ri wrote: > > This makes much more sense now :) Just noting that Elixir has the `cond` expression for this use case. > > On Tue, Feb 19, 2019 at 8:37 AM Viktor S?derqvist > wrote: > Hi! > > My example proved silly. Of course, normal 'if' is fine for guard > expressions and catch-all is to be avoided in general. I agree. > > What I failed to mention was that you can use non-guard expressions with > this trick. (The trick's actually rewriting the elseifs to nested case.) > > When using if with non-guard expressions, you need to evaluate them in > advance, which you may not want if these are side-effectful or just slow: > > f() -> > Cond1 = g1(), > Cond2 = g2(), > Cond3 = g3(), > if Cond1 -> a; > Cond2 -> b; > Cond3 -> c; > true -> d > end. > > Therefore, you often see nested case expressions instead: > > f() -> > case g1() of > true -> a; > false -> > case g2() of > true -> b; > false -> > case g3() of > true -> c; > false -> d > end > end > end. > > ... or broken down into multiple functions: > > f() -> > case g1() of > true -> a; > false -> f2() > end. > f2() -> > %% You know > > There's the throw style too, a slightly silly but a flat style: > > f() -> > try > g1() andalso throw(a); > g2() andalso throw(b); > g3() andalso throw(c); > d > catch > X -> X > end. > > The point of the ?elseif syntax trick is that it lets you write the > nested case above as the flat structure: > > f() -> > ?'if'(g1()) -> a; > ?elseif(g2()) -> b; > ?elseif(g3()) -> c; > ?else -> d. > > You can even bind variables in the conditions if you would want that, > just to avoid deeply nested code. I would look more like perl than > erlang though, so don't do that. > > >> That said... > >> The original premise of this thread is based on a misunderstanding of `if` and probably inexperience with `case`. > > > > Agreed > > > > I agree too. :-) Thanks for clarifying! > > Btw, I'm sorry for posting without having a real question. I just wanted > to show what can be done using a non-trivial parse transform. > > Viktor > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Tue Feb 19 12:23:57 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 19 Feb 2019 12:23:57 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> Message-ID: On Tue, Feb 19, 2019 at 8:37 AM Viktor S?derqvist wrote: > The point of the ?elseif syntax trick is that it lets you write the > nested case above as the flat structure: > > f() -> > ?'if'(g1()) -> a; > ?elseif(g2()) -> b; > ?elseif(g3()) -> c; > ?else -> d. So you've reimplemented LISP's COND. Please just do it properly in the Erlang parser (I belive 'cond' is already a reserved word), and write an EEP for it. FWIW, my code tends to be CASE heavy largely to avoid the inherent problems of Erlang's limited guard expressions and butt-ugly "if". From mononcqc@REDACTED Tue Feb 19 20:37:12 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 19 Feb 2019 14:37:12 -0500 Subject: [erlang-questions] how to stop rebar3 from recursively compile some directories? In-Reply-To: References: Message-ID: it's possible that this functionality broke if it wasn't tested a whole lot. It appears we had tests for src_dirs being recursive (which still appear to work), but no tests for extra_src_dirs and we might have broken these through a refactoring for newer compiler changes. You may want to open an issue for this. On Tue, Feb 19, 2019 at 4:45 AM Leo Liu wrote: > Hi there, > > I have a common test data directory that I would like to opt out of this > recursion. Based on https://github.com/erlang/rebar3/pull/1382 I have > tried > > {profiles, > [{test, [ > {extra_src_dirs, [{"test", [{recursive, false}]}]} > ]}]}. > > and > > {extra_src_dirs, [{"test", [{recursive, false}]}]}. > > neither worked. Any suggestions? Thanks. -- Leo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Tue Feb 19 21:15:14 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Tue, 19 Feb 2019 21:15:14 +0100 Subject: [erlang-questions] Use systemd's watchdog functionality instead of heart? Message-ID: One of our devops people recently commented that heart made management of beam VMs in systemd-enabled systems more complicated. I investigated a bit, and found that systemd supports being a watchdog for a service, via sd_notify() calls with WATCHDOG=1 messages, and appropriate configuration in the unit file. Interestingly enough, there _is_ systemd support in OTP, but only in epmd. Is there a good reason _not_ to use systemd's watchdog functionality instead of heart, where available? /Mikael From thomas.elsgaard@REDACTED Tue Feb 19 21:23:49 2019 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Tue, 19 Feb 2019 21:23:49 +0100 Subject: [erlang-questions] Use systemd's watchdog functionality instead of heart? In-Reply-To: References: Message-ID: Mikael, You can also look at: https://gist.github.com/maxlapshin/01773f0fca706acdcb4acb77d91d78bb Thomas tir. 19. feb. 2019 kl. 21.15 skrev Mikael Pettersson : > One of our devops people recently commented that heart made management > of beam VMs in systemd-enabled systems more complicated. > > I investigated a bit, and found that systemd supports being a watchdog > for a service, via sd_notify() calls with WATCHDOG=1 messages, and > appropriate configuration in the unit file. > > Interestingly enough, there _is_ systemd support in OTP, but only in epmd. > > Is there a good reason _not_ to use systemd's watchdog functionality > instead of heart, where available? > > /Mikael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Wed Feb 20 01:32:48 2019 From: sdl.web@REDACTED (Leo Liu) Date: Wed, 20 Feb 2019 08:32:48 +0800 Subject: [erlang-questions] how to stop rebar3 from recursively compile some directories? References: Message-ID: On 2019-02-19 14:37 -0500, Fred Hebert wrote: > it's possible that this functionality broke if it wasn't tested a whole > lot. It appears we had tests for src_dirs being recursive (which still > appear to work), but no tests for extra_src_dirs and we might have broken > these through a refactoring for newer compiler changes. You may want to > open an issue for this. Thanks for the info. See https://github.com/erlang/rebar3/issues/2023. From kushal@REDACTED Mon Feb 18 18:08:23 2019 From: kushal@REDACTED (Kushal Kumaran) Date: Mon, 18 Feb 2019 09:08:23 -0800 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> (Donald Steven's message of "Mon, 18 Feb 2019 10:55:35 -0500") References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> Message-ID: <8736olaut4.fsf@copper.locationd.net> Hi Donald, Donald Steven writes: > In a receive block, will the code 'thought' be completed before things > goes 'back' to the beginning of the block, or does another incoming > message received before the completion of that thought 'interrupt' the > thought.? If I haven't expressed that clearly, consider: > You appear to interpret receive as some kind of a loop construct. It is not. A receive statement receives at most one message that matches one of the patterns. Messages received by the process are held in the mailbox until a receive statement is executed. There is no interruption of execution by an incoming message. After the code in the matching block (or timeout) is executed, the receive statement returns with the value of the executed block, and execution continues at the statement following the receive. > ??? receive > > ??? ??? {_, ComposerList, StartTime, XCoordinate, _, _, Distance, > notdone} -> > > ??????????? some code I want executed > ??????????? some code I want executed > ??????????? some code I want executed; > > ??? ??? {Base, _, StartTime, _, _, _, done}?? -> > > ??????????? some code I want executed > > ??? end. > > ============================= > > SO, will the three lines marked "some code I want executed" be > executed always, or does a closely following message interrup it and > send things back to the 'receive' (leaving some code perhaps not > executed)? > > (...trying to find a problem...) You might have to check if some of the code there is crashing the process. That will result in some code not being executed. The closely following message you talk of will be held in the process mailbox until the process next calls receive. -- regards, kushal From johnhommer@REDACTED Mon Feb 18 19:20:33 2019 From: johnhommer@REDACTED (andrei zavada) Date: Mon, 18 Feb 2019 20:20:33 +0200 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <20190218142132.GA91859@erix.ericsson.se> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> Message-ID: One neat (for some definition of 'neat') trick to do away with `true` for the semantic `else` is `el/=se` (spotted in riak code). On Mon, 18 Feb 2019 at 16:36, Raimo Niskanen wrote: > > On Mon, Feb 18, 2019 at 06:34:11PM +0900, zxq9@REDACTED wrote: > > On 2019?2?18???? 9?02?11? JST Andreas Schultz wrote: > > > Hi Viktor, > > > > : : > > > but why would you use such a thing when you can simply write: > > > > > > if X > 0 -> pos; > > > X < 0 -> neg; > > > true -> zero > > > end. > > > > > > `if ... true` is a code stink. > > That is a personal opinion. > > if ... end selects the clause of the first succesful guard expression, > and 'true' is simply a succesful guard expression. > > > Be explicit in your range tests -- that is what `if` is actually good at: > > > > if > > X > 0 -> pos; > > X < 0 -> neg; > > X == 0 -> zero > > end > > That is a pretty example that is obviois about which cases it handles, > (note that it also handles non-numbers since all terms have an order) > but sometimes you instead want it to be obvious that you handle all cases. > > if > is_integer(X), 0 =< X -> non_negative; > true -> erlang:error(badarg) > end > > > > > If you find yourself *needing* a `true` catchall, you should be writing a function head or `case`. > > I have no problem using 'if' for that. The 'true' catchall in 'if' is just > as much a catchall as the '_' catchall in 'case'. > > > > > > > > That said... > > The original premise of this thread is based on a misunderstanding of `if` and probably inexperience with `case`. > > Agreed > > > > > > > -Craig > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From scott.finnie@REDACTED Mon Feb 18 20:23:41 2019 From: scott.finnie@REDACTED (sfinnie) Date: Mon, 18 Feb 2019 19:23:41 +0000 Subject: [erlang-questions] request: sense-check proposed approach Message-ID: <7c7aaee5-aa32-2e7f-692e-6fb47a1fb93d@gmail.com> Folks, I'm about to start a new app.? I'd be grateful if someone would run an eye over proposed approach below and highlight anything you think is suspect/better approach available/whatever. * Web application: ??? * Erlang-based REST services on the back end ??? * Front end mostly static html/css with a little Elm* for interaction. * Cowboy as the means to deliver REST services * In the spirit of keeping things simple, I'm thinking of using Cowboy to serve all static files (inc html, css and compiled js from the Elm app).? However, Cowboy docs say this is "development only".? Is Cowboy static routing a no-go in production, or is this more symptomatic of scalability?? The app won't have high load (3-4 users). * Cowboy uses erlang.mk instead of rebar3.? Is there a way to convert to rebar3?? Is it worth the hassle?? Couldn't find anything recent in searching the web. * I'd like to use sync:go() (https://github.com/rustyio/sync) or similar for automatically rebuilding.? I've tried incorporating into a simple cowboy app.? It loads and recompiles changes OK, but doesn't seem to reload into the currently-running system.? Rel setup below. Any comments/suggestions appreciated. Thanks, Scott. -- * I'm intrigued by the Elm language/architecture so want to explore in a small but meaningful example. Relevant setup as follows. relx-dev.config: {dev_mode, true}. {lib_dirs, ["/home/scoot/ERL_LIBS"]}. {release, ?{fmt_dev, "0.0.1"}, ?[fmt, jsx, sasl, syntax_tools, compiler, sync, runtime_tools] }. {extended_start_script, true}. {sys_config, "config/sys.config"}. {vm_args, "config/vm.args"}. Release built using ".erlang.mk/relx -c relx-dev.config" and run with "_rel/fmt_dev/bin/fmt_dev console" From mikpelinux@REDACTED Wed Feb 20 10:01:09 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Wed, 20 Feb 2019 10:01:09 +0100 Subject: [erlang-questions] Use systemd's watchdog functionality instead of heart? In-Reply-To: References: Message-ID: On Tue, Feb 19, 2019 at 9:24 PM Thomas Elsgaard wrote: > > Mikael, You can also look at: > > https://gist.github.com/maxlapshin/01773f0fca706acdcb4acb77d91d78bb > > Thomas > > tir. 19. feb. 2019 kl. 21.15 skrev Mikael Pettersson : >> >> One of our devops people recently commented that heart made management >> of beam VMs in systemd-enabled systems more complicated. >> >> I investigated a bit, and found that systemd supports being a watchdog >> for a service, via sd_notify() calls with WATCHDOG=1 messages, and >> appropriate configuration in the unit file. >> >> Interestingly enough, there _is_ systemd support in OTP, but only in epmd. >> >> Is there a good reason _not_ to use systemd's watchdog functionality >> instead of heart, where available? It's quite neat that one can message systemd directly from Erlang, but the basic question remains: is there any technical reason to keep using heart as a watchdog when one has systemd? From hugo@REDACTED Wed Feb 20 11:59:32 2019 From: hugo@REDACTED (Hugo Mills) Date: Wed, 20 Feb 2019 10:59:32 +0000 Subject: [erlang-questions] Odd behaviour of code:priv_dir/1 In-Reply-To: <20190217184756.GC317@carfax.org.uk> References: <20190217184756.GC317@carfax.org.uk> Message-ID: <20190220105932.GE317@carfax.org.uk> Ping? Anyone got any ideas at all how to go about diagnosing this? Hugo. On Sun, Feb 17, 2019 at 06:47:56PM +0000, Hugo Mills wrote: > I've been trying to set up a basic CI system for a few projects. > The problem is, I'm consistently getting test failures when running > tests through the CI, where those tests pass perfectly well on a > manual run. > > The test failure is: > > === Ended at 2019-02-17 15:36:20 > === Location: [{filename,join,446}, > {stdlib_SUITE,run_suite,37}, > {test_server,ts_tc,1545}, > {test_server,run_test_case_eval1,1063}, > {test_server,run_test_case_eval,995}] > === Reason: no function clause matching > filename:join({error,bad_name},"stdlib") (filename.erl, line 446) > in function stdlib_SUITE:run_suite/3 (test/stdlib_SUITE.erl, line 37) > in call from test_server:ts_tc/3 (test_server.erl, line 1545) > in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1063) > in call from test_server:run_test_case_eval/9 (test_server.erl, line 995) > > > which comes from this line of code in my CT text fixture: > > BaseContext = > #{"$module_path" > => [filename:join(code:priv_dir(childe), "stdlib")], > "$data_path" > => [?config(data_dir, Conf)]}, > > I can run this on my main dev machines ("make ct" -- I'm using > erlang.mk) quite happily, and it passes the test. I can run it > manually on the CI worker machine and it passes the test. If I run it > through a CI system, it fails with the above error. > > There's clearly _something_ odd about the environment that the CI > system is running in, but I'm damned if I can tell what it is. It's > not specific to the CI system, either: I've seen this with Buildbot, > yesterday, and now GoCD today. > > Why would code:priv_dir/1 be failing in the CI when it's working in > an interactive shell? > > My dev machines are running 21.2.2 from Debian; the CI is running > 21.2.5 from Erlang Solutions, but that's not the issue, because I can > do a manual run on the CI box, and it works. > > Thanks for any ideas, > Hugo. > -- Hugo Mills | "It was half way to Rivendell when the drugs began hugo@REDACTED carfax.org.uk | to take hold" http://carfax.org.uk/ | Hunter S Tolkien PGP: E2AB1DE4 | Fear and Loathing in Barad D?r -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From raimo+erlang-questions@REDACTED Wed Feb 20 12:01:33 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 20 Feb 2019 12:01:33 +0100 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> Message-ID: <20190220110133.GA86084@erix.ericsson.se> On Mon, Feb 18, 2019 at 08:20:33PM +0200, andrei zavada wrote: > One neat (for some definition of 'neat') trick to do away with `true` > for the semantic `else` is `el/=se` (spotted in riak code). > Oh dear! That's creative, actually quite readable, and should compile to literal 'true'. You can also use a macro: -define(else, true). -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From magnus@REDACTED Wed Feb 20 12:38:09 2019 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 20 Feb 2019 11:38:09 +0000 Subject: [erlang-questions] Odd behaviour of code:priv_dir/1 In-Reply-To: <20190220105932.GE317@carfax.org.uk> References: <20190217184756.GC317@carfax.org.uk> <20190220105932.GE317@carfax.org.uk> Message-ID: <070758b1-623d-2637-d881-2b84db9ca91e@gameanalytics.com> Code paths, most likely. Print the value of code:get_path() just before the error and see if it contains "/path/to/childe/ebin". IIRC, it is _not_ enough for the code path to contain "./ebin", even if this would point to the correct directory. The name of the application must be literally present in the directory name in the code path in order for code:priv_dir to find the application. Regards, Magnus On 20/02/2019 10:59, Hugo Mills wrote: > Ping? Anyone got any ideas at all how to go about diagnosing this? > > Hugo. > > On Sun, Feb 17, 2019 at 06:47:56PM +0000, Hugo Mills wrote: >> I've been trying to set up a basic CI system for a few projects. >> The problem is, I'm consistently getting test failures when running >> tests through the CI, where those tests pass perfectly well on a >> manual run. >> >> The test failure is: >> >> === Ended at 2019-02-17 15:36:20 >> === Location: [{filename,join,446}, >> {stdlib_SUITE,run_suite,37}, >> {test_server,ts_tc,1545}, >> {test_server,run_test_case_eval1,1063}, >> {test_server,run_test_case_eval,995}] >> === Reason: no function clause matching >> filename:join({error,bad_name},"stdlib") (filename.erl, line 446) >> in function stdlib_SUITE:run_suite/3 (test/stdlib_SUITE.erl, line 37) >> in call from test_server:ts_tc/3 (test_server.erl, line 1545) >> in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1063) >> in call from test_server:run_test_case_eval/9 (test_server.erl, line 995) >> >> >> which comes from this line of code in my CT text fixture: >> >> BaseContext = >> #{"$module_path" >> => [filename:join(code:priv_dir(childe), "stdlib")], >> "$data_path" >> => [?config(data_dir, Conf)]}, >> >> I can run this on my main dev machines ("make ct" -- I'm using >> erlang.mk) quite happily, and it passes the test. I can run it >> manually on the CI worker machine and it passes the test. If I run it >> through a CI system, it fails with the above error. >> >> There's clearly _something_ odd about the environment that the CI >> system is running in, but I'm damned if I can tell what it is. It's >> not specific to the CI system, either: I've seen this with Buildbot, >> yesterday, and now GoCD today. >> >> Why would code:priv_dir/1 be failing in the CI when it's working in >> an interactive shell? >> >> My dev machines are running 21.2.2 from Debian; the CI is running >> 21.2.5 from Erlang Solutions, but that's not the issue, because I can >> do a manual run on the CI box, and it works. >> >> Thanks for any ideas, >> Hugo. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Wed Feb 20 13:54:08 2019 From: hugo@REDACTED (Hugo Mills) Date: Wed, 20 Feb 2019 12:54:08 +0000 Subject: [erlang-questions] Odd behaviour of code:priv_dir/1 In-Reply-To: <070758b1-623d-2637-d881-2b84db9ca91e@gameanalytics.com> References: <20190217184756.GC317@carfax.org.uk> <20190220105932.GE317@carfax.org.uk> <070758b1-623d-2637-d881-2b84db9ca91e@gameanalytics.com> Message-ID: <20190220125408.GF317@carfax.org.uk> On Wed, Feb 20, 2019 at 11:38:09AM +0000, Magnus Henoch wrote: > Code paths, most likely. Print the value of code:get_path() just before > the error and see if it contains "/path/to/childe/ebin". > > IIRC, it is _not_ enough for the code path to contain "./ebin", even if > this would point to the correct directory. The name of the application > must be literally present in the directory name in the code path in > order for code:priv_dir to find the application. That was exactly it. It was checking out into a directory called "childe_code", because that was the name of the pipeline I'd given it. Shift the checkout one layer deeper, into "childe_code/childe/", and it all works. Thank you. I'd never have got that by myself. Hugo. > Regards, > Magnus > > On 20/02/2019 10:59, Hugo Mills wrote: > > Ping? Anyone got any ideas at all how to go about diagnosing this? > > > > Hugo. > > > > On Sun, Feb 17, 2019 at 06:47:56PM +0000, Hugo Mills wrote: > >> I've been trying to set up a basic CI system for a few projects. > >> The problem is, I'm consistently getting test failures when running > >> tests through the CI, where those tests pass perfectly well on a > >> manual run. > >> > >> The test failure is: > >> > >> === Ended at 2019-02-17 15:36:20 > >> === Location: [{filename,join,446}, > >> {stdlib_SUITE,run_suite,37}, > >> {test_server,ts_tc,1545}, > >> {test_server,run_test_case_eval1,1063}, > >> {test_server,run_test_case_eval,995}] > >> === Reason: no function clause matching > >> filename:join({error,bad_name},"stdlib") (filename.erl, line 446) > >> in function stdlib_SUITE:run_suite/3 (test/stdlib_SUITE.erl, line 37) > >> in call from test_server:ts_tc/3 (test_server.erl, line 1545) > >> in call from test_server:run_test_case_eval1/6 (test_server.erl, line 1063) > >> in call from test_server:run_test_case_eval/9 (test_server.erl, line 995) > >> > >> > >> which comes from this line of code in my CT text fixture: > >> > >> BaseContext = > >> #{"$module_path" > >> => [filename:join(code:priv_dir(childe), "stdlib")], > >> "$data_path" > >> => [?config(data_dir, Conf)]}, > >> > >> I can run this on my main dev machines ("make ct" -- I'm using > >> erlang.mk) quite happily, and it passes the test. I can run it > >> manually on the CI worker machine and it passes the test. If I run it > >> through a CI system, it fails with the above error. > >> > >> There's clearly _something_ odd about the environment that the CI > >> system is running in, but I'm damned if I can tell what it is. It's > >> not specific to the CI system, either: I've seen this with Buildbot, > >> yesterday, and now GoCD today. > >> > >> Why would code:priv_dir/1 be failing in the CI when it's working in > >> an interactive shell? > >> > >> My dev machines are running 21.2.2 from Debian; the CI is running > >> 21.2.5 from Erlang Solutions, but that's not the issue, because I can > >> do a manual run on the CI box, and it works. > >> > >> Thanks for any ideas, > >> Hugo. > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | To an Englishman, 100 miles is a long way; to an hugo@REDACTED carfax.org.uk | American, 100 years is a long time. http://carfax.org.uk/ | PGP: E2AB1DE4 | Earle Hitchner -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From t6sn7gt@REDACTED Wed Feb 20 14:16:41 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 20 Feb 2019 08:16:41 -0500 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2471909.jqRKXF7QSM@takoyaki> <20190218142132.GA91859@erix.ericsson.se> <644d7627-b7bc-6715-378e-553f5daa3147@zuiderkwast.se> Message-ID: <2efb05b3-4b87-d0e1-bd17-ebbfb5630998@aim.com> I don't think: -define(true,else). yielding ?else -> is much of an improvement. Why not just allow the keyword else as a synonym for true when its usage would improve the readability? (At times, "true" reminds me of trying to read (SAT-like) sentences like: "Whenever he couldn't think of whether or not he might have forgotten to turn the light on or off, or not.") On 2/19/2019 6.23 AM, Mikael Pettersson wrote: > On Tue, Feb 19, 2019 at 8:37 AM Viktor S?derqvist wrote: >> The point of the ?elseif syntax trick is that it lets you write the >> nested case above as the flat structure: >> >> f() -> >> ?'if'(g1()) -> a; >> ?elseif(g2()) -> b; >> ?elseif(g3()) -> c; >> ?else -> d. > So you've reimplemented LISP's COND. Please just do it properly in > the Erlang parser (I belive 'cond' is already a reserved word), and > write an EEP for it. > > FWIW, my code tends to be CASE heavy largely to avoid the inherent > problems of Erlang's limited guard expressions and butt-ugly "if". > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jstimpson@REDACTED Wed Feb 20 20:23:34 2019 From: jstimpson@REDACTED (Jesse Stimpson) Date: Wed, 20 Feb 2019 14:23:34 -0500 Subject: [erlang-questions] Error report printed to stdout when using remsh to a tls dist cluster In-Reply-To: References: Message-ID: Sorry to bump this topic, but does anyone have any advice on where to go from here? Is this deserving of a bug report to the Erlang/OTP tracker? Thanks! On Tue, Feb 12, 2019 at 2:21 PM Jesse Stimpson < jstimpson@REDACTED> wrote: > Hello, > > I've set up a distribution of 2 Erlang nodes, using "-proto_dist > inet_tls", and when I connect to one of the nodes with remsh, I receive the > following error report: > > Erlang/OTP 21 [erts-10.2.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] > [async-threads:1] [hipe] > > Eshell V10.2.3 (abort with ^G) > (ssl_test2@REDACTED)1> =ERROR REPORT==== 12-Feb-2019::14:17:25.653872 === > Error in process <0.68.0> on node 'ssl_test_remsh@REDACTED' with exit > value: > {{badmatch,{error,closed}}, > [{inet_tls_dist,accept_loop,4,[{file,"inet_tls_dist.erl"},{line,253}]}]} > > The error appears to be benign; I don't observe any undesirable behavior > from the Erlang nodes so far. It's also inconsistently seen; it takes a > handful of new remsh attempts to produce the error. > > I've pinpointed the 21.2 release as the culprit, but there are many > commits between 21.1.4 and 21.2, so it's difficult for me to isolate. > > To reproduce: set up 2 nodes with proto_dist inet_tls (call them A and B). > Have B issue a ping to A. Then set up a remsh to B. Close and repeat the > remsh until the error appears. From what I have seen this is the minimal > set of steps to reproduce. When just using 1 node with a remsh, I've not > seen the error. > > Thanks, > Jesse Stimpson > > -- Jesse Stimpson Site Reliability Engineering m: 9199950424 <(919)%20995-0424> RepublicWireless.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Feb 21 06:43:11 2019 From: zxq9@REDACTED (zxq9@REDACTED) Date: Thu, 21 Feb 2019 14:43:11 +0900 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <2efb05b3-4b87-d0e1-bd17-ebbfb5630998@aim.com> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2efb05b3-4b87-d0e1-bd17-ebbfb5630998@aim.com> Message-ID: <4055811.6yH3tYjjUC@takoyaki> On 2019?2?20???? 8?16?41? JST Donald Steven wrote: > I don't think: > > -define(true,else). > > yielding ?else -> > > is much of an improvement. > > Why not just allow the keyword else as a synonym for true when its usage > would improve the readability? (At times, "true" reminds me of trying to > read (SAT-like) sentences like: "Whenever he couldn't think of whether > or not he might have forgotten to turn the light on or off, or not.") Why not just use `if` where it fits semantically? These periodic discussions of how to use silly cheats to avoid proper code layout always puzzle me. -Craig From raoknz@REDACTED Thu Feb 21 08:28:09 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Thu, 21 Feb 2019 20:28:09 +1300 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: <4055811.6yH3tYjjUC@takoyaki> References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <2efb05b3-4b87-d0e1-bd17-ebbfb5630998@aim.com> <4055811.6yH3tYjjUC@takoyaki> Message-ID: Everyone wants to be a language designer. If we're lucky, we get Elixir or LFE. Usually we get a bit of fiddling around the edges, which is a good thing if it lets us express something more clearly and with fewer mistakes than before. Things like el/=se just increase the burden on the reader. I for one would rather read for (;;) { ... } than #define FOREVER for (;;) in some header ... FOREVER { ... } This whole thing with 'if' has simmered along for years. There was a proposal for 'cond'. Perhaps it's time to dust it off and make it an official part of the language. Or perhaps not: there's a message in my mailbox from a software engineering list with the title "Try programming without 'if' statements" which I have not read yet... Seriously, though, when you find yourself struggling with syntax in Erlang, you probably need to break your code up into smaller functions. On Thu, 21 Feb 2019 at 18:43, wrote: > On 2019?2?20???? 8?16?41? JST Donald Steven wrote: > > I don't think: > > > > -define(true,else). > > > > yielding ?else -> > > > > is much of an improvement. > > > > Why not just allow the keyword else as a synonym for true when its usage > > would improve the readability? (At times, "true" reminds me of trying to > > read (SAT-like) sentences like: "Whenever he couldn't think of whether > > or not he might have forgotten to turn the light on or off, or not.") > > Why not just use `if` where it fits semantically? > > These periodic discussions of how to use silly cheats to avoid proper > code layout always puzzle me. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Feb 21 08:34:04 2019 From: zxq9@REDACTED (zxq9@REDACTED) Date: Thu, 21 Feb 2019 16:34:04 +0900 Subject: [erlang-questions] if-elseif-else trick In-Reply-To: References: <782ccff3-0c63-63ca-f00e-0feca779ef48@zuiderkwast.se> <4055811.6yH3tYjjUC@takoyaki> Message-ID: <5124849.9tXoW0aiWB@takoyaki> On 2019?2?21???? 20?28?09? JST Richard O'Keefe wrote: > Seriously, though, when you find yourself > struggling with syntax in Erlang, you probably > need to break your code up into smaller > functions. Worth extracting as its own quote. -Craig From t6sn7gt@REDACTED Thu Feb 21 13:11:25 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Thu, 21 Feb 2019 07:11:25 -0500 Subject: [erlang-questions] Why do I have to recompile erl files Message-ID: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> When I take my work from my desktop to my laptop I have to recompile some erl modules or I get an error (even though the beam files are present and where they need to be and they're both modern Intel Windows machines).? What's going on? Thanks. Don PS: I do use the WIndows Linux subsystem, but both computers are up to date so I don' think there is any difference there either. From max.lapshin@REDACTED Thu Feb 21 17:08:24 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 21 Feb 2019 19:08:24 +0300 Subject: [erlang-questions] Use systemd's watchdog functionality instead of heart? In-Reply-To: References: Message-ID: We are using systemd in flussonic. I personally do not like it and consider that systemd is a very underdesigned, badly planned piece of software that do not do required things and do what it should not do. However it is everywhere and we have switched to systemd. -------------- next part -------------- An HTML attachment was scrubbed... URL: From justin.k.wood@REDACTED Thu Feb 21 19:23:53 2019 From: justin.k.wood@REDACTED (Justin Wood) Date: Thu, 21 Feb 2019 18:23:53 +0000 Subject: [erlang-questions] tcp_closed message Message-ID: Hello, I have a situation where I am closing a gen_tcp/gen_udp socket within my Erlang application. However, I would really like it to act as if it were disconnected or closed from the other end. Basically I want the {tcp_closed, S} message to be sent to the controlling process. A simple solution would be to just send the message myself. I'm just not sure if that would have the exact same behaviour (there is probably more to it than just the closed messed) as if the socket were truly disconnected or closed from the other end. Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunboshan@REDACTED Thu Feb 21 19:43:36 2019 From: sunboshan@REDACTED (Boshan Sun) Date: Thu, 21 Feb 2019 10:43:36 -0800 Subject: [erlang-questions] tcp_closed message In-Reply-To: References: Message-ID: Here's a really simple repo to demonstrate the usage of gen_tcp/gen_udp. https://github.com/sunboshan/socket You can follow the tcp.erl to use gen_tcp to set up a server, then use gen_tcp to connect to the server as a client. When one end disconnect, the other end will simply get a {tcp_closed, Socket} message. Nothing more than that. Boshan Justin Wood ?2019?2?21??? ??10:24??? > Hello, > > I have a situation where I am closing a gen_tcp/gen_udp socket within my > Erlang application. However, I would really like it to act as if it were > disconnected or closed from the other end. Basically I want the > {tcp_closed, S} message to be sent to the controlling process. > > A simple solution would be to just send the message myself. I'm just not > sure if that would have the exact same behaviour (there is probably more to > it than just the closed messed) as if the socket were truly disconnected or > closed from the other end. > > Justin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luke@REDACTED Thu Feb 21 19:53:53 2019 From: luke@REDACTED (Luke Bakken) Date: Thu, 21 Feb 2019 10:53:53 -0800 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> Message-ID: Hi Donald, I'm curious to know what the error is, and if you're using the same version of Erlang in both places. On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: > > When I take my work from my desktop to my laptop I have to recompile > some erl modules or I get an error (even though the beam files are > present and where they need to be and they're both modern Intel Windows > machines). What's going on? > > > Thanks. > > > Don > > > PS: I do use the WIndows Linux subsystem, but both computers are up to > date so I don' think there is any difference there either. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From justin.k.wood@REDACTED Thu Feb 21 20:03:07 2019 From: justin.k.wood@REDACTED (Justin Wood) Date: Thu, 21 Feb 2019 19:03:07 +0000 Subject: [erlang-questions] tcp_closed message In-Reply-To: References: Message-ID: <_h1BfQjKDnQfXsNsdH8RGQJnSAVPcnbTwndy3JLnNjK1JVy0Aq8QXjrcTlTGhf6AEjG7A4SBqcFLyYuQ4B99ncSOeiNoKx92VAySwjTaSMg=@protonmail.com> Forgive me, I must not have explained my situation very well. I have an Erlang process acting as the server. It accepts tcp and udp connections. I have various clients that connect to my server. When one of those clients disconnect from my server, I get the {tcp_closed, S} message. However, from within my Erlang server, if I call gen_tcp:close(S), the controlling process does not receive the {tcp_closed, S} message. I am looking for a way to fully emulate the disconnect / close from the client within my server. As I said previously, I can just send the {tcp_closed, S} message from within my own application. I'm just not sure what else, if anything, the VM does when a socket is closed from the other end. For reference, I am writing a testing tool that will randomly kill processes and sockets in order to test that your BEAM application(s) can properly handle the non-happy paths that the real world may bring to your application. Justin ??????? Original Message ??????? On Thursday, 21 February 2019 13:43, Boshan Sun wrote: > Here's a really simple repo to demonstrate the usage of gen_tcp/gen_udp. https://github.com/sunboshan/socket > > You can follow the tcp.erl to use gen_tcp to set up a server, then use gen_tcp to connect to the server as a client. When one end disconnect, the other end will simply get a {tcp_closed, Socket} message. Nothing more than that. > > Boshan > > Justin Wood ?2019?2?21??? ??10:24??? > >> Hello, >> >> I have a situation where I am closing a gen_tcp/gen_udp socket within my Erlang application. However, I would really like it to act as if it were disconnected or closed from the other end. Basically I want the {tcp_closed, S} message to be sent to the controlling process. >> >> A simple solution would be to just send the message myself. I'm just not sure if that would have the exact same behaviour (there is probably more to it than just the closed messed) as if the socket were truly disconnected or closed from the other end. >> >> Justin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunboshan@REDACTED Thu Feb 21 20:32:16 2019 From: sunboshan@REDACTED (Boshan Sun) Date: Thu, 21 Feb 2019 11:32:16 -0800 Subject: [erlang-questions] tcp_closed message In-Reply-To: <_h1BfQjKDnQfXsNsdH8RGQJnSAVPcnbTwndy3JLnNjK1JVy0Aq8QXjrcTlTGhf6AEjG7A4SBqcFLyYuQ4B99ncSOeiNoKx92VAySwjTaSMg=@protonmail.com> References: <_h1BfQjKDnQfXsNsdH8RGQJnSAVPcnbTwndy3JLnNjK1JVy0Aq8QXjrcTlTGhf6AEjG7A4SBqcFLyYuQ4B99ncSOeiNoKx92VAySwjTaSMg=@protonmail.com> Message-ID: If you have an Erlang process acting as a server, then you have 1 listening socket listening on a port(say 8080), it will be stored as an Erlang Port. When someone connect to the server, then you will have 1 accept socket. If you use gen_tcp:connect as client *in another shell*, then the client also has 1 socket. shell1 - server listening socket #Port<0.6> accept socket #Port<0.7> ----------------------- shell2 - client socket #Port<0.6> When you do gen_tcp:close(S), which socket did you close? If you close #Port<0.7> in shell1 server side, then no {tcp_closed,S} will be sent. If you close #Port<0.6> in shell2 client side, then there will be {tcp_closed,#Port<0.7>} sent to server process. A process is linked with the port, you can use `erlang:port_info(S)` to check. When either side close/disconnect the connection, both side's ports are closed and the linked processes are dead. Boshan Justin Wood ?2019?2?21??? ??11:03??? > Forgive me, I must not have explained my situation very well. > > I have an Erlang process acting as the server. It accepts tcp and udp > connections. I have various clients that connect to my server. When one of > those clients disconnect from my server, I get the {tcp_closed, S} message. > However, from within my Erlang server, if I call gen_tcp:close(S), the > controlling process does not receive the {tcp_closed, S} message. > > I am looking for a way to fully emulate the disconnect / close from the > client within my server. As I said previously, I can just send the > {tcp_closed, S} message from within my own application. I'm just not sure > what else, if anything, the VM does when a socket is closed from the other > end. > > For reference, I am writing a testing tool that will randomly kill > processes and sockets in order to test that your BEAM application(s) can > properly handle the non-happy paths that the real world may bring to your > application. > > Justin > > ??????? Original Message ??????? > On Thursday, 21 February 2019 13:43, Boshan Sun > wrote: > > Here's a really simple repo to demonstrate the usage of gen_tcp/gen_udp. > https://github.com/sunboshan/socket > > You can follow the tcp.erl to use gen_tcp to set up a server, then use > gen_tcp to connect to the server as a client. When one end disconnect, the > other end will simply get a {tcp_closed, Socket} message. Nothing more than > that. > > Boshan > > Justin Wood ?2019?2?21??? ??10:24??? > >> Hello, >> >> I have a situation where I am closing a gen_tcp/gen_udp socket within my >> Erlang application. However, I would really like it to act as if it were >> disconnected or closed from the other end. Basically I want the >> {tcp_closed, S} message to be sent to the controlling process. >> >> A simple solution would be to just send the message myself. I'm just not >> sure if that would have the exact same behaviour (there is probably more to >> it than just the closed messed) as if the socket were truly disconnected or >> closed from the other end. >> >> Justin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Thu Feb 21 22:38:26 2019 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Fri, 22 Feb 2019 08:38:26 +1100 Subject: [erlang-questions] tcp_closed message In-Reply-To: References: Message-ID: Can you make your test code to actually open a connection to the server you are testing? There is nothing wrong in doing that from inside the same Erlang node. Then you'd be able to close that client socket and observe tcp_closed on the controlling process On 22 February 2019 05:23:53 GMT+11:00, Justin Wood wrote: >Hello, > >I have a situation where I am closing a gen_tcp/gen_udp socket within >my Erlang application. However, I would really like it to act as if it >were disconnected or closed from the other end. Basically I want the >{tcp_closed, S} message to be sent to the controlling process. > >A simple solution would be to just send the message myself. I'm just >not sure if that would have the exact same behaviour (there is probably >more to it than just the closed messed) as if the socket were truly >disconnected or closed from the other end. > >Justin -- Kind regards, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Feb 21 22:57:50 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 21 Feb 2019 22:57:50 +0100 Subject: [erlang-questions] tcp_closed message In-Reply-To: References: Message-ID: The idea is to have a normal application running and creating havoc (a la chaos monkey) in it without any code change (the remote end might not be on the same host anyway). Just start havoc, wait a little, stop it, and see if your application is still in a good state. The ability to close sockets like this might require either a change in OTP to allow this or using os:cmd or similar to do it at the OS level. But hopefully there's a portable solution. On 21/02/2019 22:38, Dmitry Belyaev wrote: > Can you make your test code to actually open a connection to the server > you are testing? There is nothing wrong in doing that from inside the > same Erlang node. Then you'd be able to close that client socket and > observe tcp_closed on the controlling process > > On 22 February 2019 05:23:53 GMT+11:00, Justin Wood > wrote: > > Hello, > > I have a situation where I am closing a gen_tcp/gen_udp socket > within my Erlang application. However, I would really like it to act > as if it were disconnected or closed from the other end. Basically I > want the {tcp_closed, S} message to be sent to the controlling process. > > A simple solution would be to just send the message myself. I'm just > not sure if that would have the exact same behaviour (there is > probably more to it than just the closed messed) as if the socket > were truly disconnected or closed from the other end. > > Justin > > > -- > Kind regards, > Dmitry Belyaev > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From t6sn7gt@REDACTED Fri Feb 22 00:46:15 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Thu, 21 Feb 2019 18:46:15 -0500 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> Message-ID: <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] ... On 2/21/2019 1.53 PM, Luke Bakken wrote: > Hi Donald, > > I'm curious to know what the error is, and if you're using the same > version of Erlang in both places. > > On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: >> When I take my work from my desktop to my laptop I have to recompile >> some erl modules or I get an error (even though the beam files are >> present and where they need to be and they're both modern Intel Windows >> machines). What's going on? >> >> >> Thanks. >> >> >> Don >> >> >> PS: I do use the WIndows Linux subsystem, but both computers are up to >> date so I don' think there is any difference there either. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From luke@REDACTED Fri Feb 22 01:02:31 2019 From: luke@REDACTED (Luke Bakken) Date: Thu, 21 Feb 2019 16:02:31 -0800 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> Message-ID: Hi Donald, I suspect the different Erlang versions are causing this. Sometimes beam files are compatible, but I've found re-compilation is almost always required when moving between major versions. On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: > > My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... > My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] ... > > On 2/21/2019 1.53 PM, Luke Bakken wrote: > > Hi Donald, > > > > I'm curious to know what the error is, and if you're using the same > > version of Erlang in both places. > > > > On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: > >> When I take my work from my desktop to my laptop I have to recompile > >> some erl modules or I get an error (even though the beam files are > >> present and where they need to be and they're both modern Intel Windows > >> machines). What's going on? > >> > >> > >> Thanks. > >> > >> > >> Don > >> > >> > >> PS: I do use the WIndows Linux subsystem, but both computers are up to > >> date so I don' think there is any difference there either. > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > From eric.pailleau@REDACTED Fri Feb 22 01:07:38 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Fri, 22 Feb 2019 01:07:38 +0100 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> Message-ID: <51855109-d23e-452a-81ec-bc6cc1d4432e@wanadoo.fr> Hello, Compilation datetime is not anymore in beam file starting 19.0. maybe ? regards Le 22/02/2019 ? 01:02, Luke Bakken a ?crit?: > Hi Donald, > > I suspect the different Erlang versions are causing this. Sometimes > beam files are compatible, but I've found re-compilation is almost > always required when moving between major versions. > > On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: >> >> My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... >> My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] ... >> >> On 2/21/2019 1.53 PM, Luke Bakken wrote: >>> Hi Donald, >>> >>> I'm curious to know what the error is, and if you're using the same >>> version of Erlang in both places. >>> >>> On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: >>>> When I take my work from my desktop to my laptop I have to recompile >>>> some erl modules or I get an error (even though the beam files are >>>> present and where they need to be and they're both modern Intel Windows >>>> machines). What's going on? >>>> >>>> >>>> Thanks. >>>> >>>> >>>> Don >>>> >>>> >>>> PS: I do use the WIndows Linux subsystem, but both computers are up to >>>> date so I don' think there is any difference there either. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eric.pailleau@REDACTED Fri Feb 22 01:11:36 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Fri, 22 Feb 2019 01:11:36 +0100 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: <51855109-d23e-452a-81ec-bc6cc1d4432e@wanadoo.fr> References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> <51855109-d23e-452a-81ec-bc6cc1d4432e@wanadoo.fr> Message-ID: <3d7fe05a-163c-1a08-ebfc-db6c62affb6c@wanadoo.fr> Hello, And BEAM format starting 20.0 add new chunk types "AtU8" and "Dbgi" if I remember. regards Le 22/02/2019 ? 01:07, PAILLEAU Eric a ?crit?: > Hello, > Compilation datetime is not anymore in beam file starting 19.0. maybe ? > > regards > > > Le 22/02/2019 ? 01:02, Luke Bakken a ?crit?: >> Hi Donald, >> >> I suspect the different Erlang versions are causing this. Sometimes >> beam files are compatible, but I've found re-compilation is almost >> always required when moving between major versions. >> >> On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: >>> >>> My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... >>> My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] >>> [64-bit] ... >>> >>> On 2/21/2019 1.53 PM, Luke Bakken wrote: >>>> Hi Donald, >>>> >>>> I'm curious to know what the error is, and if you're using the same >>>> version of Erlang in both places. >>>> >>>> On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: >>>>> When I take my work from my desktop to my laptop I have to recompile >>>>> some erl modules or I get an error (even though the beam files are >>>>> present and where they need to be and they're both modern Intel >>>>> Windows >>>>> machines).? What's going on? >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> Don >>>>> >>>>> >>>>> PS: I do use the WIndows Linux subsystem, but both computers are up to >>>>> date so I don' think there is any difference there either. >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From S.J.Thompson@REDACTED Fri Feb 22 14:29:18 2019 From: S.J.Thompson@REDACTED (Simon Thompson) Date: Fri, 22 Feb 2019 14:29:18 +0100 Subject: [erlang-questions] Advert for: 12 fully-funded 3-year PhD scholarships at Kent References: <00651c1c-539d-a15a-9390-d7698e89b334@kent.ac.uk> Message-ID: <2919037E-4611-43C2-9070-BE2D4417A985@kent.ac.uk> The Programming Languages and Systems (PLAS) group at the University of Kent's School of Computing invites applications for 12 fully-funded 3-year PhD scholarships (for UK/EU students). Applications are due by the 26th April 2019. The PLAS group is one of the largest programming languages research groups in Europe. It is currently ranked 17th worldwide by the independent CSrankings website: http://csrankings.org/#/index?plan&world. We provide a supportive environment for research and we have a vibrant postgraduate population. We encourage our students to engage with the wider research community through attending conferences and taking internships with leading industrial companies. We are located in Canterbury, a lively and cosmopolitan historic town with convenient travel links to London and Europe. You can apply to study for a PhD in any topic that falls within our range of expertise. We have studentships up to a value of ?19,945 per annum that are available by competition. Application process: Select a potential supervisor (see below) and send them an informal project proposal as well as a brief CV (preferably by the first week of April 2019). Staff contact details can be found on their web pages. Submit your formal applications through the university admission system by the 26th April 2019. Your application should include a completed online admission form; the name and contact details of two referees; an original document providing confirmation of your degree (or a transcript if the degree is not yet awarded). For non-native English speakers, a certificate of competence in English is required at IELTS 6.5 or higher, with no element less than 6.0 (or equivalent). Programming Languages and Systems Group: https://www.cs.kent.ac.uk/research/groups/plas/index.html Topics suggested by our group https://www.cs.kent.ac.uk/research/groups/plas/pgprojects.html Applications process: https://www.cs.kent.ac.uk/research/studyingforaphd/howtoapply.html For general inquiries about the process, please e-mail: cs-phd-plas@REDACTED PLAS is a large research group with potential supervisors who work across the breadth of programming languages and systems research. *Dr Mark Batty* (Additional scholarship: https://www.cs.kent.ac.uk/people/staff/mjb211/studentship/studentship.htm) Concurrency; software verification; systems; relaxed memory; programming language semantics; GPU concurrency. *Dr Laura Bocchi* Foundations and engineering of API with complex behaviour, verification of distributed concurrent systems; behavioural types; real-time systems; transactions and transaction protocols. *Dr Olaf Chitil* Tracing, semantics, algorithmic debugging, type error debugging, compilation and functional programming. *Dr Radu Grigore* Program analysis; runtime verification; probabilistic models of computation. *Dr Rogerio de Lemos* Software engineering for self-adaptive systems: assurances and resilience evaluation; architecting resilient systems. *Prof. Richard Jones* Language implementation; memory management; garbage collection; object demographics; program analysis for improved memory management; program visualisation, rigorous performance evaluation. *Dr Stefan Kahrs* Expressiveness of programming languages, type systems, term rewriting, infinitary rewriting. *Dr Stephen Kell* Language implementation, tools, interoperability, runtimes and operating systems. *Prof. Andy King* Abstract interpretation, logic programming, decompilation and reverse engineering *Dr Julien Lange* Process calculi, automata theory, behavioural types, model checking and their application to the implementation and verification of concurrent and distributed systems. *Dr Stefan Marr* Language implementation techniques, concurrency, parallel programming, optimizations, tooling, debugging, virtual machines, interpreters, compilation. *Dr Matteo Migliavacca* On-line data processing, distributed publish-subscribe, and high-performance event processing in large scale and cloud scenarios. *Dr Dominic Orchard* Mathematical structure of programs; logical foundations of programming; categorical semantics; behavioural type theories; programming language design; program verification for computational science. *Dr Scott Owens* Semantics of shared memory concurrency; design of programming languages; formal verification for software and interactive theorem proving, especially for CakeML (https://cakeml.org). *Dr Tomas Petricek* Programming languages and tools, especially for data science, studying interactions of programming, bridging the gap between data and types; foundations of programming in a broad sense, including design and human experience; philosophy and history of computing and programming. *Prof. Simon Thompson* Functional programming in Haskell, Erlang and OCaml; refactoring functional programs: tool building. theory and practice: dependently-typed functional programming; DLT: languages for smart contracts on blockchains, including Marlowe on Cardano. Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Fri Feb 22 18:14:07 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Fri, 22 Feb 2019 12:14:07 -0500 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> Message-ID: <735feb8c-ffe7-6548-282e-018bc2c479e7@aim.com> Is it the case that beam files usually have to be recompiled when using a newer OTP version? On 2/21/2019 7.02 PM, Luke Bakken wrote: > Hi Donald, > > I suspect the different Erlang versions are causing this. Sometimes > beam files are compatible, but I've found re-compilation is almost > always required when moving between major versions. > > On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: >> My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... >> My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] ... >> >> On 2/21/2019 1.53 PM, Luke Bakken wrote: >>> Hi Donald, >>> >>> I'm curious to know what the error is, and if you're using the same >>> version of Erlang in both places. >>> >>> On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: >>>> When I take my work from my desktop to my laptop I have to recompile >>>> some erl modules or I get an error (even though the beam files are >>>> present and where they need to be and they're both modern Intel Windows >>>> machines). What's going on? >>>> >>>> >>>> Thanks. >>>> >>>> >>>> Don >>>> >>>> >>>> PS: I do use the WIndows Linux subsystem, but both computers are up to >>>> date so I don' think there is any difference there either. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions From sverker@REDACTED Fri Feb 22 18:30:38 2019 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 22 Feb 2019 18:30:38 +0100 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: <735feb8c-ffe7-6548-282e-018bc2c479e7@aim.com> References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> <735feb8c-ffe7-6548-282e-018bc2c479e7@aim.com> Message-ID: <1550856638.3990.8.camel@erlang.org> A new ERTS can usually load beam files compiled with an older compiler at least a couple of major versions back. An old ERTS may not be able to load beam files compiled with a newer compiler as it may find futuristic instructions that it does not understand. We usually only do such changes between major versions. /Sverker, Erlang/OTP On fre, 2019-02-22 at 12:14 -0500, Donald Steven wrote: > Is it the case that beam files usually have to be recompiled when using? > a newer OTP version? > > On 2/21/2019 7.02 PM, Luke Bakken wrote: > > > > Hi Donald, > > > > I suspect the different Erlang versions are causing this. Sometimes > > beam files are compatible, but I've found re-compilation is almost > > always required when moving between major versions. > > > > On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: > > > > > > My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... > > > My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] > > > ... > > > > > > On 2/21/2019 1.53 PM, Luke Bakken wrote: > > > > > > > > Hi Donald, > > > > > > > > I'm curious to know what the error is, and if you're using the same > > > > version of Erlang in both places. > > > > > > > > On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: > > > > > > > > > > When I take my work from my desktop to my laptop I have to recompile > > > > > some erl modules or I get an error (even though the beam files are > > > > > present and where they need to be and they're both modern Intel > > > > > Windows > > > > > machines).??What's going on? > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > Don > > > > > > > > > > > > > > > PS: I do use the WIndows Linux subsystem, but both computers are up to > > > > > date so I don' think there is any difference there either. > > > > > > > > > > > > > > > _______________________________________________ > > > > > erlang-questions mailing list > > > > > erlang-questions@REDACTED > > > > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t6sn7gt@REDACTED Fri Feb 22 18:54:21 2019 From: t6sn7gt@REDACTED (Donald Steven) Date: Fri, 22 Feb 2019 12:54:21 -0500 Subject: [erlang-questions] Why do I have to recompile erl files In-Reply-To: <1550856638.3990.8.camel@erlang.org> References: <73dc546c-1c00-63f5-3b78-627291084080@aim.com> <40ebca86-1a91-3455-8dd7-8fe7ad5b7542@aim.com> <735feb8c-ffe7-6548-282e-018bc2c479e7@aim.com> <1550856638.3990.8.camel@erlang.org> Message-ID: Thanks. On 2/22/2019 12.30 PM, Sverker Eriksson wrote: > A new ERTS can usually load beam files compiled with an older compiler > at least a couple of major versions back. > > An old ERTS may not be able to load beam files compiled with a newer compiler > as it may find futuristic instructions that it does not understand. > > We usually only do such changes between major versions. > > > /Sverker, Erlang/OTP > > > On fre, 2019-02-22 at 12:14 -0500, Donald Steven wrote: >> Is it the case that beam files usually have to be recompiled when using >> a newer OTP version? >> >> On 2/21/2019 7.02 PM, Luke Bakken wrote: >>> Hi Donald, >>> >>> I suspect the different Erlang versions are causing this. Sometimes >>> beam files are compatible, but I've found re-compilation is almost >>> always required when moving between major versions. >>> >>> On Thu, Feb 21, 2019 at 3:46 PM Donald Steven wrote: >>>> My PC says: Erlang/OTP 20 [erts-0.9] [source] [64-bit] ... >>>> My laptop says: My PC says: Erlang/OTP 18- [erts-7.3] [source] [64-bit] >>>> ... >>>> >>>> On 2/21/2019 1.53 PM, Luke Bakken wrote: >>>>> Hi Donald, >>>>> >>>>> I'm curious to know what the error is, and if you're using the same >>>>> version of Erlang in both places. >>>>> >>>>> On Thu, Feb 21, 2019 at 4:11 AM Donald Steven wrote: >>>>>> When I take my work from my desktop to my laptop I have to recompile >>>>>> some erl modules or I get an error (even though the beam files are >>>>>> present and where they need to be and they're both modern Intel >>>>>> Windows >>>>>> machines).??What's going on? >>>>>> >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>> Don >>>>>> >>>>>> >>>>>> PS: I do use the WIndows Linux subsystem, but both computers are up to >>>>>> date so I don' think there is any difference there either. >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vances@REDACTED Sat Feb 23 07:06:59 2019 From: vances@REDACTED (Vance Shipley) Date: Sat, 23 Feb 2019 11:36:59 +0530 Subject: [erlang-questions] Completing the thought (a question about receive) In-Reply-To: <20190218184549.GD317@carfax.org.uk> References: <4340e49e-40a4-4c53-714e-6132dcc1da49@aim.com> <20190218184549.GD317@carfax.org.uk> Message-ID: On Tue, Feb 19, 2019 at 1:05 PM Hugo Mills wrote: > The receive takes the first matching message off the queue Unless it's a "selective receive" in which case it may take some earlier message off the queue. -- -Vance From achowe@REDACTED Mon Feb 25 17:27:43 2019 From: achowe@REDACTED (Anthony Howe) Date: Mon, 25 Feb 2019 11:27:43 -0500 Subject: [erlang-questions] escript error handling Message-ID: <3d72d999-232c-cf15-deb8-43b04fb55de7@snert.com> I have some Erlang (v19) command line tools built with "rebar3 escriptize". I want my Erlang escript based tools to report any runtime errors to stderr, not stdout, since all my other error messages go to standard error. I've tried creating a custom error handler and disabling the tty handler: https://stackoverflow.com/questions/34250469/erlang-cowboy-runtime-errors-handling But that doesn't appear to work with an escript built tool since it continues to write to standard out. -- Anthony C Howe SnertSoft achowe@REDACTED Twitter: SirWumpus BarricadeMX & Milters http://snert.com/ http://nanozen.info/ http://snertsoft.com/ From mikpelinux@REDACTED Mon Feb 25 21:21:33 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 25 Feb 2019 21:21:33 +0100 Subject: [erlang-questions] escript error handling In-Reply-To: <3d72d999-232c-cf15-deb8-43b04fb55de7@snert.com> References: <3d72d999-232c-cf15-deb8-43b04fb55de7@snert.com> Message-ID: On Mon, Feb 25, 2019 at 6:23 PM Anthony Howe wrote: > > I have some Erlang (v19) command line tools built with "rebar3 > escriptize". I want my Erlang escript based tools to report any runtime > errors to stderr, not stdout, since all my other error messages go to > standard error. > > I've tried creating a custom error handler and disabling the tty handler: > > https://stackoverflow.com/questions/34250469/erlang-cowboy-runtime-errors-handling > > But that doesn't appear to work with an escript built tool since it > continues to write to standard out. I faced the same issue in some command-line tools I converted from C to Erlang, and ended up wrapping my escript entry points with a function that catches any exception and formats it to stderr. Not pretty, but good enough for now. I'd certainly like to see a cleaner solution; an escript's errors going to stdout is a bug IMNSHO. From halturin@REDACTED Tue Feb 26 10:53:09 2019 From: halturin@REDACTED (Taras Halturin) Date: Tue, 26 Feb 2019 12:53:09 +0300 Subject: [erlang-questions] Ergonode release 0.2.0 Message-ID: Hi all! Ergonode - Implementation of Erlang/OTP node in Go https://github.com/halturin/ergonode Just released new version of ergonode - 0.2.0 with new significant feature: embedded epmd. It allows you to create completely independent node without erlang's epmd dependencies, but with great go's performance Any feedback is highly appreciated -- Best Regards. Taras Halturin -------------- next part -------------- An HTML attachment was scrubbed... URL: From gergo.nyiro@REDACTED Tue Feb 26 13:11:17 2019 From: gergo.nyiro@REDACTED (=?UTF-8?B?TnlpcsWRIEdlcmfFkQ==?=) Date: Tue, 26 Feb 2019 13:11:17 +0100 Subject: [erlang-questions] trending json rpc library Message-ID: Dear List, I would like to write a small service in Erlang with json-rpc interface. The goal would be to introduce Erlang at my company so the dependencies and configuration of the service should be kept on minimum. At my first search I found three projects: - https://github.com/zuiderkwast/jsonrpc2-erlang.git - https://github.com/tonyg/erlang-rfc4627.git - https://github.com/alavrik/piqi-erlang.git Erlang-rfc4627 seems good for me, but I struggle with the debugging of the json service. Could you give me some advice which json-rpc is mainstream nowadays? Cheers, Gerg? From m4ver1k.a@REDACTED Wed Feb 27 02:09:17 2019 From: m4ver1k.a@REDACTED (maverik a) Date: Tue, 26 Feb 2019 20:09:17 -0500 Subject: [erlang-questions] How to create a bit array / bit vector Message-ID: Hi All, I'm trying to create a bit array in Erlang with simple new() , set() and get() operations. My new() looks like: new(Max) -> N = 1 bsl Max-1, <>. but problem I'm stuck with is, my max can be at most 8 . How can I create a *bitstring *of Max length ? Regards Maverick -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Feb 27 06:23:58 2019 From: erlang@REDACTED (Robert Carbone) Date: Tue, 26 Feb 2019 21:23:58 -0800 Subject: [erlang-questions] How to create a bit array / bit vector In-Reply-To: References: Message-ID: <100be4b9-ab0d-4f8d-bfee-57f6bf5dc760@scriptculture.com> Hey Maverick - Perhaps what you are looking for is this? fill(Len) -> <<(bnot 0):Len>>. Cheers, Robert Carbone scriptculture.com On 2/26/19 5:09 PM, maverik a wrote: > Hi All, > > I'm trying to create a bit array in Erlang with simple new() , set() > and get() operations. > > My new() looks like: > new(Max) -> > N=1bslMax-1, > <>. > > but problem I'm stuck with is, my max can be at most 8 . > > How can I create a *bitstring *of Max length ? > > Regards > Maverick > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From henrik.x.nord@REDACTED Wed Feb 27 13:22:47 2019 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Wed, 27 Feb 2019 12:22:47 +0000 Subject: [erlang-questions] Erlang OTP 22.0-rc1 is available for testing! Message-ID: OTP 22 Release Candidate 1 This is the first of three planned release candidates before the OTP 22 release. The intention with this release is to get feedback from our users. All feedback i wellcome, even if it is only to say that it works for you, Erlang/OTP 22 is a new major release with new features, improvements as well as incompatibilities. Potential Incompatibilities * gen_* behaviours, If logging of the last N messages through sys:log/2,3 is active for the server, this log is included in the terminate report. * New element, Opts, can now be included in a rel tuple in the reltool release specific configuration format: {rel, Name, Vsn, RelApps, Opts}. * All external pids/ports/refs created by erlang:list_to_pid and smilar functions now compare equal to any other pid/port/ref with same number from that node. * Old legacy erl_interface library is deprecated as of OTP 22, and will be removed in OTP 23. This does not apply to the ei library. * VxWorks is deprecated as of OTP 22 and will be removed in OTP 23. Known problems Native code generation does not work for all modules due to new beam instructions not supported by HiPE the native compiler. Dialyzers automatic compilation to native code does still work. Building OTP with the configure option enable-native-libs will not work in this release candidate. Highlights Erts: * Support for Erlang Distribution protocol to split the payload of large signals into several fragments. * ETS option write_concurrency now also effects and improves scalability of ordered_set tables. * The lenght/1 BIF used to calculate the length of the list in one go without yielding, even if the list was very long, Now yields when called with long lists. * A new (still experimental) module socket is introduced. It is implemented as a NIF and the idea is that it shall be as "close as possible" to the OS level socket interface. Compiler: * The compiler has been rewritten to internally use an intermediate representation based on Static Single Assignment (SSA). The new intermediate representation makes more optimizations possible. * The binary matching optimizations are now applicable in many more circumstances than before. * Type optimizations are now applied across local function calls, and will remove a lot more redundant type tests than before. * All compiler options that can be given in the source file can now be given in the option list on the command line for erlc. Standard libraries: * Cover now uses the counters module instead of ets for updating counters. New function cover:local_only/0 allows running Cover in a restricted but faster local-only mode. The increase in speed will vary depending on the type of code being cover-compiled, e.g. the compiler test suite runs more than twice as fast with the new Cover. * SSL now uses the new logger API, including log levels and verbose debug logging. For more details see http://erlang.org/download/otp_src_22.0-rc1.readme Pre built versions for Windows can be fetched here: http://erlang.org/download/otp_win32_22.0-rc1.exe http://erlang.org/download/otp_win64_22.0-rc1.exe Online documentation can be browsed here: http://erlang.org/documentation/doc-11.0-rc1/doc The Erlang/OTP source can also be found at GitHub on the official Erlang repository, https://github.com/erlang/otp [https://avatars3.githubusercontent.com/u/153393?s=400&v=4] GitHub - erlang/otp: Erlang/OTP github.com Erlang/OTP. Erlang is a programming language and runtime system for building massively scalable soft real-time systems with requirements on high availability.. OTP is a set of Erlang libraries, which consists of the Erlang runtime system, a number of ready-to-use components mainly written in Erlang, and a set of design principles for Erlang programs. ... OTP-22.0-rc1 Thank you for all your contributions! -------------- next part -------------- An HTML attachment was scrubbed... URL: From dieter@REDACTED Wed Feb 27 14:41:52 2019 From: dieter@REDACTED (dieter@REDACTED) Date: Wed, 27 Feb 2019 13:41:52 +0000 Subject: [erlang-questions] How to create a bit array / bit vector In-Reply-To: References: Message-ID: <495dd4403891c39640c7ec7f1313b8e0@afterlogic.edis.at> Hi, how about new(Max) -> . This creates a binary with Max bits, with the MSB set to 1. I found this doc page very helpful: http://erlang.org/doc/programming_examples/bit_syntax.html (http://erlang.org/doc/programming_examples/bit_syntax.html) Kind regards, Dieter Am Mi., Febr. 27, 2019 06:13 schrieb maverik a : Hi All, I'm trying to create a bit array in Erlang with simple new() , set() and get() operations. My new() looks like: new(Max) -> N = 1 bsl Max-1, . but problem I'm stuck with is, my max can be at most 8 . How can I create a bitstring of Max length ? Regards Maverick -------------- next part -------------- An HTML attachment was scrubbed... URL: From m4ver1k.a@REDACTED Wed Feb 27 18:15:29 2019 From: m4ver1k.a@REDACTED (maverik a) Date: Wed, 27 Feb 2019 12:15:29 -0500 Subject: [erlang-questions] How to create a bit array / bit vector In-Reply-To: <495dd4403891c39640c7ec7f1313b8e0@afterlogic.edis.at> References: <495dd4403891c39640c7ec7f1313b8e0@afterlogic.edis.at> Message-ID: Hi Dieter, Thank you that did work. And thanks for sharing the doc, the constructing binaries part explains this. Regards Maverick On Wed, Feb 27, 2019 at 8:41 AM wrote: > Hi, > > how about > > new(Max) -> > <<1:1, 0:(Max-1)>>. > > This creates a binary with Max bits, with the MSB set to 1. > > I found this doc page very helpful: > http://erlang.org/doc/programming_examples/bit_syntax.html > > Kind regards, > Dieter > > > > Am Mi., Febr. 27, 2019 06:13 schrieb maverik a : > > Hi All, > > I'm trying to create a bit array in Erlang with simple new() , set() and > get() operations. > > My new() looks like: > new(Max) -> > N = 1 bsl Max-1, > <>. > > but problem I'm stuck with is, my max can be at most 8 . > > How can I create a *bitstring *of Max length ? > > Regards > Maverick > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From halturin@REDACTED Thu Feb 28 08:53:03 2019 From: halturin@REDACTED (Taras Halturin) Date: Thu, 28 Feb 2019 10:53:03 +0300 Subject: [erlang-questions] Distribution protocol fragmentation feature coming in next release. Need help Message-ID: Hi all! I'm developing golang implementation of erlang node ( github.com/halturin/ergonode) and would like to know more information about this feature (fragmentation) to support it in my project. Is there any documentation/specification for this feature? It's a bit difficult to try understand by the commits -- Best Regards. Taras Halturin -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Thu Feb 28 09:39:35 2019 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 28 Feb 2019 09:39:35 +0100 Subject: [erlang-questions] Distribution protocol fragmentation feature coming in next release. Need help In-Reply-To: References: Message-ID: The documentation of the Distribution Protocol will be updated in the next release candidate to cover the new messages in the protocol. /Bj?rn On Thu, Feb 28, 2019 at 9:06 AM Taras Halturin wrote: > > Hi all! > > I'm developing golang implementation of erlang node (github.com/halturin/ergonode) and would like to know more information about this feature (fragmentation) to support it in my project. Is there any documentation/specification for this feature? It's a bit difficult to try understand by the commits > > -- > Best Regards. > Taras Halturin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From andreas.schultz@REDACTED Thu Feb 28 13:05:49 2019 From: andreas.schultz@REDACTED (Andreas Schultz) Date: Thu, 28 Feb 2019 13:05:49 +0100 Subject: [erlang-questions] Erlang OTP 22.0-rc1 is available for testing! In-Reply-To: References: Message-ID: Henrik Nord X schrieb am Mi., 27. Feb. 2019 um 13:22 Uhr: > OTP 22 Release Candidate 1 > [...] > > - A new (still experimental) module socket is introduced. It is > implemented as a NIF and the idea is that it shall be as "close as > possible" to the OS level socket interface. > > Is a {active, N} like delivery of data planed or will polling the recv methods be the only way the receive data? What about non-blocking connect and accept? My preference would be to get an active notification when data is available for receive and another one when when a non-blocking write is possible. Sending and reception should then be left to the caller. Same goes for non-block accept and connect. Call accept/connect with a async option and get some message when the procedures finishes. Thanks Andreas -- -- Dipl.-Inform. Andreas Schultz ----------------------- enabling your networks ---------------------- Travelping GmbH Phone: +49-391-81 90 99 0 Roentgenstr. 13 Fax: +49-391-81 90 99 299 39108 Magdeburg Email: info@REDACTED GERMANY Web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann VAT ID No.: DE236673780 --------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: