From kluchnikovas@REDACTED Mon Sep 2 10:53:14 2019 From: kluchnikovas@REDACTED (Aleksey Kluchnikov) Date: Mon, 2 Sep 2019 11:53:14 +0300 Subject: [erlang-questions] gen_server:init(Args) -> {stop, {normal, Reply}} Message-ID: Hi look at https://github.com/erlang/otp/blob/maint-22/lib/stdlib/src/gen_server.erl line 337-371 gen_server:init/1 hasn`t {normal, Reply} answer. It will be pretty good have gen_server:start/start_link normal reply without exit CRASH REPLY What do you think of it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Mon Sep 2 12:10:39 2019 From: matthias@REDACTED (Matthias Lang) Date: Mon, 2 Sep 2019 12:10:39 +0200 Subject: [erlang-questions] Who uses Erlang for research? Message-ID: <20190902101039.GA21127@hec.corelatus.se> Hi, I wanted to update FAQ 1.6: "Who uses Erlang for research (and teaching)?". A decade or two ago, my uninvolved view of _research_ was "Uppsala, Chalmers, Kent and Spain, and they all know each other, and then there are some smaller efforts elsewhere." Looking at the ICFP 2019, the actual conference has nothing involving Erlang, but the Erlang workshop has both familiar names and new faces. Being a workshop, it's about applications, so maybe that's what a settled-down Erlang is about now. Thoughts? Is there something major I've missed? (The 'teaching' part of the FAQ is unfortunate; it turned into an ad-hoc list of universities where someone mailed me. I intend to chop the list because it's such a small, arbitrary selection. I expect Erlang to pop up quite frequently in courses about distributed/concurrent systems, so just mentioning a few seems misleading.) Matthias From raoknz@REDACTED Mon Sep 2 12:36:51 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Mon, 2 Sep 2019 22:36:51 +1200 Subject: [erlang-questions] phash2 portability In-Reply-To: References: <9106264e-2b1b-0c3f-bb10-b05db45712e5@gmail.com> Message-ID: Hashing maps using xor? Oh dear. That's really not a good method. I found a much better way to hash sets and maps, but couldn't find a journal that thought it was in their scope. Actually, there are several good ways. Now you are locked into an inferior one. On Thu, 29 Aug 2019 at 03:19, Lukas Larsson wrote: > > > On Wed, Aug 28, 2019 at 8:45 AM Frans Schneider > wrote: > >> Hi list, >> >> According to the docs, the erlang:phash2/2 will return the same result >> regardless of machine architecture and ERTS version. Does this imply >> that this is still true in case the implementation of a basic type, lets >> say a list or map (#{}), will change in the future? Can I assume that >> calculating a hash for a list will have the same result now and in the >> future? >> > > Yes. The way hashing of maps works is that it hashes each k/v pair and > then takes the xor of each of those hashes. Since a series of xor's can be > applied in any order without changing the result, the hash value will > always be the same. > > >> >> Thanks >> >> >> Frans >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Sep 2 13:17:41 2019 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 2 Sep 2019 13:17:41 +0200 Subject: [erlang-questions] phash2 portability In-Reply-To: References: <9106264e-2b1b-0c3f-bb10-b05db45712e5@gmail.com> Message-ID: On Mon, Sep 2, 2019 at 12:37 PM Richard O'Keefe wrote: > Hashing maps using xor? Oh dear. That's really not a good method. > I found a much better way to hash sets and maps, but couldn't find > a journal that thought it was in their scope. Actually, there are > several good ways. > > Now you are locked into an inferior one. > If you have any links to better methods it would be great to have them if we ever decide to make a phash3 :) or maybe we can improve our internal hash used by ets and friends that don't need to work across versions. > > On Thu, 29 Aug 2019 at 03:19, Lukas Larsson wrote: > >> >> >> On Wed, Aug 28, 2019 at 8:45 AM Frans Schneider >> wrote: >> >>> Hi list, >>> >>> According to the docs, the erlang:phash2/2 will return the same result >>> regardless of machine architecture and ERTS version. Does this imply >>> that this is still true in case the implementation of a basic type, lets >>> say a list or map (#{}), will change in the future? Can I assume that >>> calculating a hash for a list will have the same result now and in the >>> future? >>> >> >> Yes. The way hashing of maps works is that it hashes each k/v pair and >> then takes the xor of each of those hashes. Since a series of xor's can be >> applied in any order without changing the result, the hash value will >> always be the same. >> >> >>> >>> Thanks >>> >>> >>> Frans >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gergo.nyiro@REDACTED Mon Sep 2 14:07:50 2019 From: gergo.nyiro@REDACTED (=?UTF-8?B?TnlpcsWRIEdlcmfFkQ==?=) Date: Mon, 2 Sep 2019 14:07:50 +0200 Subject: [erlang-questions] openssh-key-v1 pkcs #8 conversion for ed25519 Message-ID: Dear list, my question is not strictly belong to erlang, but I hope my problem is familiar to many list members. We are planning to support ed25519 for ssh and ssl too, and we would like to store the private keys in the same format (pkcs #8). Unfortunately the conversion between pkcs8 and openssh-key-1 for ed25519 private key is not an easy task. Does anyone know any library or cli tool which could be used for this conversion? thank for your advice in advance, Gergo From dave.wolf@REDACTED Tue Sep 3 14:48:41 2019 From: dave.wolf@REDACTED (Wolf, Dave) Date: Tue, 3 Sep 2019 12:48:41 +0000 Subject: [erlang-questions] MIB compiler errors for common MIBS In-Reply-To: References: Message-ID: Hi Daniel, I think I finally got it! I followed your advice and restructered all the tables, but was still getting a ?corrupt table definition?. Whomever created this MIB file commented out the circuits, circuitBreakers, outlets, tHSensors in the rackPowerPDUGroup below so there was a mismatch between PduEntry and the group (or at least I think that was it because I commented out the same lines in the PduEntry and it compiles now). Once again, many thanks for your answers to my questions, I?m glad I didn?t quite reach the end of your tips and tricks! Best regards, Dave Wolf PduEntry ::= SEQUENCE { pduIndex INTEGER, pduModel DisplayString, pduID DisplayString, pduName DisplayString, pduLocation DisplayString, pduGroupNumber DisplayString, pduMainLoadVoltage DisplayString, pduMainLoadAmp INTEGER, pduMainPowerFactor DisplayString, pduMainActivePower INTEGER, pduMainApparentPower INTEGER, pduMainCumulativeEnergy INTEGER, pduRealTimeClock DisplayString circuits DisplayString, circuitBreakers DisplayString, outlets DisplayString, tHSensors DisplayString } rackPowerPDUGroup OBJECT-GROUP OBJECTS { pduIndex, pduModel, pduID, pduName, pduLocation, pduGroupNumber, pduMainLoadVoltage, pduMainLoadAmp, pduMainPowerFactor, pduMainActivePower, pduMainApparentPower, pduMainCumulativeEnergy, pduRealTimeClock --circuits, --circuitBreakers, --outlets, --tHSensors } STATUS current DESCRIPTION "A collection of objects providing basic instrumentation and control of the PDU." ::= { rackPowerMIBGroups 2 } From: D?niel Szoboszlay Sent: Friday, August 30, 2019 2:36 PM To: Wolf, Dave (SI SSP R&D ATL) Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] MIB compiler errors for common MIBS Try reordering the definitions related to the pduTable like this: pduTable (the table definition) pduEntry (the row definition) definitions of all the columns in the table in the order they appear in the pduEntry sequence (so pduIndex, pduModel, pduID, ...) Make sure no other object definitions are left between the above (so e.g. define circuitTable and its row and columns after fully defining everything related to pduTable). Keeping definitions of all objects related to a table together in one block is one more thing the MIB compiler is picky about. Also, if table X uses a column from table Y as an index (e.g. simlar to circuitTable using pduIndex as an index) than X should be defined after Y. I hope you won't run into any more issues, because I'm running out of tips & tricks. :) I remember we once encountered a MIB with a table where the row's OID was not the { tableOID 1} but { tableOID 2 } or something. This definition was impossible to compile with OTP, and we either had to change the MIB or patch OTP, I don't remember which one. Fingers crossed for you not to encounter any weirdness like this! Cheers, Daniel On Fri, 30 Aug 2019 at 16:57, Wolf, Dave > wrote: After further review, I have fixed all of the errors, except the following (they were as Daniel described, object identifiers defined in reverse order): C:\Program Files\erl10.1\lib\snmp-5.2.12\mibs>erlc RACKPOWER-MIB.mib RACKPOWER-MIB.mib: 1654: Corrupt table definition. RACKPOWER-MIB.mib: Table 'pduTable' must have at least one accessible column. compilation_failed If anyone could help with this last one, I would be very grateful! Thanks, Dave. From: erlang-questions-bounces@REDACTED > On Behalf Of [ext] Wolf, Dave Sent: Friday, August 30, 2019 10:04 AM To: D?niel Szoboszlay >; erlang-questions@REDACTED Subject: Re: [erlang-questions] MIB compiler errors for common MIBS Hi Daniel, Thanks, this helps a lot; however, I am now met with another set of errors that I can?t seem to figure out as well, from another MIB file we are using from RackPower for their line of PDUs. I?m met with several of these errors below (full list is in attached output.txt file). OBJECT-TYPE is imported by SNMPv2-SMI, so that shouldn?t be an issue. What puzzles me is the ?not-accessible? possibility. I?ve looked at the objects in my first file I posted from yesterday and these don?t look different and that file compiles after the assistance I received yesterday. I gotta be missing something. I appreciate the responses I have received, this is all rather new and I am slowly learning. Any assistance anyone can provide would be greatly appreciated! Thanks, Dave. C:\Program Files\erl10.1\lib\snmp-5.2.12\mibs>erlc RACKPOWER-MIB.mib RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayModel' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayFirmware' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayName' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayLocation' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayMacAddress' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayIpAddressIpv4' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewaySubnetIpv4' RACKPOWER-MIB.mib: 100: OBJECT-TYPE definition missing or 'not-accessible' for 'ipgatewayGatewayIpv4' RACKPOWER-MIB.mib: 114: OBJECT-TYPE definition missing or 'not-accessible' for 'pduIndex' RACKPOWER-MIB.mib: 114: OBJECT-TYPE definition missing or 'not-accessible' for 'pduModel' RACKPOWER-MIB.mib: 114: OBJECT-TYPE definition missing or 'not-accessible' for 'pduID' RACKPOWER-MIB.mib: 114: OBJECT-TYPE definition missing or 'not-accessible' for 'pduName' <> From: erlang-questions-bounces@REDACTED > On Behalf Of D?niel Szoboszlay Sent: Thursday, August 29, 2019 3:41 PM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] MIB compiler errors for common MIBS Hi, My memories about the OTP MIB compiler are not very fresh, but back in the days I used it a lot I noticed that it's much more picky than other MIB-handling tools. Errors like the ones you encountered had to be fixed manually in most MIBs we got from equipment vendors. For example the TEXTUAL-CONVENTION macro is defined in SNMPv2-TC, and I guess most other tools simply auto-import all these definitions from the standard MIBs. When working with the OTP MIB compiler, you have to explicitly import it. mib-2 is similarly defined in SNMPv2-SMI, and you have to explicitly import it from there. An other common issue was object identifiers defined in reverse order (parent comes after child in the MIB file). You have to simply reverse the order of the definitions to please the compiler. Hope it helps, Daniel On Thu, 29 Aug 2019 at 18:46, Wolf, Dave > wrote: Hi all, I am working on an SNMP application that polls SNMP devices and currently it works fine if I use OIDs, but I?d like to be able to load the MIB file for the device and poll using the name associated with the OID. The issue I am running into is that for most of my MIB files I downloaded from the manufacturer sites, the MIB compiler in Erlang encounters errors during the compile to .bin format. I am wondering if I need to compile it in a certain directory, but I am not sure why it is failing. Here are beginning lines of the output from an example (full errors attached in output.txt) of this command: ?erlc UPS-MIB.mib?. I have tried this on Windows and Linux hosts with the same results. UPS-MIB.mib: 38: Macro 'TEXTUAL-CONVENTION' not imported. UPS-MIB.mib: 45: Macro 'TEXTUAL-CONVENTION' not imported. UPS-MIB.mib: 18: OBJECT IDENTIFIER [upsMIB] defined in terms of undefined parent object. Parent: ''mib-2''.(Sub-indexes: [33].) UPS-MIB.mib: 52: OBJECT IDENTIFIER [upsObjects] defined in terms of undefined parent object. Parent: 'upsMIB'.(Sub-indexes: [1].) UPS-MIB.mib: 62: OBJECT IDENTIFIER [upsIdent] defined in terms of undefined parent object. Parent: 'upsObjects'.(Sub-indexes: [1].) UPS-MIB.mib: 64: OBJECT IDENTIFIER [upsIdentManufacturer] defined in terms of undefined parent object. Parent: 'upsIdent'.(Sub-indexes: [1].) UPS-MIB.mib: 72: OBJECT IDENTIFIER [upsIdentModel] defined in terms of undefined parent object. Parent: 'upsIdent'.(Sub-indexes: [2].) <> I?ve attached the MIB file in case someone wants to give it a shot and tell me the errors of my ways. I have loaded this exact MIB file into a MIB browser and it works just fine, I can browse the device without any issues. Is there another way I can compile this MIB file? Is it in a common format that other tools might be able to compile? If anyone has any suggestions or needs further infomation, please let me know. Thanks! Dave Wolf. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyson.buzza@REDACTED Tue Sep 3 15:06:02 2019 From: tyson.buzza@REDACTED (Tyson Buzza) Date: Tue, 3 Sep 2019 21:06:02 +0800 Subject: [erlang-questions] ODBC SQL_BIGINT type Message-ID: Does anyone know why the ODBC SQL_BIGINT is implemented as SQL_C_CHAR? There must be a reason for it, but this code is 10 years old so I'm not sure who to ask. I'm wondering if the original reason would even be relevant anymore. I'm currently writing an Ecto adapter using ODBC and this is making it difficult. If you use a big int in your schema, Ecto expects an integer and gets a string instead. Either way the solution is to create a cache holding the type information and parse the string if a number is expected. I'm just wondering if this can be fixed for other people with my problem in the future. I'm talking about this file https://github.com/erlang/otp/blob/master/lib/odbc/c_src/odbcserver.c -------------- next part -------------- An HTML attachment was scrubbed... URL: From kjnilsson@REDACTED Tue Sep 3 17:45:43 2019 From: kjnilsson@REDACTED (Karl Nilsson) Date: Tue, 3 Sep 2019 16:45:43 +0100 Subject: [erlang-questions] [ANN] Ra 1.0 released Message-ID: Ra, a Raft library by Team RabbitMQ has reached v1.0 and has been released to hex.pm and github: https://hex.pm/packages/ra https://github.com/rabbitmq/ra/releases/tag/v1.0.0 Cheers -- *Karl Nilsson* -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.chochol@REDACTED Wed Sep 4 08:58:48 2019 From: jan.chochol@REDACTED (Jan Chochol) Date: Wed, 4 Sep 2019 08:58:48 +0200 Subject: [erlang-questions] ODBC SQL_BIGINT type In-Reply-To: References: Message-ID: Hi Tyson, Reason for using SQL_C_CHAR as physical encoding for SQL_BIGINT is, that there are not much better options. You can use SQL_C_NUMERIC, but it will not help in your case (it is still string containing digits plus some metainformation). Reason for not using SQL_C_SBIGINT or SQL_C_UBIGINT is, that it is not possible to found if SQL_BIGINT is signed, or unsigned, so it can lead to situation, that some values could not be returned. And it also requires ODBC 3.0, which is not supported by all drivers. What can Erlang ODBC client do, is to encode returned SQL_C_CHAR to number, but it can affect backward compatibility (it can be implemented in configurable way). >From my point of view, problem is that Erlang ODBC implementation tries to simplify interface too much, so it can not be used in some situations. Most problematic part is, that it does not return type information about columns (there is 'odbc:describe_table/2', but it can not be used in all cases for all requests), and different column types are encode to same Erlang type (we had own problems with distinguishing between SQL_WCHAR and SQL_CHAR). I do not see any easy way to fix this - it would probably require complete rewrite of Erlang ODBC client. On the other way - other languages have also big problems with ODBC - e.g. Perl DBD::ODBC has quite a lot problems with Unicode. Regards, Jan Chochol On Tue, Sep 3, 2019 at 3:13 PM Tyson Buzza wrote: > > Does anyone know why the ODBC SQL_BIGINT is implemented as SQL_C_CHAR? There must be a reason for it, but this code is 10 years old so I'm not sure who to ask. I'm wondering if the original reason would even be relevant anymore. > > I'm currently writing an Ecto adapter using ODBC and this is making it difficult. If you use a big int in your schema, Ecto expects an integer and gets a string instead. Either way the solution is to create a cache holding the type information and parse the string if a number is expected. I'm just wondering if this can be fixed for other people with my problem in the future. > > I'm talking about this file https://github.com/erlang/otp/blob/master/lib/odbc/c_src/odbcserver.c > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From block.rxckin.beats@REDACTED Thu Sep 5 09:39:04 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 5 Sep 2019 16:39:04 +0900 Subject: [erlang-questions] Strange matching behavior Message-ID: I found strange behavior, see code below. why last match doesn't works ? ```main.erl #!/usr/bin/env escript -module(main). -mode(compile). -compile(export_all). xy() -> #{ 160506610 => x, 3960650446 => y }. x() -> #{ 160506610 => x }. y() -> #{ 3960650446 => y }. xy2() -> #{ 888888888 => x, 9999999999 => y }. main(_) -> #{ 160506610 := x } = xy(), % ok #{ 3960650446 := y } = xy(), % ok #{ 160506610 := x } = x(), % ok #{ 3960650446 := y } = y(), % ok #{ 888888888 := x, 9999999999 := y } = xy2(), % ok #{ 160506610 := x, 3960650446 := y } = xy(), % no match of right hand side value, WHY ?? ok. ``` Erlang: Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] OS: Ubuntu 19.04 disco -------------- next part -------------- An HTML attachment was scrubbed... URL: From tommy.mattsson@REDACTED Thu Sep 5 09:41:09 2019 From: tommy.mattsson@REDACTED (Mattsson, Tommy) Date: Thu, 5 Sep 2019 07:41:09 +0000 Subject: [erlang-questions] Specify cwd for VM for a release built with rebar3 Message-ID: Hello, I have a problem with that when I start a release built with rebar3 the cwd is set to "/usr/lib64/my_application" (the installattion folder for the release) instead of the path that I do want "/my/custom/cwd/path/" Is there some option in some config file or some such that can help me achieve this? it worked well when building releases with reltool before using rebarX. Thanks for any help! Best regards, Tommy This email may contain information which is privileged or protected against unauthorized disclosure or communication. If you are not the intended recipient, please notify the sender and delete this message and any attachments from your system without producing, distributing or retaining copies thereof or disclosing its contents to any other person. Telia Company processes emails and other files that may contain personal data in accordance with Telia Company's Privacy Policy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Sep 5 09:59:40 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 5 Sep 2019 09:59:40 +0200 Subject: [erlang-questions] Strange matching behavior In-Reply-To: References: Message-ID: Hello, On Thu, Sep 5, 2019 at 9:39 AM block.rxckin.beats@REDACTED < block.rxckin.beats@REDACTED> wrote: > I found strange behavior, see code below. > > why last match doesn't works ? > It is a bug in the compiler in OTP-22 that has been fixed. If you upgrade to the latest OTP-22 patched version it works as it should. Lukas > > > ```main.erl > #!/usr/bin/env escript > -module(main). > > -mode(compile). > -compile(export_all). > > xy() -> > #{ 160506610 => x, 3960650446 => y }. > > x() -> > #{ 160506610 => x }. > > y() -> > #{ 3960650446 => y }. > > xy2() -> > #{ 888888888 => x, 9999999999 => y }. > > main(_) -> > #{ 160506610 := x } = xy(), % ok > #{ 3960650446 := y } = xy(), % ok > > #{ 160506610 := x } = x(), % ok > #{ 3960650446 := y } = y(), % ok > > > #{ 888888888 := x, > 9999999999 := y > } = xy2(), % ok > > > #{ 160506610 := x, > 3960650446 := y > } = xy(), % no match of right hand side value, WHY ?? > > ok. > ``` > > Erlang: Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] > [async-threads:1] [hipe] > OS: Ubuntu 19.04 disco > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From block.rxckin.beats@REDACTED Thu Sep 5 10:06:21 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 5 Sep 2019 17:06:21 +0900 Subject: [erlang-questions] Strange matching behavior In-Reply-To: References: Message-ID: Hi Lukas Updating 10.4.4 works fine for me thanks. Can I see the bug ticket for this behavior ? Jxck 2019?9?5?(?) 16:59 Lukas Larsson : > Hello, > > On Thu, Sep 5, 2019 at 9:39 AM block.rxckin.beats@REDACTED < > block.rxckin.beats@REDACTED> wrote: > >> I found strange behavior, see code below. >> >> why last match doesn't works ? >> > > It is a bug in the compiler in OTP-22 that has been fixed. If you upgrade > to the latest OTP-22 patched version it works as it should. > > Lukas > > >> >> >> ```main.erl >> #!/usr/bin/env escript >> -module(main). >> >> -mode(compile). >> -compile(export_all). >> >> xy() -> >> #{ 160506610 => x, 3960650446 => y }. >> >> x() -> >> #{ 160506610 => x }. >> >> y() -> >> #{ 3960650446 => y }. >> >> xy2() -> >> #{ 888888888 => x, 9999999999 => y }. >> >> main(_) -> >> #{ 160506610 := x } = xy(), % ok >> #{ 3960650446 := y } = xy(), % ok >> >> #{ 160506610 := x } = x(), % ok >> #{ 3960650446 := y } = y(), % ok >> >> >> #{ 888888888 := x, >> 9999999999 := y >> } = xy2(), % ok >> >> >> #{ 160506610 := x, >> 3960650446 := y >> } = xy(), % no match of right hand side value, WHY ?? >> >> ok. >> ``` >> >> Erlang: Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] >> [async-threads:1] [hipe] >> OS: Ubuntu 19.04 disco >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Sep 5 10:27:54 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 5 Sep 2019 10:27:54 +0200 Subject: [erlang-questions] Strange matching behavior In-Reply-To: References: Message-ID: On Thu, Sep 5, 2019 at 10:06 AM block.rxckin.beats@REDACTED < block.rxckin.beats@REDACTED> wrote: > Hi Lukas > > Updating 10.4.4 works fine for me thanks. > 10.4.4 is the run-time version, not the compiler version. Though I get why you are confused as that is what is printed when you start an erlang shell. > Can I see the bug ticket for this behavior ? > I'm not sure which compiler fix it was that fixed this problem, but you can see what was fixed in each patch here: http://blog.erlang.org/cd/docs/maint-22/lib/compiler-7.4.4/doc/html/notes.html. My guess would be that it is OTP-15838 that fixed the problem that you had. > > Jxck > > > 2019?9?5?(?) 16:59 Lukas Larsson : > >> Hello, >> >> On Thu, Sep 5, 2019 at 9:39 AM block.rxckin.beats@REDACTED < >> block.rxckin.beats@REDACTED> wrote: >> >>> I found strange behavior, see code below. >>> >>> why last match doesn't works ? >>> >> >> It is a bug in the compiler in OTP-22 that has been fixed. If you upgrade >> to the latest OTP-22 patched version it works as it should. >> >> Lukas >> >> >>> >>> >>> ```main.erl >>> #!/usr/bin/env escript >>> -module(main). >>> >>> -mode(compile). >>> -compile(export_all). >>> >>> xy() -> >>> #{ 160506610 => x, 3960650446 => y }. >>> >>> x() -> >>> #{ 160506610 => x }. >>> >>> y() -> >>> #{ 3960650446 => y }. >>> >>> xy2() -> >>> #{ 888888888 => x, 9999999999 => y }. >>> >>> main(_) -> >>> #{ 160506610 := x } = xy(), % ok >>> #{ 3960650446 := y } = xy(), % ok >>> >>> #{ 160506610 := x } = x(), % ok >>> #{ 3960650446 := y } = y(), % ok >>> >>> >>> #{ 888888888 := x, >>> 9999999999 := y >>> } = xy2(), % ok >>> >>> >>> #{ 160506610 := x, >>> 3960650446 := y >>> } = xy(), % no match of right hand side value, WHY ?? >>> >>> ok. >>> ``` >>> >>> Erlang: Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:8:8] >>> [ds:8:8:10] [async-threads:1] [hipe] >>> OS: Ubuntu 19.04 disco >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From block.rxckin.beats@REDACTED Thu Sep 5 10:31:45 2019 From: block.rxckin.beats@REDACTED (block.rxckin.beats@REDACTED) Date: Thu, 5 Sep 2019 17:31:45 +0900 Subject: [erlang-questions] Strange matching behavior In-Reply-To: References: Message-ID: Thanks a lot Lukas ! Jxck 2019?9?5?(?) 17:28 Lukas Larsson : > > > On Thu, Sep 5, 2019 at 10:06 AM block.rxckin.beats@REDACTED < > block.rxckin.beats@REDACTED> wrote: > >> Hi Lukas >> >> Updating 10.4.4 works fine for me thanks. >> > > 10.4.4 is the run-time version, not the compiler version. Though I get why > you are confused as that is what is printed when you start an erlang shell. > > >> Can I see the bug ticket for this behavior ? >> > > I'm not sure which compiler fix it was that fixed this problem, but you > can see what was fixed in each patch here: > http://blog.erlang.org/cd/docs/maint-22/lib/compiler-7.4.4/doc/html/notes.html. > My guess would be that it is OTP-15838 that fixed the problem that you had. > > >> >> Jxck >> >> >> 2019?9?5?(?) 16:59 Lukas Larsson : >> >>> Hello, >>> >>> On Thu, Sep 5, 2019 at 9:39 AM block.rxckin.beats@REDACTED < >>> block.rxckin.beats@REDACTED> wrote: >>> >>>> I found strange behavior, see code below. >>>> >>>> why last match doesn't works ? >>>> >>> >>> It is a bug in the compiler in OTP-22 that has been fixed. If you >>> upgrade to the latest OTP-22 patched version it works as it should. >>> >>> Lukas >>> >>> >>>> >>>> >>>> ```main.erl >>>> #!/usr/bin/env escript >>>> -module(main). >>>> >>>> -mode(compile). >>>> -compile(export_all). >>>> >>>> xy() -> >>>> #{ 160506610 => x, 3960650446 => y }. >>>> >>>> x() -> >>>> #{ 160506610 => x }. >>>> >>>> y() -> >>>> #{ 3960650446 => y }. >>>> >>>> xy2() -> >>>> #{ 888888888 => x, 9999999999 => y }. >>>> >>>> main(_) -> >>>> #{ 160506610 := x } = xy(), % ok >>>> #{ 3960650446 := y } = xy(), % ok >>>> >>>> #{ 160506610 := x } = x(), % ok >>>> #{ 3960650446 := y } = y(), % ok >>>> >>>> >>>> #{ 888888888 := x, >>>> 9999999999 := y >>>> } = xy2(), % ok >>>> >>>> >>>> #{ 160506610 := x, >>>> 3960650446 := y >>>> } = xy(), % no match of right hand side value, WHY ?? >>>> >>>> ok. >>>> ``` >>>> >>>> Erlang: Erlang/OTP 22 [erts-10.4] [source] [64-bit] [smp:8:8] >>>> [ds:8:8:10] [async-threads:1] [hipe] >>>> OS: Ubuntu 19.04 disco >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Thu Sep 5 11:54:38 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Thu, 5 Sep 2019 16:54:38 +0700 Subject: [erlang-questions] Any remote job traineeship / remote junior position Erlang based software development job? Message-ID: Hi, I'm looking for remote job traineeship / remote junior position Erlang based software development opportunity. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From luke@REDACTED Thu Sep 5 16:59:36 2019 From: luke@REDACTED (Luke Bakken) Date: Thu, 5 Sep 2019 07:59:36 -0700 Subject: [erlang-questions] Specify cwd for VM for a release built with rebar3 In-Reply-To: References: Message-ID: Hello, You may want to read this issue - https://github.com/erlang/rebar3/issues/1738 As well as this one - https://bugs.erlang.org/browse/ERL-715 Thanks, Luke On Thu, Sep 5, 2019 at 12:41 AM Mattsson, Tommy wrote: > > Hello, > > > I have a problem with that when I start a release built with rebar3 the cwd is set to "/usr/lib64/my_application" (the installattion folder for the release) instead of the path that I do want "/my/custom/cwd/path/" > > Is there some option in some config file or some such that can help me achieve this? it worked well when building releases with reltool before using rebarX. > > > Thanks for any help! > Best regards, > Tommy > > > > This email may contain information which is privileged or protected against unauthorized disclosure or communication. If you are not the intended recipient, please notify the sender and delete this message and any attachments from your system without producing, distributing or retaining copies thereof or disclosing its contents to any other person. > > Telia Company processes emails and other files that may contain personal data in accordance with Telia Company?s Privacy Policy. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t@REDACTED Thu Sep 5 17:08:55 2019 From: t@REDACTED (Tristan Sloughter) Date: Thu, 05 Sep 2019 09:08:55 -0600 Subject: [erlang-questions] =?utf-8?q?Specify_cwd_for_VM_for_a_release_bui?= =?utf-8?q?lt_with=09rebar3?= In-Reply-To: References: Message-ID: <20272e67-d959-4fd1-a9b8-566a35fc8563@www.fastmail.com> How were you running the release with reltool? This is likely a difference between the start script that rebar3/relx provide and what you were using to boot before. You can disable the script's generation and use a custom one. But maybe there is something we can do to allow setting the cwd. But I believe there are issues with upgrades when not having it run from the root of the release. Tristan On Thu, Sep 5, 2019, at 01:41, Mattsson, Tommy wrote: > > Hello, > > I have a problem with that when I start a release built with rebar3 the cwd is set to "/usr/lib64/my_application" (the installattion folder for the release) instead of the path that I do want "/my/custom/cwd/path/" > Is there some option in some config file or some such that can help me achieve this? it worked well when building releases with reltool before using rebarX. > > Thanks for any help! > Best regards, > Tommy > > > > *This email may contain information which is privileged or protected against unauthorized disclosure or communication. If you are not the intended recipient, please notify the sender and delete this message and any attachments from your system without producing, distributing or retaining copies thereof or disclosing its contents to any other person. > > Telia Company processes emails and other files that may contain personal data in accordance with Telia Company?s Privacy Policy .* > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Sep 7 16:11:29 2019 From: g@REDACTED (Guilherme Andrade) Date: Sat, 7 Sep 2019 15:11:29 +0100 Subject: [erlang-questions] gen_server:init(Args) -> {stop, {normal, Reply}} In-Reply-To: References: Message-ID: Hello Aleksey, You can gracefully stop a gen_server on `init` by wrapping the reason for stopping (or a "reply") in a `{shutdown, _}` tuple, e.g.: init(_) -> {stop, {shutdown, Reason}}. This way it won't be interpreted as a crash but still be returned to whomever called `gen_server:start_link`. On Mon, 2 Sep 2019 at 10:04, Aleksey Kluchnikov wrote: > Hi > > look at > https://github.com/erlang/otp/blob/maint-22/lib/stdlib/src/gen_server.erl > line 337-371 > gen_server:init/1 hasn`t {normal, Reply} answer. > It will be pretty good have gen_server:start/start_link normal reply > without exit CRASH REPLY > What do you think of it? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Sep 7 16:33:15 2019 From: g@REDACTED (Guilherme Andrade) Date: Sat, 7 Sep 2019 15:33:15 +0100 Subject: [erlang-questions] gen_server:init(Args) -> {stop, {normal, Reply}} In-Reply-To: References: Message-ID: On Sat, 7 Sep 2019 at 15:11, Guilherme Andrade wrote: > This way it won't be interpreted as a crash but still be returned to > whomever called `gen_server:start_link`. > (Assuming that, if you're launching the gen_server under a supervisor, its child specification declares `restart` as either `transient` or `temporary`.) > > On Mon, 2 Sep 2019 at 10:04, Aleksey Kluchnikov > wrote: > >> Hi >> >> look at >> https://github.com/erlang/otp/blob/maint-22/lib/stdlib/src/gen_server.erl >> line 337-371 >> gen_server:init/1 hasn`t {normal, Reply} answer. >> It will be pretty good have gen_server:start/start_link normal reply >> without exit CRASH REPLY >> What do you think of it? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Guilherme > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Sun Sep 8 15:44:28 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Sun, 8 Sep 2019 15:44:28 +0200 Subject: [erlang-questions] ODBC SQL_BIGINT type In-Reply-To: References: Message-ID: Hi! I did some maintaing of this code a long time ago. I know that one objectives of the erlang ODBC-client was to try and make it easy for Erlang code to access SQL databases and the Erlang user should only have to care about Erlang vs SQL-datatypes. However not all ODBC-backends are truly transparent in this regard, so I am afraid it fails in this respect. Also you can input a lot of backend specific strings which makes the code dependent on the backend, in which case you might as well use a backend specific client. Also, as I remember it, the implementation was made for a specific purpose, for which it works, and never extended to be more compleate (no communicated demand for it). As you observed this application has not really been developed for the last 10 Years and there is no plans from OTP to devleop it further (we have no own use of it). So it is really up to Open Source users to contribute if they think it useful. As long as it is included in OTP it is possible to make pullrequests. Regards Ingela Erlang/OTP team - Ericsson AB Den ons 4 sep. 2019 kl 08:59 skrev Jan Chochol : > Hi Tyson, > > Reason for using SQL_C_CHAR as physical encoding for SQL_BIGINT is, > that there are not much better options. > You can use SQL_C_NUMERIC, but it will not help in your case (it is > still string containing digits plus some metainformation). > > Reason for not using SQL_C_SBIGINT or SQL_C_UBIGINT is, that it is not > possible to found if SQL_BIGINT is signed, or unsigned, so it can lead > to situation, that some values could not be returned. And it also > requires ODBC 3.0, which is not supported by all drivers. > > What can Erlang ODBC client do, is to encode returned SQL_C_CHAR to > number, but it can affect backward compatibility (it can be > implemented in configurable way). > > From my point of view, problem is that Erlang ODBC implementation > tries to simplify interface too much, so it can not be used in some > situations. > Most problematic part is, that it does not return type information > about columns (there is 'odbc:describe_table/2', but it can not be > used in all cases for all requests), and different column types are > encode to same Erlang type (we had own problems with distinguishing > between SQL_WCHAR and SQL_CHAR). > I do not see any easy way to fix this - it would probably require > complete rewrite of Erlang ODBC client. > > On the other way - other languages have also big problems with ODBC - > e.g. Perl DBD::ODBC has quite a lot problems with Unicode. > > Regards, > Jan Chochol > > On Tue, Sep 3, 2019 at 3:13 PM Tyson Buzza wrote: > > > > Does anyone know why the ODBC SQL_BIGINT is implemented as SQL_C_CHAR? > There must be a reason for it, but this code is 10 years old so I'm not > sure who to ask. I'm wondering if the original reason would even be > relevant anymore. > > > > I'm currently writing an Ecto adapter using ODBC and this is making it > difficult. If you use a big int in your schema, Ecto expects an integer and > gets a string instead. Either way the solution is to create a cache holding > the type information and parse the string if a number is expected. I'm just > wondering if this can be fixed for other people with my problem in the > future. > > > > I'm talking about this file > https://github.com/erlang/otp/blob/master/lib/odbc/c_src/odbcserver.c > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Sun Sep 8 15:57:53 2019 From: freeakk@REDACTED (Michael Uvarov) Date: Sun, 8 Sep 2019 15:57:53 +0200 Subject: [erlang-questions] ODBC SQL_BIGINT type In-Reply-To: References: Message-ID: You can try my odbc fork https://github.com/arcusfelis/eodbc/blob/master/README.md It has an option to return types and some unicode fixes. It should work in Linux with mssql fine. On Sun, 8 Sep 2019, 15:44 Ingela Andin, wrote: > Hi! > > I did some maintaing of this code a long time ago. I know that one > objectives of the erlang ODBC-client was to try and make it easy for Erlang > code to access SQL databases and the Erlang user should only have to care > about Erlang vs SQL-datatypes. However not all ODBC-backends are truly > transparent in this regard, so I am afraid it fails in this respect. Also > you can input a lot of backend specific strings which makes the code > dependent on the backend, in which case you might as well use a backend > specific client. Also, as I remember it, the implementation was made for a > specific purpose, for which it works, and never extended to be more > compleate (no communicated demand for it). As you observed this application > has not really been developed for the last 10 Years and there is no plans > from OTP to devleop it further (we have no own use of it). So it is really > up to Open Source users to contribute if they think it useful. As long as > it is included in OTP it is possible to make pullrequests. > > Regards Ingela Erlang/OTP team - Ericsson AB > > Den ons 4 sep. 2019 kl 08:59 skrev Jan Chochol : > >> Hi Tyson, >> >> Reason for using SQL_C_CHAR as physical encoding for SQL_BIGINT is, >> that there are not much better options. >> You can use SQL_C_NUMERIC, but it will not help in your case (it is >> still string containing digits plus some metainformation). >> >> Reason for not using SQL_C_SBIGINT or SQL_C_UBIGINT is, that it is not >> possible to found if SQL_BIGINT is signed, or unsigned, so it can lead >> to situation, that some values could not be returned. And it also >> requires ODBC 3.0, which is not supported by all drivers. >> >> What can Erlang ODBC client do, is to encode returned SQL_C_CHAR to >> number, but it can affect backward compatibility (it can be >> implemented in configurable way). >> >> From my point of view, problem is that Erlang ODBC implementation >> tries to simplify interface too much, so it can not be used in some >> situations. >> Most problematic part is, that it does not return type information >> about columns (there is 'odbc:describe_table/2', but it can not be >> used in all cases for all requests), and different column types are >> encode to same Erlang type (we had own problems with distinguishing >> between SQL_WCHAR and SQL_CHAR). >> I do not see any easy way to fix this - it would probably require >> complete rewrite of Erlang ODBC client. >> >> On the other way - other languages have also big problems with ODBC - >> e.g. Perl DBD::ODBC has quite a lot problems with Unicode. >> >> Regards, >> Jan Chochol >> >> On Tue, Sep 3, 2019 at 3:13 PM Tyson Buzza wrote: >> > >> > Does anyone know why the ODBC SQL_BIGINT is implemented as SQL_C_CHAR? >> There must be a reason for it, but this code is 10 years old so I'm not >> sure who to ask. I'm wondering if the original reason would even be >> relevant anymore. >> > >> > I'm currently writing an Ecto adapter using ODBC and this is making it >> difficult. If you use a big int in your schema, Ecto expects an integer and >> gets a string instead. Either way the solution is to create a cache holding >> the type information and parse the string if a number is expected. I'm just >> wondering if this can be fixed for other people with my problem in the >> future. >> > >> > I'm talking about this file >> https://github.com/erlang/otp/blob/master/lib/odbc/c_src/odbcserver.c >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cean.ebengt@REDACTED Mon Sep 9 15:28:15 2019 From: cean.ebengt@REDACTED (bengt) Date: Mon, 9 Sep 2019 15:28:15 +0200 Subject: [erlang-questions] A localtime from UTC Message-ID: <082AB287-BFC8-4EBF-A09C-2A9E4F5FFD15@gmail.com> Greetings, I have Erlang datetime (UTC) and need to provide MET to a customer. Searching located qdate(1) and that pointed me to localtime(2) and ezic(3). Are there more alternatives? Does anybody have experience with these and recommendations? Since localtime is younger I added it as a dependency and it built. But the database is a few years old(4) and when I try to build new hlr files from a new db (manually, since I do not see any way to get it automatically at build time, please correct me if I am wrong) I get Perl errors(5). Best Wishes, bengt (1) https://github.com/choptastic/qdate (2) https://github.com/drfloob/ezic (3) https://github.com/dmitryme/erlang_localtime (4) From 2013i (5) make curl -O "ftp://ftp.iana.org/tz/tzdata-latest.tar.gz " % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 10 375k 10 41760 0 0 22343 0 0:00:17 0:00:01 0:00:16 22331 90 375k 90 339k 0 0 119k 0 0:00:03 0:00:02 0:00:01 119k 100 375k 100 375k 0 0 124k 0 0:00:03 0:00:03 --:--:-- 124k mkdir tzdata && cd tzdata && tar xzf ../tzdata-latest.tar.gz TZ_VERSION=`perl -n -e 'm/^VERSION\s*=\s*(\S+)/ and print $1;' tzdata/Makefile`; \ ./tz-erl --version $TZ_VERSION -o tzout tzdata/africa tzdata/antarctica tzdata/asia tzdata/australasia tzdata/backward tzdata/etcetera tzdata/europe tzdata/northamerica tzdata/southamerica no match for 5 in rule Rule Morocco 2019 only - May 5 3:00 -1:00 - at ./tz-erl line 219, line 939. no match for 9 in rule Rule Morocco 2019 only - Jun 9 2:00 0 - at ./tz-erl line 219, line 940. Day '33' out of range 1..31 at ./tz-erl line 488. Makefile:5: recipe for target 'tzout' failed -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Mon Sep 9 18:02:05 2019 From: zkessin@REDACTED (Zachary Kessin) Date: Mon, 9 Sep 2019 12:02:05 -0400 Subject: [erlang-questions] Difference between esl-erlang and erlang rpm packages Message-ID: We have been using the esl-erlang package here for some time now and were wondering what is the difference between the esl-erlang and the non esl labeled packages on the esl RPM repository. Thanks Zach Kessin zach@REDACTED +1 508 745 7940 https://youtu.be/Z3Yk7ipAAyk ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Sep 10 05:48:52 2019 From: zxq9@REDACTED (zxq9) Date: Tue, 10 Sep 2019 12:48:52 +0900 Subject: [erlang-questions] Difference between esl-erlang and erlang rpm packages In-Reply-To: References: Message-ID: <58c4c835-bea3-38ac-c755-c7d4b3edc674@zxq9.com> On 2019/09/10 1:02, Zachary Kessin wrote: > We have been using the esl-erlang package here for some time now and > were wondering what is the difference between the esl-erlang and the non > esl labeled packages on the esl RPM repository. Last I remember, the labeling indicates how the internal applications and utilities are packaged. I might have these backwards in my head, but IIRC: - The "esl-erlang" package has everything rolled into one bundle. This is a Bad Thing in some communities (and not everyone wants everything). - The "erlang" package is the start of a set that packages each utility and application separately across different rpms. -Craig From zkessin@REDACTED Tue Sep 10 17:11:48 2019 From: zkessin@REDACTED (Zachary Kessin) Date: Tue, 10 Sep 2019 11:11:48 -0400 Subject: [erlang-questions] Difference between esl-erlang and erlang rpm packages In-Reply-To: <58c4c835-bea3-38ac-c755-c7d4b3edc674@zxq9.com> References: <58c4c835-bea3-38ac-c755-c7d4b3edc674@zxq9.com> Message-ID: Thanks Zach Kessin zach@REDACTED +1 508 745 7940 https://youtu.be/Z3Yk7ipAAyk ? On Mon, Sep 9, 2019 at 11:49 PM zxq9 wrote: > On 2019/09/10 1:02, Zachary Kessin wrote: > > We have been using the esl-erlang package here for some time now and > > were wondering what is the difference between the esl-erlang and the non > > esl labeled packages on the esl RPM repository. > > Last I remember, the labeling indicates how the internal applications > and utilities are packaged. > > I might have these backwards in my head, but IIRC: > > - The "esl-erlang" package has everything rolled into one bundle. This > is a Bad Thing in some communities (and not everyone wants everything). > - The "erlang" package is the start of a set that packages each utility > and application separately across different rpms. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Thu Sep 12 11:59:34 2019 From: otp@REDACTED (Erlang/OTP) Date: Thu, 12 Sep 2019 11:59:34 +0200 Subject: [erlang-questions] Patch Package OTP 21.3.8.7 Released Message-ID: <20190912095933.GA63181@erix.ericsson.se> Patch Package: OTP 21.3.8.7 Git Tag: OTP-21.3.8.7 Date: 2019-09-12 Trouble Report Id: OTP-15962, OTP-15978, OTP-15983, OTP-15997, OTP-16012, OTP-16041, OTP-16049 Seq num: ERIERL-366, ERIERL-395, ERL-990 System: OTP Release: 21 Application: erts-10.3.5.5, inets-7.0.7.1, kernel-6.3.1.3, ssh-4.7.6.1, syntax_tools-2.1.7.1 Predecessor: OTP 21.3.8.6 Check out the git tag OTP-21.3.8.7, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.3.5.5 --------------------------------------------------- --------------------------------------------------------------------- Note! The erts-10.3.5.5 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependencies have to be satisfied: -- kernel-6.1 (first satisfied in OTP 21.1) -- sasl-3.3 (first satisfied in OTP 21.2) --- Fixed Bugs and Malfunctions --- OTP-15978 Application(s): erts Related Id(s): ERIERL-366 process_info(P,binary) would neglect to look through heap fragments, potentially missing a few binaries associated with the process. OTP-16041 Application(s): erts Fixed bug triggered if a process is killed during call to persistent_term:put or persistent_term:erase. --- Improvements and New Features --- OTP-15983 Application(s): erts Fixed rare emulator crash in instrument:allocations/0-1. Full runtime dependencies of erts-10.3.5.5: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- inets-7.0.7.1 --------------------------------------------------- --------------------------------------------------------------------- The inets-7.0.7.1 application can be applied independently of other applications on a full OTP 21 installation. --- Improvements and New Features --- OTP-16049 Application(s): inets Related Id(s): ERIERL-395 mod_esi will now always propagate the actual HTTP status code that it anwsered with, to later mod-modules, and not in some cases hardcode 200. Full runtime dependencies of inets-7.0.7.1: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-3.5 --------------------------------------------------------------------- --- kernel-6.3.1.3 -------------------------------------------------- --------------------------------------------------------------------- Note! The kernel-6.3.1.3 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependency has to be satisfied: -- erts-10.2.5 (first satisfied in OTP 21.2.7) --- Fixed Bugs and Malfunctions --- OTP-15997 Application(s): kernel Related Id(s): PR-2331 Fix bug where the log file in logger_std_h would not be closed when the inode of the file changed. This would in turn cause a file descriptor leak when tools like logrotate are used. Full runtime dependencies of kernel-6.3.1.3: erts-10.2.5, sasl-3.0, stdlib-3.5 --------------------------------------------------------------------- --- ssh-4.7.6.1 ----------------------------------------------------- --------------------------------------------------------------------- The ssh-4.7.6.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15962 Application(s): ssh Related Id(s): ERL-990 Fixed a possible SSH logging crash if there was a problem in an early stage of session setup. Full runtime dependencies of ssh-4.7.6.1: crypto-4.2, erts-6.0, kernel-3.0, public_key-1.5.2, stdlib-3.3 --------------------------------------------------------------------- --- syntax_tools-2.1.7.1 -------------------------------------------- --------------------------------------------------------------------- The syntax_tools-2.1.7.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-16012 Application(s): syntax_tools Related Id(s): PR-2348 Add missing calls to erl_syntax:unwrap/1. The nodes concerned represent names and values of maps and map types. Full runtime dependencies of syntax_tools-2.1.7.1: compiler-7.0, erts-9.0, kernel-5.0, stdlib-3.4 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From achowe@REDACTED Thu Sep 12 16:29:11 2019 From: achowe@REDACTED (Anthony Howe) Date: Thu, 12 Sep 2019 10:29:11 -0400 Subject: [erlang-questions] ASN.1 decode errors Message-ID: <07b593e3-5a4f-e27b-9e3c-27dbe9abf5af@snert.com> I've written a couple ASN.1 BER and PER transcoders to JSON for LTE, TAP3, NRTRDE files before. ( It was Erlang's open source ASN.1 support that got me to learn the language in the first place. ) I'm trying to write one now for GPRS charging records (ETSI TS 101 393 V7.4.0 (2000-02)); I have something that builds, but does not decode the input and just reports a vague Erlang ASN error. I figure the problem revolves around a mismatched set of ASN module files (which are a frik'n pain to find, but I digress). Anyway, I figure if I understood the error better it might point me finally in the right direction. So the error is... $ ./_build/default/bin/gprsdump -r cdr_0f1 cdr_0f1: processing... runtime error: cdr_0f1: {badmatch, {error, {asn1, {invalid_choice_tag, {1, <<0,168,161,129,165,128,1,19,131,8,48,36,128, 1,113,0,0,63,164>>}}}}} Reset of stack omitted. My interpretation is that initial CallEventRecord choice 1 (GGSNPDPRecord) is not decoding, but I have no idea what the binary is suppose to be telling me, other than "You Are Here in the BER input". CallEventRecord ::= CHOICE { sgsnPDPRecord [0] SGSNPDPRecord, ggsnPDPRecord [1] GGSNPDPRecord, sgsnMMRecord [2] SGSNMMRecord, sgsnSMORecord [3] SGSNSMORecord, sgsnSMTRecord [4] SGSNSMTRecord } Any clues to would be welcome. Right now its looking like I'll have to learn how to decode BER by hand in order to understand the failure better. Maybe a generic BER to ASN syntax decoder tool? -- Anthony C Howe SnertSoft achowe@REDACTED Twitter: SirWumpus BarricadeMX & Milters http://snert.com/ http://nanozen.info/ http://snertsoft.com/ From dszoboszlay@REDACTED Thu Sep 12 20:41:08 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Thu, 12 Sep 2019 20:41:08 +0200 Subject: [erlang-questions] ASN.1 decode errors In-Reply-To: <07b593e3-5a4f-e27b-9e3c-27dbe9abf5af@snert.com> References: <07b593e3-5a4f-e27b-9e3c-27dbe9abf5af@snert.com> Message-ID: Hi, I'm not very familiar with ASN.1, but I believe the error comes from a different place than the CallEventRecord. {invalid_choice_tag, {1, _}} suggests that you're trying to decode a CHOICE and found tag 1 in the data, but there's no tag 1 in the definition. CallEventRecord has tag 1 (GGSNPDPRecord), so the error cannot come from this place (and if there were an error decoding the GGSNPDPRecord, the error message won't complain about the invalid choice tag). Could you please include the top of the call stack? i think the name of the function where this error is coming from may hint to what kind of data fails to decode. Or, you can search the ASN.1 files for a CHOICE that has no tag 1. Cheers, Daniel On Thu, 12 Sep 2019 at 16:37, Anthony Howe wrote: > I've written a couple ASN.1 BER and PER transcoders to JSON for LTE, > TAP3, NRTRDE files before. ( It was Erlang's open source ASN.1 support > that got me to learn the language in the first place. ) I'm trying to > write one now for GPRS charging records (ETSI TS 101 393 V7.4.0 > (2000-02)); I have something that builds, but does not decode the input > and just reports a vague Erlang ASN error. > > I figure the problem revolves around a mismatched set of ASN module > files (which are a frik'n pain to find, but I digress). Anyway, I > figure if I understood the error better it might point me finally in the > right direction. > > So the error is... > > $ ./_build/default/bin/gprsdump -r cdr_0f1 > cdr_0f1: processing... > runtime error: cdr_0f1: {badmatch, > {error, > {asn1, > {invalid_choice_tag, > {1, > <<0,168,161,129,165,128,1,19,131,8,48,36,128, > 1,113,0,0,63,164>>}}}}} > > Reset of stack omitted. > > My interpretation is that initial CallEventRecord choice 1 > (GGSNPDPRecord) is not decoding, but I have no idea what the binary is > suppose to be telling me, other than "You Are Here in the BER input". > > CallEventRecord ::= CHOICE > { > sgsnPDPRecord [0] SGSNPDPRecord, > ggsnPDPRecord [1] GGSNPDPRecord, > sgsnMMRecord [2] SGSNMMRecord, > sgsnSMORecord [3] SGSNSMORecord, > sgsnSMTRecord [4] SGSNSMTRecord > } > > Any clues to would be welcome. > > Right now its looking like I'll have to learn how to decode BER by hand > in order to understand the failure better. Maybe a generic BER to ASN > syntax decoder tool? > > > -- > Anthony C Howe SnertSoft > achowe@REDACTED Twitter: SirWumpus BarricadeMX & Milters > http://snert.com/ http://nanozen.info/ http://snertsoft.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Sep 13 18:03:38 2019 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 13 Sep 2019 18:03:38 +0200 Subject: [erlang-questions] [ANN] erlang-rocksdb 1.3.2 has been released Message-ID: erlang-rocksdb, binding the RocksDB key-value store [1] to Erlang applications has been released. The version 1.3.2 fix builds using installed rocksdb on the system: https://gitlab.com/barrel-db/erlang-rocksdb The builds are a lot more faster with it. This version is also available on hex: https://hex.pm/packages/rocksdb ? and for #elixirlang applications obviously. To get started: https://gitlab.com/barrel-db/erlang-rocksdb/wikis/Getting-started Enjoy! Beno?t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Matt.Olson@REDACTED Fri Sep 13 18:09:45 2019 From: Matt.Olson@REDACTED (Matt Olson) Date: Fri, 13 Sep 2019 16:09:45 +0000 Subject: [erlang-questions] Making home drive detection more flexible Message-ID: Folks, There are multiple issues on stackoverflow where the home drive detection logic causes problems in corporate environments due to the frequent use of setting HOMEDRIVE and HOMEPATH to network file shares via group policy (which gets set upon each login to the workstation). See here on the problem description: https://groups.google.com/forum/#!topic/rabbitmq-users/8_JQd8druCI My suggestion is to change the small number of files indicated in the link directly below to do this logic instead: IF (ERLANGHOME exists) then use ERLANGHOME variable else use existing HOMEDRIVE variable logic as fallback end if https://github.com/erlang/otp/search?q=HOMEDRIVE&unscoped_q=HOMEDRIVE People have been working around this issue for years and would be nice to see it finally resolved. For reference: https://stackoverflow.com/questions/17496037/change-default-home-path-in-erlang-to-resolve-rabbitmq-start-up-error https://groups.google.com/forum/#!searchin/rabbitmq-users/homedrive%7Csort:date https://stackoverflow.com/search?q=erlang+homedrive -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sat Sep 14 19:06:21 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 14 Sep 2019 19:06:21 +0200 Subject: [erlang-questions] [ANN] samovar 1.0.0 Message-ID: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Hi ! samovar 1.0.0 has been released ! samovar is an Erlang library implementing SEMVER standard. Implementation is Erlang/OTP friendly by allowing use of old OTP release version syntax both for version and ranges. Available on github[1] and hex.pm[2] . Enjoy ! [1] https://github.com/crownedgrouse/samovar [2] https://hex.pm/packages/samovar From max.lapshin@REDACTED Sat Sep 14 20:05:08 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 14 Sep 2019 21:05:08 +0300 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: We have completely refused from semver, because it seems to be hardly useable today. Main reason is that nowadays nobody want to make major releases each 1,5 year with large changes, big migration path and 1,2 years of pain after each big release. Features are split into small steps and slowly implemented in services, softwares, libraries, etc. So it is too hard to say when 0.9.999-pre15-beta17 becomes 1.0.0. Ubuntu lives with YY.MM and it is very convenient. Firefox and Chrome just give some release number without major/minor/patch. I'm afraid that semantic versioning is not where modern software development practice is going. -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Sat Sep 14 20:42:27 2019 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sat, 14 Sep 2019 14:42:27 -0400 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: On Sat, Sep 14, 2019 at 2:05 PM Max Lapshin wrote: > > We have completely refused from semver, because it seems to be hardly useable today. > > Main reason is that nowadays nobody want to make major releases each 1,5 year with large changes, big migration path and 1,2 years of pain after each big release. > > Features are split into small steps and slowly implemented in services, softwares, libraries, etc. > > So it is too hard to say when 0.9.999-pre15-beta17 becomes 1.0.0. > > > Ubuntu lives with YY.MM and it is very convenient. Firefox and Chrome just give some release number without major/minor/patch. > > I'm afraid that semantic versioning is not where modern software development practice is going. It isn't without inconveniences, but the fundamental question is: what does a given versioning scheme intend to express? Semver defines a very clear and simple method for incrementing any and all parts of the version, each of which expresses something meaningful and useful for dependency constraint resolution (both mental and automated). 0.9.999-pre15-beta17 becomes 1.0.0 as soon as any other software package relies on it as a dependency. From siraaj@REDACTED Sat Sep 14 20:49:30 2019 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sat, 14 Sep 2019 14:49:30 -0400 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: On Sat, Sep 14, 2019 at 2:42 PM Siraaj Khandkar wrote: > > On Sat, Sep 14, 2019 at 2:05 PM Max Lapshin wrote: > > > > We have completely refused from semver, because it seems to be hardly useable today. > > > > Main reason is that nowadays nobody want to make major releases each 1,5 year with large changes, big migration path and 1,2 years of pain after each big release. > > > > Features are split into small steps and slowly implemented in services, softwares, libraries, etc. > > > > So it is too hard to say when 0.9.999-pre15-beta17 becomes 1.0.0. > > > > > > Ubuntu lives with YY.MM and it is very convenient. Firefox and Chrome just give some release number without major/minor/patch. > > > > I'm afraid that semantic versioning is not where modern software development practice is going. > > It isn't without inconveniences, but the fundamental question is: what > does a given versioning scheme intend to express? > > Semver defines a very clear and simple method for incrementing any and > all parts of the version, each of which expresses something meaningful > and useful for dependency constraint resolution (both mental and > automated). > > 0.9.999-pre15-beta17 becomes 1.0.0 as soon as any other software > package relies on it as a dependency. Oh, and as far as release cycles, sizes if changes, migrations, etc. - that is not related to the versioning scheme. Just do whatever is convent for you and then somehow express the impact of your changes on the dependent software - semver is only concerned with that expression. From essen@REDACTED Sat Sep 14 20:51:53 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 14 Sep 2019 20:51:53 +0200 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: On 14/09/2019 20:42, Siraaj Khandkar wrote:> 0.9.999-pre15-beta17 becomes 1.0.0 as soon as any other software > package relies on it as a dependency. Wishful thinking. In practice those other packages will happily rely on software that is far from ready for general consumption, let alone documented. Even if you depend on an 1.0.0 library, if it's not documented then you don't have any way of knowing what is considered internal and might break in the future. Semver is a useful tool but it doesn't guarantee much on its own. -- Lo?c Hoguin https://ninenines.eu From petergi@REDACTED Sun Sep 15 07:56:51 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Sun, 15 Sep 2019 15:56:51 +1000 Subject: [erlang-questions] node_not_running Message-ID: <00966b3ad0ed9bb8ab1a8e3cbfb221501770895c@webmail.iinet.net.au> dear friends,i am trying to interpret this message; Error in process with exit value: {{badmatch, {aborted, {node_not_running,nonode@REDACTED}}} thank you in advance for your guidance,happy coding,peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From cean.ebengt@REDACTED Sun Sep 15 08:12:42 2019 From: cean.ebengt@REDACTED (bengt) Date: Sun, 15 Sep 2019 08:12:42 +0200 Subject: [erlang-questions] node_not_running In-Reply-To: <00966b3ad0ed9bb8ab1a8e3cbfb221501770895c@webmail.iinet.net.au> References: <00966b3ad0ed9bb8ab1a8e3cbfb221501770895c@webmail.iinet.net.au> Message-ID: <3E4848BA-3C80-445D-B789-599C4A5AB321@gmail.com> Greetings, How did you start Erlang? With -name or -sname flag? Or without any of those? See http://erlang.org/doc/man/erl.html Best Wishes, bengt > On 15 Sep 2019, at 07:56, Peter J Etheridge wrote: > > dear friends, > i am trying to interpret this message; > > Error in process with exit value: > {{badmatch, {aborted, {node_not_running,nonode@REDACTED}}} > > thank you in advance for your guidance, > happy coding, > peter > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Sun Sep 15 09:55:42 2019 From: puzza007@REDACTED (Paul Oliver) Date: Sun, 15 Sep 2019 19:55:42 +1200 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: On Sun, Sep 15, 2019 at 6:05 AM Max Lapshin wrote: > We have completely refused from semver, because it seems to be hardly > useable today. > > Main reason is that nowadays nobody want to make major releases each 1,5 > year with large changes, big migration path and 1,2 years of pain after > each big release. > > Features are split into small steps and slowly implemented in services, > softwares, libraries, etc. > > So it is too hard to say when 0.9.999-pre15-beta17 becomes 1.0.0. > > > Ubuntu lives with YY.MM and it is very convenient. Firefox and Chrome > just give some release number without major/minor/patch. > > I'm afraid that semantic versioning is not where modern software > development practice is going. > _______________________________________________ > > The ecosystems I'm familiar with have all aligned on semver or something very close to it. I'd be interested to know of one that hasn't. Python: https://packaging.python.org/guides/distributing-packages-using-setuptools/#semantic-versioning-preferred Rust: https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field Beam: https://hex.pm/docs/publish#adding-metadata-to-code-classinlinemixexscode The network effects of choosing semver seem to outweigh the disadvantages of it not being perfect for everything. I'll certainly be using samovar, safe in the knowledge that the author is unlikely to break compatibility in a patch release ;) -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Sep 15 10:38:38 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 15 Sep 2019 10:38:38 +0200 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: Hello, I was currently writing something equivalent than your mail below, for Max remarks. I'm not saying that SEMVER is alpha (or 1.0.0-alpha ;>) ...) and omega . Erlang ecosystem is using semver or something very close. I didn't found libraries for this or very few documented ones. As any human action, setting version number on a project is not error proof ; but any version scheme have this problem. I think this could be automatized by a tool in order to propose a version change by comparing current code to former one. Exported functions, function specifications could help to propose a major or minor or patch change. I was using CVS in the old times. I loved the $Id$ tag expanding to CVS version of the file while commiting. How many people here forgot once to change the .app.src file before releasing ? Semver versioning or a project should not be handled by human but probably helped by AI. BTW, semver itself is currently 2.0.0 with a low risk of any major change that breaks compatibility, but always possible ... Regards Le 15/09/2019 ? 09:55, Paul Oliver a ?crit?: > > > On Sun, Sep 15, 2019 at 6:05 AM Max Lapshin > wrote: > > We have completely refused from semver, because it seems to be > hardly useable today. > > Main reason?is that nowadays nobody want to make major releases each > 1,5 year with large changes, big migration path and 1,2 years of > pain after each big release. > > Features are split into small steps and slowly?implemented in > services, softwares, libraries, etc. > > So it is too hard to say when 0.9.999-pre15-beta17 becomes 1.0.0. > > > Ubuntu lives with YY.MM and it is very convenient. > Firefox and Chrome just give some release number without > major/minor/patch. > > I'm afraid that semantic versioning is not where modern software > development practice is going. > _______________________________________________ > > > The ecosystems I'm familiar with have all aligned on semver or something > very close to it. I'd be interested to know of one that hasn't. > > Python: > https://packaging.python.org/guides/distributing-packages-using-setuptools/#semantic-versioning-preferred > Rust: > https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field > Beam: > https://hex.pm/docs/publish#adding-metadata-to-code-classinlinemixexscode > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eric.pailleau@REDACTED Sun Sep 15 11:41:16 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 15 Sep 2019 11:41:16 +0200 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: <4c7224c4-97ff-78d8-815a-c59922ce72ac@wanadoo.fr> Before somebody write me about 'rebar3 hex cut' and "git" tag. It is a good help, but still a human decision to change major, minor or patch. An analyze of changes should propose a new semver version, and giving the reasons in link to semver standard. Le 15/09/2019 ? 10:38, PAILLEAU Eric a ?crit?: > As any human action, setting version number on a project is not error > proof ; but any version scheme have this problem. > I think this could be automatized by a tool in order to propose a > version change by comparing current code to former one. > Exported functions, function specifications could help to propose a > major or minor or patch change. From kuna.prime@REDACTED Sun Sep 15 19:44:38 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Sun, 15 Sep 2019 19:44:38 +0200 Subject: [erlang-questions] behavior of lists:append/1 Message-ID: Hello, i have found surprising behavior of function lists:append/1: spec and documentation are in a agreement that this function should accept lists of lists ( [List] ) , and return list of T ( [T] ), however when we use function like: lists:append([a]) %wrong input parameter one gets: a % wrong type of a return implementation assumes correct type: append([E]) -> E; % E is not checked for type append([H|T]) -> H ++ append(T); append([]) -> []. simple solution could be: lists:append([E]) when is_list(E) -> E am i missing something? -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal@REDACTED Sun Sep 15 19:54:02 2019 From: michal@REDACTED (=?utf-8?Q?Micha=C5=82_Muska=C5=82a?=) Date: Sun, 15 Sep 2019 18:54:02 +0100 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: My understanding is that for most functions in the Erlang standard library, if you don't uphold the contract the documentation specifies the function can crash, but it could also return whatever - in short "garbage in, garbage out" applies. Micha?. On 15 Sep 2019, 18:45 +0100, Karlo Kuna , wrote: > Hello, > > i have found surprising? behavior of function lists:append/1: > > spec and documentation? are in a agreement that this function should accept lists of lists ( [List] ) , > and return list of T ( [T] ), however when we use function like: > > ? ? ?lists:append([a]) %wrong input parameter > one gets: > ? ? ?a? % wrong type of a return > > implementation assumes correct type: > > append([E]) -> E; % E is not checked for type > append([H|T]) -> H ++ append(T); > append([]) -> []. > > simple solution could be: > > lists:append([E]) when is_list(E) -> E > > am i missing something? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Sun Sep 15 20:02:25 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Sun, 15 Sep 2019 20:02:25 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: Michal, i really prefer crashing than garbage in, garbage out policy. also it would be nice if someone from OTP team confirms if stdlib has "garbage in garbage out" policy. i can certainly see benefits of it but in this case fix is easy and small. On Sun, Sep 15, 2019 at 7:54 PM Micha? Muska?a wrote: > My understanding is that for most functions in the Erlang standard > library, if you don't uphold the contract the documentation specifies the > function can crash, but it could also return whatever - in short "garbage > in, garbage out" applies. > > Micha?. > On 15 Sep 2019, 18:45 +0100, Karlo Kuna , wrote: > > Hello, > > i have found surprising behavior of function lists:append/1: > > spec and documentation are in a agreement that this function should > accept lists of lists ( [List] ) , > and return list of T ( [T] ), however when we use function like: > > lists:append([a]) %wrong input parameter > one gets: > a % wrong type of a return > > implementation assumes correct type: > > append([E]) -> E; % E is not checked for type > append([H|T]) -> H ++ append(T); > append([]) -> []. > > simple solution could be: > > lists:append([E]) when is_list(E) -> E > > am i missing something? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Mon Sep 16 00:43:31 2019 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Mon, 16 Sep 2019 08:43:31 +1000 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: <0EF3C72F-D65E-40DC-986A-84363B63B994@gmail.com> Every additional check further reduces performance and some of us are concerned about it. If you need correctness checks try to use static analysis tools like dialyzer/gradualizer, that way you catch the bug, we don't have reduced performance. On 16 September 2019 4:02:25 am AEST, Karlo Kuna wrote: >Michal, >i really prefer crashing than garbage in, garbage out policy. >also it would be nice if someone from OTP team confirms if stdlib has >"garbage in garbage out" policy. i can certainly see benefits of it but >in >this case fix is easy and small. > >On Sun, Sep 15, 2019 at 7:54 PM Micha? Muska?a >wrote: > >> My understanding is that for most functions in the Erlang standard >> library, if you don't uphold the contract the documentation specifies >the >> function can crash, but it could also return whatever - in short >"garbage >> in, garbage out" applies. >> >> Micha?. >> On 15 Sep 2019, 18:45 +0100, Karlo Kuna , >wrote: >> >> Hello, >> >> i have found surprising behavior of function lists:append/1: >> >> spec and documentation are in a agreement that this function should >> accept lists of lists ( [List] ) , >> and return list of T ( [T] ), however when we use function like: >> >> lists:append([a]) %wrong input parameter >> one gets: >> a % wrong type of a return >> >> implementation assumes correct type: >> >> append([E]) -> E; % E is not checked for type >> append([H|T]) -> H ++ append(T); >> append([]) -> []. >> >> simple solution could be: >> >> lists:append([E]) when is_list(E) -> E >> >> am i missing something? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- Kind regards, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Mon Sep 16 04:24:13 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Mon, 16 Sep 2019 14:24:13 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: This particular issue goes back nearly 60 years. Here is the traditional definition of "append", using Erlang syntax. append([X|Xs], Ys) -> [X | append(Xs, Ys)]; append(_, Ys) -> Ys. This was tightened up in more recent Lisp systems to the equivalent of append([X|Xs], Ys) -> [X | append(Xs, Ys)]; append([], Ys) -> Ys. To this day, if you fire up a Common Lisp REPL, you can get this * (append '(x) 'y) (X . Y) That was CMU Common Lisp 21d. You get the same in Scheme. And in Erlang, 1> [a]++b. [a|b] The point of this is that you want the cost of (append Xs Ys) to be O(length(Xs)). We really really don't want it to be O(length(Xs++Ys)). Now suppose we did append([X|Xs], Ys) -> [X | append(Xs, Ys)]; append([], Ys) when is_list(Ys) -> Ys. What good would that actually do? is_list is not well named. 3> is_list([a|b]). true So instead of append([a], b) -> [a|b] we would still have append([], [a|b]) -> [a|b]. The only way to ensure that the result of append is a well formed list is to check the *entire* spine of the second argument, and now we have this big cost we really wanted to avoid. I've been talking about the traditional "append" function. You were talking about append/1. But it's the same thing. Instead of append([[a],b]) -> [a|b], we'd still have append([[],[a|b]]) -> [a|b]. Inserting a call to is_list/1 would *only* add overhead without doing anything practical to address the issue. Erlang's solution is the type checker. This is also the solution adopted by SML, O'CAML, Haskell, Clean, Goedel, Mercury, the DEC-10 Prolog type checker (originally inspired in part by a desire to prove theorems about Prolog's append/3 that are not in fact sound without a type checker) and several other languages in the Lisp family. On Mon, 16 Sep 2019 at 05:45, Karlo Kuna wrote: > Hello, > > i have found surprising behavior of function lists:append/1: > > spec and documentation are in a agreement that this function should > accept lists of lists ( [List] ) , > and return list of T ( [T] ), however when we use function like: > > lists:append([a]) %wrong input parameter > one gets: > a % wrong type of a return > > implementation assumes correct type: > > append([E]) -> E; % E is not checked for type > append([H|T]) -> H ++ append(T); > append([]) -> []. > > simple solution could be: > > lists:append([E]) when is_list(E) -> E > > am i missing something? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Mon Sep 16 04:49:51 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Mon, 16 Sep 2019 14:49:51 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: Your proposed fix may be easy and small, but it is not a fix. Why do you suppose append/1 wasn't written *this* way? app([X|Xs], Ys) -> [X | app(Xs, Ys)]; app([], Ys) -> Ys. app([Xs|Xss]) -> app(Xs, app(Xss)); app([]) -> []. I coded this up, using app instead of append, and got this: 2> app:app([[],[a|b]]). ** exception error: no function clause matching app:app(b,[]) (app.erl, line 4) in function app:app/2 (app.erl, line 4) in call from app:app/1 (app.erl, line 7) That would seem to be what you are after, and it's *simpler* than the existing code. So what is the point? Let L be a well formed list, and consider append([[1],[2],...,[N],L]) The fact that L is a well-formed list is verified N times, for a total cost O(N * |L|). But with the current definition, the cost is O(N), independent of |L|. Oddly enough, there *is* a way to check that a list is well formed in a guard. Let's try this. append([Xs]) when length(Xs) >= 0 -> Xs; append([Xs|Xss]) -> Xs ++ append(Xss); append([]) -> []. This actually *would* be a "fix", and the cost would be O(N + |L|) instead of O(N * |L|). But that would make append([Xs,Ys]) inconsistent with Xs++Ys. I will say that I've been using languages in the Lisp family for a little over 40 years, and this has been the very *least* of my problems. On Mon, 16 Sep 2019 at 06:02, Karlo Kuna wrote: > Michal, > i really prefer crashing than garbage in, garbage out policy. > also it would be nice if someone from OTP team confirms if stdlib has > "garbage in garbage out" policy. i can certainly see benefits of it but in > this case fix is easy and small. > > On Sun, Sep 15, 2019 at 7:54 PM Micha? Muska?a wrote: > >> My understanding is that for most functions in the Erlang standard >> library, if you don't uphold the contract the documentation specifies the >> function can crash, but it could also return whatever - in short "garbage >> in, garbage out" applies. >> >> Micha?. >> On 15 Sep 2019, 18:45 +0100, Karlo Kuna , wrote: >> >> Hello, >> >> i have found surprising behavior of function lists:append/1: >> >> spec and documentation are in a agreement that this function should >> accept lists of lists ( [List] ) , >> and return list of T ( [T] ), however when we use function like: >> >> lists:append([a]) %wrong input parameter >> one gets: >> a % wrong type of a return >> >> implementation assumes correct type: >> >> append([E]) -> E; % E is not checked for type >> append([H|T]) -> H ++ append(T); >> append([]) -> []. >> >> simple solution could be: >> >> lists:append([E]) when is_list(E) -> E >> >> am i missing something? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Sep 16 09:54:31 2019 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 16 Sep 2019 09:54:31 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: On Sun, Sep 15, 2019 at 8:02 PM Karlo Kuna wrote: > Michal, > i really prefer crashing than garbage in, garbage out policy. > also it would be nice if someone from OTP team confirms if stdlib has > "garbage in garbage out" policy. i can certainly see benefits of it but in > this case fix is easy and small. > It is impossible to guard against every possible faulty input, so yes, we do have "a garbage in, garbage out policy". However, we do try to guide the user to write correct code where possible without losing too much performance. Where to draw the line of what "too much performance" means is a constant debate as shown by the other answers to your question. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Mon Sep 16 10:41:05 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 16 Sep 2019 10:41:05 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: On Mon, Sep 16, 2019 at 9:54 AM Lukas Larsson wrote: > > > On Sun, Sep 15, 2019 at 8:02 PM Karlo Kuna wrote: >> >> Michal, >> i really prefer crashing than garbage in, garbage out policy. >> >> also it would be nice if someone from OTP team confirms if stdlib has "garbage in garbage out" policy. i can certainly see benefits of it but in this case fix is easy and small. > > > It is impossible to guard against every possible faulty input, so yes, we do have "a garbage in, garbage out policy". However, we do try to guide the user to write correct code where possible without losing too much performance. Where to draw the line of what "too much performance" means is a constant debate as shown by the other answers to your question. The bottom line, if I may draw on ROK's comment, is that runtime type checking adds a measurable cost, generally O(size of term), in dynamically typed languages. Adding such a check when not necessary for the actual operation in question would be a performance bug, and could easily even raise the asymptotic complexity of the code involved. That's why dynamically typed languages generally do not do any more type checking that absolutely necessary. From kostis@REDACTED Mon Sep 16 12:01:42 2019 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 16 Sep 2019 12:01:42 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: On 9/16/19 9:54 AM, Lukas Larsson wrote: > > On Sun, Sep 15, 2019 at 8:02 PM Karlo Kuna > wrote: > > Michal, > i really prefer crashing than garbage in, garbage out policy. > > also it would be nice if someone from OTP team confirms if stdlib > has "garbage in garbage out" policy. i can certainly see benefits of > it but in this case fix is easy and small. > > > It is impossible to guard against every possible faulty input, ... Although I agree with the rest of Lukas' answer, "impossible" is too strong a word here; we are not talking about an undecidable problem after all. I think that "undesirable", "costly", etc. would have been a better choice in this context. Kostis From zxq9@REDACTED Mon Sep 16 15:53:57 2019 From: zxq9@REDACTED (zxq9) Date: Mon, 16 Sep 2019 22:53:57 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: On 2019/09/16 11:49, Richard O'Keefe wrote: > The fact that L is a well-formed list is verified N times, > for a total cost O(N * |L|).? But with the current definition, > the cost is O(N), independent of |L|. Hm... just to beat a dead horse...I suppose we could get away with a single check: -export([append/2]). append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; combine([], Ys) -> Ys. > I will say that I've been using languages in the Lisp family for > a little over 40 years, and this has been the very *least* of my > problems. The whole issue boils down to the above. I can see some trivial merit in doing a type check over Ys at the outset (since we'll crash on bad Xs in the actual procedure) but this business of moving to the last element to check whether the list is properly formed is both insane and almost certainly a legacy code killer, as there are a handful of systems out there that depend on being able to manipulate improper lists for whatever reason. I didn't follow this thread closely, but I'm surprised I didn't see doing a single type check on entry as a possibility. What issues would make this a bad idea? -Craig From by@REDACTED Mon Sep 16 15:58:23 2019 From: by@REDACTED (by) Date: Mon, 16 Sep 2019 21:58:23 +0800 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: <07B3C1B0-6CD1-437F-A66D-574E92752C8F@meetlost.com> Well, if we can put the type check on entry, we can put the type check before we call this function :-) Yao > ? 2019?9?16??21:53?zxq9 ??? > > On 2019/09/16 11:49, Richard O'Keefe wrote: >> The fact that L is a well-formed list is verified N times, >> for a total cost O(N * |L|). But with the current definition, >> the cost is O(N), independent of |L|. > > Hm... just to beat a dead horse...I suppose we could get away with a single check: > > -export([append/2]). > > append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). > > combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; > combine([], Ys) -> Ys. > >> I will say that I've been using languages in the Lisp family for >> a little over 40 years, and this has been the very *least* of my >> problems. > > The whole issue boils down to the above. I can see some trivial merit in doing a type check over Ys at the outset (since we'll crash on bad Xs in the actual procedure) but this business of moving to the last element to check whether the list is properly formed is both insane and almost certainly a legacy code killer, as there are a handful of systems out there that depend on being able to manipulate improper lists for whatever reason. > > I didn't follow this thread closely, but I'm surprised I didn't see doing a single type check on entry as a possibility. What issues would make this a bad idea? > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From andreas.schultz@REDACTED Mon Sep 16 17:14:23 2019 From: andreas.schultz@REDACTED (Andreas Schultz) Date: Mon, 16 Sep 2019 17:14:23 +0200 Subject: [erlang-questions] need help with a DIAMETER test case Message-ID: I need some insight with the test case for this change: https://github.com/RoadRunnr/otp/commit/6f54f71d3b4db3199d49fbb64d113563dd13b21e What it is supposed to fix is: > Two consecutive DIAMETER messages with just the right combination of fragmentation and delay would cause the fragment_timeout to trigger on the wrong fragment and flush it. The test case fails because the second message is not received back completely. After digging through the code and adding lots of debug output it appears as if gen_tcp does not deliver the pending data in time (within 50ms) after a setopts(Socket, [{active, once}]). I have left the log messages in place to be able to trace the problem. And they seem to confirm that the tail part of the second message in the test is not being delivered to the transport module. A PCAP shows that is data is send over the wire at the expected moment, so the delay can't be there. Comparing the timestamps in the PCAP with the timestamps in the log confirms that the first and second part are received. At this point I'm not sure whether the transport module is doing something strange or if gen_tcp is not delivering the data in time (or at all). Many thanks Andreas -- Andreas Schultz -- Principal Engineer t: +49 391 819099-224 ------------------------------- enabling your networks ----------------------------- Travelping GmbH Roentgenstra?e 13 39108 Magdeburg Germany t: +49 391 819099-0 f: +49 391 819099-299 e: info@REDACTED w: https://www.travelping.com/ Company registration: Amtsgericht Stendal Reg. No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann VAT ID: DE236673780 -------------- next part -------------- An HTML attachment was scrubbed... URL: From olopierpa@REDACTED Mon Sep 16 21:03:06 2019 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Mon, 16 Sep 2019 21:03:06 +0200 Subject: [erlang-questions] behavior of lists:append/1 Message-ID: Il giorno 16 settembre 2019, alle ore 15:54, zxq9 ha scritto: >I didn't follow this thread closely, but I'm surprised I didn't see >doing a single type check on entry as a possibility. What issues would >make this a bad idea? Only that you add a linear factor to every use of append. Many linear algorithms would became quadratic, etc etc -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Tue Sep 17 00:25:37 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Tue, 17 Sep 2019 10:25:37 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: Briefly, the problem with append(Xs, Ys) when is_list(Ys) -> combine(Xs, Ys). is that is_list/1 ***ISN'T A TYPE CHECK FOR LISTS***. It is misnamed. is_list([_|_]) -> true; is_list([]) -> true; is_list(_) -> false. Try is_list([a|b]). The result is true, despite [a|b] not being a list. Unsurprisingly, (LISTP '()) and (LISTP '(1 . 2)) are true in Common Lisp. There is an interesting difference between Common Lisp and Scheme: (list? '(1 . 2)) is false in Scheme. But the Scheme function takes O(|list|) time. Oddly enough, since Erlang is strict and functional, it *would* be possible for Erlang to tag pairs whose tail is a proper list differently from pairs whose tail is *not* a proper list, with small constant overhead. *That* is what it would take to have an affordable is_really_a_well_formed_list/1. On Tue, 17 Sep 2019 at 01:54, zxq9 wrote: > On 2019/09/16 11:49, Richard O'Keefe wrote: > > The fact that L is a well-formed list is verified N times, > > for a total cost O(N * |L|). But with the current definition, > > the cost is O(N), independent of |L|. > > Hm... just to beat a dead horse...I suppose we could get away with a > single check: > > -export([append/2]). > > append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). > > combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; > combine([], Ys) -> Ys. > > > I will say that I've been using languages in the Lisp family for > > a little over 40 years, and this has been the very *least* of my > > problems. > > The whole issue boils down to the above. I can see some trivial merit in > doing a type check over Ys at the outset (since we'll crash on bad Xs in > the actual procedure) but this business of moving to the last element to > check whether the list is properly formed is both insane and almost > certainly a legacy code killer, as there are a handful of systems out > there that depend on being able to manipulate improper lists for > whatever reason. > > I didn't follow this thread closely, but I'm surprised I didn't see > doing a single type check on entry as a possibility. What issues would > make this a bad idea? > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Tue Sep 17 05:21:54 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Tue, 17 Sep 2019 13:21:54 +1000 Subject: [erlang-questions] node_not_running Message-ID: <0920da125c4de6d6f51b3d287416578801661ad3@webmail.iinet.net.au> Greetings,As far as I can tell, I started Erlang without a flag. Following your referral, I re-read the erl module more closely.Thank you for your helpful direction, Bengt. Happy coding,Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From soverdor@REDACTED Tue Sep 17 01:44:29 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Mon, 16 Sep 2019 16:44:29 -0700 Subject: [erlang-questions] lists Message-ID: How do I tell the difference between: "sam1" ["sam1","sam2"] They are both lists but behave differently when I apply [H|T] = List. H = "s" H = "sam1". The second one is what I want and then stop breaking them down. Weird problem... Thanks, Sam From zxq9@REDACTED Tue Sep 17 05:51:09 2019 From: zxq9@REDACTED (zxq9) Date: Tue, 17 Sep 2019 12:51:09 +0900 Subject: [erlang-questions] lists In-Reply-To: References: Message-ID: <2c4ca944-b69a-bef2-c790-e17325e5b1c9@zxq9.com> On 2019/09/17 8:44, Sam Overdorf wrote: > How do I tell the difference between: > "sam1" > ["sam1","sam2"] > They are both lists but behave differently when I apply > [H|T] = List. > H = "s" > H = "sam1". > The second one is what I want and then stop breaking them down. "sam1" is syntactic sugar for [$s, $a, $m, $1]. ["sam1", "sam2"] is likewise syntactic sugar for [[$s, $a, $m, $1], [$s, $a, $m, $1]] With this in mind, if I want to take the first 's' character from "sam1" I can do: [H | _] = "sam1". If I want to get the first character from the second string in a list of strings I could do this: 1> [_, [H | _]] = ["sam", "bob"]. ["sam","bob"] 2> H. 98 3> $b. 98 The syntax $[character] is a shorthand for the unicode codepoint of a given character. 98 is the numeric value of the character 'b', so that is what we see there. With this in mind, I can construct the string/list "sam" by finding the codepoints for the characters 's', 'a', and 'm' and putting the numeric values in a list: 4> $s. 115 5> $a. 97 6> $m. 109 7> [115, 97, 109]. "sam" Play around with lists inside of lists a bit and you'll find that a list of strings is just a nested list. There are many convenience functions for operating over deep lists of strings like this in the unicode and string modules. -Craig From raoknz@REDACTED Tue Sep 17 08:14:15 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Tue, 17 Sep 2019 18:14:15 +1200 Subject: [erlang-questions] lists In-Reply-To: References: Message-ID: "sam1" is a list of length 4 whose first element is the integer $s. ["sam1","sam2"] is a list 2 whose first element is the list "sam1". I'm guessing that you're heading towards a problem that I discussed in my Prolog book: "How do I design a data structure in this language?" The basic rule is - what are the different situations I need to represent? - for each of these different situations, what additional information is there? - decide how to represent that context-dependent information - in Prolog, give each situation its own function symbol; in Erlang, consider the {Case_Label,Case_Dependent,Info...} approach. Whatever you do, make each case OBVIOUSLY different by a trivial pattern match (or a similarly trivial guard test in Erlang). - if some cases have one chunk of info and that is a sequence, you might use a bare list but ONLY for one. With a well-designed Erlang data structure, there won't be any point in asking the question 'how do I tell the difference between "sam1" and ["sam1","sam2"]' because you will have made sure it is never the case that both are live possibilities. For example, if you want to model JSON, you might choose true | false | null -- the corresponding Erlang atom Number -- the same number String -- {string,String} [T1,....,Tn] -- {array,[T'1,....,T'n]} {K1:V1,...,Kn:Vn} -- [{K1,V1},...,{Kn,Vn}] Of course in current Erlang you might use a 'map' for the last. The point is that {string,"..."} and {array,[...]} are trivially easy to distinguish. You might want to give some consideration to using <<"sam1">> and [<<"sam1">>,<<"sam2">>] instead. Perhaps you could go into more detail about what you want to achieve. On Tue, 17 Sep 2019 at 15:44, Sam Overdorf wrote: > How do I tell the difference between: > "sam1" > ["sam1","sam2"] > They are both lists but behave differently when I apply > [H|T] = List. > H = "s" > H = "sam1". > The second one is what I want and then stop breaking them down. > > Weird problem... > > Thanks, > Sam > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sperber@REDACTED Tue Sep 17 11:04:12 2019 From: sperber@REDACTED (Michael Sperber) Date: Tue, 17 Sep 2019 11:04:12 +0200 Subject: [erlang-questions] Call for Contributions: BOB 2020 [Feb 28, Deadline Nov 8] Message-ID: BOB Conference 2020 "What happens when we use what's best for a change?" http://bobkonf.de/2020/cfc.html Berlin, February 28 Call for Contributions Deadline: November 8, 2019 You are actively engaged in advanced software engineering methods, implement ambitious architectures and are open to cutting-edge innovation? Attend this conference, meet people that share your goals, and get to know the best software tools and technologies available today. We strive to offer a day full of new experiences and impressions that you can use to immediately improve your daily life as a software developer. If you share our vision and want to contribute, submit a proposal for a talk or tutorial! NOTE: The conference fee will be waived for presenters. Travel expenses will not be covered (for exceptions see "Speaker Grants"). Speaker Grants BOB has Speaker Grants available to support speakers from groups under-represented in technology. We specifically seek women speakers and speakers who are not be able to attend the conference for financial reasons. Shepherding ----------- The program committee offers shepherding to all speakers. Shepherding provides speakers assistance with preparing their sessions, as well as a review of the talk slides. Topics ------ We are looking for talks about best-of-breed software technology, e.g.: - functional programming - persistent data structures and database - event-based modelling and architectures - types - formal methods for correctness and robustness - abstractions for concurrency and parallelism - metaprogramming - probabilistic programming - math and programming - controlled side effects - beyond REST and SOAP - effective abstractions for data analytics - ... everything really that isn?t mainstream, but you think should be. Presenters should provide the audience with information that is practically useful for software developers. We're especially interested in experience reports. Other topics are also relevant, e.g.: - introductory talks on technical background - overviews of a given field - demos and how-tos Requirements ------------- We accept proposals for presentations of 45 minutes (40 minutes talk + 5 minutes questions), as well as 90 minute tutorials for beginners. The language of presentation should be either English or German. Your proposal should include (in your presentation language of choice): - An abstract of max. 1500 characters. - A short bio/cv - Contact information (including at least email address) - A list of 3-5 concrete ideas of how your work can be applied in a developer's daily life - additional material (websites, blogs, slides, videos of past presentations, ?) Submit here: https://bobcfc.active-group.de/en/bob2020/cfp Organisation ------------ - Direct questions to contact at bobkonf dot de - Proposal deadline: November 8, 2019 - Notification: November 22, 2019 - Program: December 6, 2019 Program Committee ----------------- (more information here: https://bobkonf.de/2020/programmkomitee.html) - Matthias Fischmann, Wire - Matthias Neubauer, SICK AG - Nicole Rauch, Softwareentwicklung und Entwicklungscoaching - Michael Sperber, Active Group - Stefan Wehr, factis research Scientific Advisory Board - Annette Bieniusa, TU Kaiserslautern - Torsten Grust, Uni T?bingen - Peter Thiemann, Uni Freiburg From v@REDACTED Tue Sep 17 13:39:33 2019 From: v@REDACTED (Valentin Micic) Date: Tue, 17 Sep 2019 13:39:33 +0200 Subject: [erlang-questions] lists In-Reply-To: References: Message-ID: <77BEE6F1-E88F-470D-B51C-14D1F05F23F9@micic.co.za> > On 17 Sep 2019, at 01:44, Sam Overdorf wrote: > > How do I tell the difference between: > "sam1" > ["sam1","sam2"] > They are both lists but behave differently when I apply > [H|T] = List. > H = "s" > H = "sam1". > The second one is what I want and then stop breaking them down. > > Weird problem... > > Thanks, > Sam I don?t think you could accomplish this without writing some additional code. A naive example may be: % --------- % Normalize % --------- get_sam( [[E|L]|_] ) -> get_sam( L, [E] ); get_sam( [[]|T] ) -> get_sam( T ); get_sam( [E|L] ) -> get_sam( L, [E] ); get_sam( _ ) -> throw( no_sam ). % -------------- % Accumulate SAM % -------------- get_sam( [E|L], Acc ) -> get_sam( L, [E|Acc] ); get_sam( [], Acc ) -> lists:reverse( Acc ). NOTE: Didn?t test it, but it should achieve what you?ve described. Kind regards V/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue Sep 17 15:13:11 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 17 Sep 2019 15:13:11 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: Message-ID: <20190917131311.GA63922@erix.ericsson.se> On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: > On 2019/09/16 11:49, Richard O'Keefe wrote: > > The fact that L is a well-formed list is verified N times, > > for a total cost O(N * |L|).? But with the current definition, > > the cost is O(N), independent of |L|. > > Hm... just to beat a dead horse...I suppose we could get away with a > single check: > > -export([append/2]). > > append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). > > combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; > combine([], Ys) -> Ys. No. lists:append/2 can be used and *is* used by existing code to construct improper lists. It would break backwards compatibility. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From henrik.x.nord@REDACTED Tue Sep 17 16:18:10 2019 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Tue, 17 Sep 2019 14:18:10 +0000 Subject: [erlang-questions] Patch package OTP 22.1 released Message-ID: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Erlang/OTP 22.1 is the first service release for the 22 major release with new features, improvements as well as bugfixes Potential Incompatibilities * Mnesia: Transactions with sticky locks could with async_asym transactions be committed in the wrong order, since asym transactions are spawned on the remote nodes. To fix this bug the communication protocol between mnesia nodes had to be updated, thus mnesia will no longer be able to connect to nodes earlier than mnesia-4.14 , OTP-19.0. * Stdlib: Debugging of time-outs in gen_statem has been improved. Starting a time-out is now logged in sys:log and sys:trace. Running time-outs are visible in server crash logs, and with sys:get_status. Due to this system events {start_timer, Action, State} and {insert_timout, Event, State} have been added, which may surprise tools that rely on the format of these events. New features: The EventContent of a running time-out can be updated with {TimeoutType, update, NewEventContent}. Running time-outs can be cancelled with {TimeoutType, cancel} which is more readable than using Time = infinity.{rel, Name, Vsn, RelApps, Opts}. Highlights Compiler: * erlc can now automatically use a compile server to avoid starting an Erlang system for each file to be compiled in a multi-file project. See the documentation for how to enable it. Standard libraries: * SSL: Basic support for TLS 1.3 Client for experimental use. For more information see the Standards Compliance chapter of the User's Guide. * crypto: The Message Authentication Codes (MAC) CMAC, HMAC and Poly1305 are unified into common functions in the New Crypto API. See the manual for CRYPTO. cipher_info/1 functions returns maps with information about the hash or cipher in the argument. For more details see http://erlang.org/download/otp_src_22.1.readme Pre built versions for Windows can be fetched here: http://erlang.org/download/otp_win32_22.1.exe http://erlang.org/download/otp_win64_22.1.exe Online documentation can be browsed here: http://erlang.org/doc/search/ The source tarball can be fetched here: http://erlang.org/download/otp_src_22.1.tar.gz The documentation can be fetched here: http://erlang.org/download/otp_doc_html_22.1.tar.gz The man pages can be fetched here: http://erlang.org/download/otp_doc_man_22.1.tar.gz The Erlang/OTP source can also be found at GitHub on the official Erlang repository: https://github.com/erlang/otp OTP-22.1 Thank you for all your contributions! -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Sep 17 16:25:11 2019 From: zxq9@REDACTED (zxq9) Date: Tue, 17 Sep 2019 23:25:11 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <20190917131311.GA63922@erix.ericsson.se> References: <20190917131311.GA63922@erix.ericsson.se> Message-ID: <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> On 2019/09/17 22:13, Raimo Niskanen wrote: > On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: >> On 2019/09/16 11:49, Richard O'Keefe wrote: >>> The fact that L is a well-formed list is verified N times, >>> for a total cost O(N * |L|).? But with the current definition, >>> the cost is O(N), independent of |L|. >> >> Hm... just to beat a dead horse...I suppose we could get away with a >> single check: >> >> -export([append/2]). >> >> append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). >> >> combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; >> combine([], Ys) -> Ys. > > No. > > lists:append/2 can be used and *is* used by existing code to construct > improper lists. It would break backwards compatibility. The accept/2 -> combine/2 combination accepts improper lists just fine. That was the whole point and I addressed the detail of actually *needing* to accept improper lists for the sake of legacy code. It only checks whether the 'Ys' is a list *at all* before proceeding. The beaten horse is now absolutely craven for some abuse. -Craig From siraaj@REDACTED Tue Sep 17 22:22:12 2019 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 17 Sep 2019 16:22:12 -0400 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: On Sat, Sep 14, 2019 at 2:51 PM Lo?c Hoguin wrote: > > On 14/09/2019 20:42, Siraaj Khandkar wrote:> 0.9.999-pre15-beta17 > becomes 1.0.0 as soon as any other software > > package relies on it as a dependency. > > Wishful thinking. > > In practice those other packages will happily rely on software that > is far from ready for general consumption, let alone documented. > > Even if you depend on an 1.0.0 library, if it's not documented then you > don't have any way of knowing what is considered internal and might > break in the future. > > Semver is a useful tool but it doesn't guarantee much on its own. Of course. We can write fiction in any language. To know facts, scientific method is still needed :) Yet, language is still immensely useful! From gavinmroy@REDACTED Tue Sep 17 22:27:09 2019 From: gavinmroy@REDACTED (Gavin M. Roy) Date: Tue, 17 Sep 2019 16:27:09 -0400 Subject: [erlang-questions] [ANN] samovar 1.0.0 In-Reply-To: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> References: <753df664-f8e6-c4ce-4a95-1ec99fde6ba1@wanadoo.fr> Message-ID: Thanks for this. It?s a shame that people used your thread announcing a contribution to the Erlang ecosystem by having a debate about the semvar standard. Keep up the good work! Gavin On Sat, Sep 14, 2019 at 1:06 PM PAILLEAU Eric wrote: > Hi ! > > samovar 1.0.0 has been released ! > > samovar is an Erlang library implementing SEMVER standard. > > Implementation is Erlang/OTP friendly by allowing use of old OTP release > version syntax both for version and ranges. > > Available on github[1] and hex.pm[2] . > > Enjoy ! > > [1] https://github.com/crownedgrouse/samovar > [2] https://hex.pm/packages/samovar > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed Sep 18 00:08:37 2019 From: vasdeveloper@REDACTED (Theepan) Date: Wed, 18 Sep 2019 03:38:37 +0530 Subject: [erlang-questions] Duplicate Entries in Mnesia Message-ID: Hi There, Once in a while our Mnesia database tables (of default type : SET) find duplicate records. The whole recod duplicates I mean, not just the key duplicates. We are using dirty reads/writes very often to cater to speed requirements. Running on Erlang/OTP 18.0 Is thee are known issue regarding? Best, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Sep 18 14:00:50 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 18 Sep 2019 14:00:50 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> Message-ID: <20190918120050.GA65939@erix.ericsson.se> On Tue, Sep 17, 2019 at 11:25:11PM +0900, zxq9 wrote: > On 2019/09/17 22:13, Raimo Niskanen wrote: > > On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: > >> On 2019/09/16 11:49, Richard O'Keefe wrote: > >>> The fact that L is a well-formed list is verified N times, > >>> for a total cost O(N * |L|).? But with the current definition, > >>> the cost is O(N), independent of |L|. > >> > >> Hm... just to beat a dead horse...I suppose we could get away with a > >> single check: > >> > >> -export([append/2]). > >> > >> append(Xs, Ys) when is_list(Ys) -> combine(Xs,Ys). > >> > >> combine([X | Xs], Ys) -> [X | combine(Xs, Ys)]; > >> combine([], Ys) -> Ys. > > > > No. > > > > lists:append/2 can be used and *is* used by existing code to construct > > improper lists. It would break backwards compatibility. ...to construct improper lists even as in appending a tail element (that is need not be a list) to a list... > > The accept/2 -> combine/2 combination accepts improper lists just fine. > That was the whole point and I addressed the detail of actually > *needing* to accept improper lists for the sake of legacy code. It only > checks whether the 'Ys' is a list *at all* before proceeding. > > The beaten horse is now absolutely craven for some abuse. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From zxq9@REDACTED Wed Sep 18 23:54:29 2019 From: zxq9@REDACTED (zxq9) Date: Thu, 19 Sep 2019 06:54:29 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <20190918120050.GA65939@erix.ericsson.se> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> Message-ID: <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> On 2019/09/18 21:00, Raimo Niskanen wrote: > On Tue, Sep 17, 2019 at 11:25:11PM +0900, zxq9 wrote: >> On 2019/09/17 22:13, Raimo Niskanen wrote: >>> On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: >>> No. >>> >>> lists:append/2 can be used and *is* used by existing code to construct >>> improper lists. It would break backwards compatibility. > > ...to construct improper lists even as in appending a tail element (that is > need not be a list) to a list... At least here we can put down a line and say "that's a violation of the typespec", because it is: http://erlang.org/doc/man/lists.html#append-2 So if we want to continue to use append/2 to *create* improper lists (which I've never once encountered anywhere in the last decade, but maybe you've got some examples of super wizard code where this is necessary?) then the typespec should be changed, otherwise I think it isn't unreasonable to do at +O(1) cost at runtime what Dialyzer would do, or just leave things as they are and actively discourage deliberate construction of improper lists via append/2. I'm not particularly opposed to changing the typespec to say "List2 can be whatever because who cares whether lists are lists or not? Let's ditch type sanity because a handful of people feel like it! WOOO!" but I *do* believe it is inconsistent to the extreme to say "Well, we're going to have Dialyzer (and all users who read the documentation... har har har! Joke's on them!) believe that both arguments must be lists!" and then in unofficial -- but heavily referenced -- discussion have an OTP developer say "well, append/2 is used by some people to deliberately construct improper lists, so let's just ignore the discussion entirely even though there is actually a way to efficiently check this at runtime". -Craig From frank.muller.erl@REDACTED Thu Sep 19 00:17:00 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 19 Sep 2019 00:17:00 +0200 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: Anything changed regarding wx (wxmac 3.0.4 from brew) on macOS 10.13.6 with this release? Checking out Erlang/OTP git repository from https://github.com/erlang/otp.git... Building Erlang/OTP 22.1 from git, please wait... APPLICATIONS DISABLED (See: /Users/frankm/.kerl/builds/22.1/otp_build_git.log) * debugger : User gave --without-debugger option DOCUMENTATION INFORMATION (See: /Users/frankm/.kerl/builds/22.1/otp_build_git.log) * documentation : * fop is missing. * Using fakefop to generate placeholder PDF files. Build failed. "_ei_x_new", referenced from: _main in erl_call-cda0e1.o "_ei_x_new_with_version", referenced from: _main in erl_call-cda0e1.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [/Users/frankm/.kerl/builds/22.1/otp_src_git/lib/erl_interface/bin/x86_64-apple-darwin17.7.0/erl_call] Error 1 make[2]: *** [opt] Error 2 make[1]: *** [opt] Error 2 make: *** [build_erl_interface] Error 2 22.0.7 compiled perfectly? /Frank Erlang/OTP 22.1 is the first service release for the 22 major release with > new features, improvements as well as bugfixes > > Potential Incompatibilities > > - Mnesia: Transactions with sticky locks could with async_asym > transactions be committed in the wrong order, since asym transactions are > spawned on the remote nodes. To fix this bug the communication protocol > between mnesia nodes had to be updated, thus mnesia will no longer be able > to connect to nodes earlier than mnesia-4.14 , OTP-19.0. > - Stdlib: Debugging of time-outs in gen_statem has been improved. > Starting a time-out is now logged in sys:log and sys:trace. Running > time-outs are visible in server crash logs, and with sys:get_status. Due to > this system events {start_timer, Action, State} and {insert_timout, Event, > State} have been added, which may surprise tools that rely on the format of > these events. New features: The EventContent of a running time-out can be > updated with {TimeoutType, update, NewEventContent}. Running time-outs can > be cancelled with {TimeoutType, cancel} which is more readable than using > Time = infinity.{rel, Name, Vsn, RelApps, Opts}. > > > Highlights > > Compiler: > > - erlc can now automatically use a compile server to avoid starting an > Erlang system for each file to be compiled in a multi-file project. See the > documentation for how to enable it. > > > Standard libraries: > > - SSL: Basic support for TLS 1.3 Client for experimental use. For more > information see the Standards Compliance chapter of the User's Guide. > - crypto: The Message Authentication Codes (MAC) CMAC, HMAC and > Poly1305 are unified into common functions in the New Crypto API. See the > manual for CRYPTO. cipher_info/1 functions returns maps with > information about the hash or cipher in the argument. > > > For more details see > http://erlang.org/download/otp_src_22.1.readme > > Pre built versions for Windows can be fetched here: > http://erlang.org/download/otp_win32_22.1.exe > http://erlang.org/download/otp_win64_22.1.exe > > Online documentation can be browsed here: > http://erlang.org/doc/search/ > > The source tarball can be fetched here: > > http://erlang.org/download/otp_src_22.1.tar.gz > > The documentation can be fetched here: > > http://erlang.org/download/otp_doc_html_22.1.tar.gz > > The man pages can be fetched here: > > http://erlang.org/download/otp_doc_man_22.1.tar.gz > > The Erlang/OTP source can also be found at GitHub on the official Erlang > repository: > > https://github.com/erlang/otp > > OTP-22.1 > > > Thank you for all your contributions! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Thu Sep 19 03:33:28 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Thu, 19 Sep 2019 13:33:28 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> Message-ID: I'm sorry, but what O(1) runtime check are we talking about here? I don't know of any O(1) way to test "is this a proper list" in Erlang. A little bit of history here. Erlang has roots in Strand88 which has roots in Prolog which (in its current form) is an "Edinburgh" language which has roots in Pop-2 and Lisp. In Prolog, you can construct a term [H|T] before you know what H and/or T are. Indeed, [1,2,...,1000000|T] is possible. When eventually T is bound, the runtime system has no way of telling that it is used as the final tail of an incomplete list. In Lisp, (cons H T) must know what H and T are. So (set! X (cons 1 (cons 2 (cons 3 nil)))) (list? X) is possible. But it's pretty much useless, because the fact that X is a list *now* doesn't mean it will *always* be a list. Things like (set-cdr! X 42) are possible, as indeed are things like (set-cdr! X X) I used Scheme syntax here because Scheme has list? and Lisp doesn't. Pop-2, the rival language where Edinburgh Prolog was developed, is an interesting case. It's a strict imperative language that also has lazy lists. So function islist x; if x.ispair then x.back -> x; x.ispair or x = nil or x.isfunc else x = nil close end; takes O(1) time, recognises cons(1,[]) as a list, rejects cons(1,2), but given what *might* be a lazy list guesses that it is, rather than expand what might well be an infinite sequence. Thanks to lazy lists, x.islist might be true *now*, but after evaluating it one step, it might be false quite soon. So in the 1960s and 1970s, programmers using Lisp, Pop-2, and Prolog got used to the idea that "is this a well-formed list" was somewhere between impossible to implement, usable only at the instant of testing, or dangerously expensive. Lisp and Pop-2 settled on having LISTP or islist tests that are fast but unreliable. Prolog settled on using pattern matching. And Erlang followed Lisp. Returning now to Erlang. Historically, just like Lisp, Pop-2, and Prolog, Erlang uses the *same* data type for arbitrary pairs and for nodes in a list. [1|2] is legal Erlang data. This means that we can do fakeout(0) -> 0; fakeout(N) -> [N | fakeout(N-1)]. ... [a,b] ++ fakeout(1000000) Because Erlang * uses only strict evaluation, so all data are known in full * does not allow data to be changed it _would_ be possible to change the data representation used by Erlang to distinguish between - a pair whose tl is a proper list - a pair whose tl is not a proper list. This would allow an "is_proper_list/1" test that is not only constant time but fast, at the price of slowing down every constructive use of [H|T]. Now no-one who is unwilling to make a speed-for-safety tradeoff would be using Erlang in the first place. So the community may well decide that this major overhaul to BEAm is worth while. But until such a change is made, there is no O(1) test that we can use in (++) that will guarantee the result is well formed. Now I am a bear of very little brain, and I haven't been getting as much sleep as I should lately, so it would be the very reverse of surprising if I was wrong about this. But if I am, it would be very much appreciated if people saying that we should use an O(1) test in appending would actually say what test they are talking about. (Hint: is_list/1 is *not* that test.) On Thu, 19 Sep 2019 at 09:54, zxq9 wrote: > On 2019/09/18 21:00, Raimo Niskanen wrote: > > On Tue, Sep 17, 2019 at 11:25:11PM +0900, zxq9 wrote: > >> On 2019/09/17 22:13, Raimo Niskanen wrote: > >>> On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: > >>> No. > >>> > >>> lists:append/2 can be used and *is* used by existing code to construct > >>> improper lists. It would break backwards compatibility. > > > > ...to construct improper lists even as in appending a tail element (that > is > > need not be a list) to a list... > > At least here we can put down a line and say "that's a violation of the > typespec", because it is: > http://erlang.org/doc/man/lists.html#append-2 > > So if we want to continue to use append/2 to *create* improper lists > (which I've never once encountered anywhere in the last decade, but > maybe you've got some examples of super wizard code where this is > necessary?) then the typespec should be changed, otherwise I think it > isn't unreasonable to do at +O(1) cost at runtime what Dialyzer would > do, or just leave things as they are and actively discourage deliberate > construction of improper lists via append/2. > > I'm not particularly opposed to changing the typespec to say "List2 can > be whatever because who cares whether lists are lists or not? Let's > ditch type sanity because a handful of people feel like it! WOOO!" but I > *do* believe it is inconsistent to the extreme to say "Well, we're going > to have Dialyzer (and all users who read the documentation... har har > har! Joke's on them!) believe that both arguments must be lists!" and > then in unofficial -- but heavily referenced -- discussion have an OTP > developer say "well, append/2 is used by some people to deliberately > construct improper lists, so let's just ignore the discussion entirely > even though there is actually a way to efficiently check this at runtime". > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maximfca@REDACTED Thu Sep 19 03:45:25 2019 From: maximfca@REDACTED (Maxim Fedorov) Date: Wed, 18 Sep 2019 18:45:25 -0700 Subject: [erlang-questions] [ANN] power_shell 1.1.2 has been released Message-ID: power_shell is an application providing a convenient way to evaluate functions extracted from compiled *.beam file, or read from *.erl source. How do we use it at WhatsApp? Debugging on production machines: evaluating functions that are not exported. For example, dist_util.erl contains gen_challenge/0, but it is not accessible without making a change to OTP kernel application. Using power_shell:eval(dist_util, gen_challenge, []) yields the desired result. Common Tests: testing functions that are not exported. We want to test against production build, providing production API. Development: changing *.erl file and immediately evaluating functions directly from *.erl file (no need to c(mymodule) to compile and load updated code). Package is available on hex: https://hex.pm/packages/power_shell Source code is available on GitHub: https://github.com/WhatsApp/power_shell Enjoy! From raoknz@REDACTED Thu Sep 19 06:06:33 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Thu, 19 Sep 2019 16:06:33 +1200 Subject: [erlang-questions] [ANN] power_shell 1.1.2 has been released In-Reply-To: References: Message-ID: The example is power_shell:eval(dist_util, gen_challenge, []). With the arguments being Module, Function, Arguments, why is it not called power_shell:apply/3? Since I'm currently trying to learn PowerShell, I was initially confused by the module name. On Thu, 19 Sep 2019 at 16:02, Maxim Fedorov wrote: > power_shell is an application providing a convenient way to evaluate > functions extracted from compiled *.beam file, or read from *.erl > source. > > How do we use it at WhatsApp? > > Debugging on production machines: evaluating functions that are not > exported. For example, dist_util.erl contains gen_challenge/0, but it > is not accessible without making a change to OTP kernel application. > Using power_shell:eval(dist_util, gen_challenge, []) yields the > desired result. > > Common Tests: testing functions that are not exported. We want to test > against production build, providing production API. > > Development: changing *.erl file and immediately evaluating functions > directly from *.erl file (no need to c(mymodule) to compile and load > updated code). > > Package is available on hex: > https://hex.pm/packages/power_shell > > Source code is available on GitHub: > https://github.com/WhatsApp/power_shell > > Enjoy! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 19 08:58:19 2019 From: zxq9@REDACTED (zxq9) Date: Thu, 19 Sep 2019 15:58:19 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> Message-ID: <5be6e738-f56d-759b-c2c9-1afddc470017@zxq9.com> On 2019/09/19 10:33, Richard O'Keefe wrote: > I'm sorry, but what O(1) runtime check are we talking about here? > I don't know of any O(1) way to test "is this a proper list" in > Erlang. Not checking for whether something is a properly formed list (the whole objection some have is that append/2 should be able to accept an improper list as the second argument) but merely checking whether the second argument is a list of any form at all. A single is_list/1 check guarding the leading clause can do that. The discussion has gone through: A1: "The typespec says both arguments must be lists" B1: "Use Dialyzer" A2: "The check should be dynamic at runtime" B2: "Checking for *proper* lists is too much overhead" A3: "We should accept improper lists as a second argument" B3: "Then is_list/2 could check the 2nd arg on entry only" A4: "Some people use append/2 to *construct* improper lists" B4: "Then why do we even have a typespec at all?" My position is that B2 and A3 are valid, and B3 is a reasonable compromise that shouldn't break legacy code. I think A4 is unreasonably inconsistent, though, and would wonder why we're even going to have a typespec. -Craig From raoknz@REDACTED Thu Sep 19 10:29:15 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Thu, 19 Sep 2019 20:29:15 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <5be6e738-f56d-759b-c2c9-1afddc470017@zxq9.com> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> <5be6e738-f56d-759b-c2c9-1afddc470017@zxq9.com> Message-ID: "Not checking for whether something is a properly formed list (the whole objection some have is that append/2 should be able to accept an improper list as the second argument) but merely checking whether the second argument is a list of any form at all." But what the heck is the point of *that*? Why is it important for [a] ++ c to fail but OK for [a] ++ [b|c] to succeed? I'm sorry, but that makes no sense to me at all. On Thu, 19 Sep 2019 at 18:58, zxq9 wrote: > On 2019/09/19 10:33, Richard O'Keefe wrote: > > I'm sorry, but what O(1) runtime check are we talking about here? > > I don't know of any O(1) way to test "is this a proper list" in > > Erlang. > > Not checking for whether something is a properly formed list (the whole > objection some have is that append/2 should be able to accept an > improper list as the second argument) but merely checking whether the > second argument is a list of any form at all. > > A single is_list/1 check guarding the leading clause can do that. > > The discussion has gone through: > > A1: "The typespec says both arguments must be lists" > B1: "Use Dialyzer" > > A2: "The check should be dynamic at runtime" > B2: "Checking for *proper* lists is too much overhead" > > A3: "We should accept improper lists as a second argument" > B3: "Then is_list/2 could check the 2nd arg on entry only" > > A4: "Some people use append/2 to *construct* improper lists" > B4: "Then why do we even have a typespec at all?" > > My position is that B2 and A3 are valid, and B3 is a reasonable > compromise that shouldn't break legacy code. I think A4 is unreasonably > inconsistent, though, and would wonder why we're even going to have a > typespec. > > -Craig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me@REDACTED Thu Sep 19 12:30:14 2019 From: me@REDACTED (mko) Date: Thu, 19 Sep 2019 22:30:14 +1200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> <5be6e738-f56d-759b-c2c9-1afddc470017@zxq9.com> Message-ID: Hi, Richard, Thanks for explaining the fascinating tradition behind append/1 in Lisp and Prolog. I totally agree with you on the simplicity and performance aspect of append/1 and append/2. I doesn?t make sense at all to add the O(n) check for ?proper list?. I remember when I was following a beginner tutorial on Prolog, I?ve found the append/1 confused me same as Karlo. I think most people including me is confused append/1 with something like flat (not lists:flatten/1) or chain in other languages like the the new addition: flat method of Array in Javascript: >[[1, 2], 3].flat() [1, 2, 3] >[[1, [2]], 3].flat() [1, [2], 3] >[1].flat() [1] >[].flat() [] it only ?flat? once instead ?flat? all the way like erlang lists:flatten/1 does I think it fit to into people?s mindset about ?flat once? of an ordered collection and return the same type. simple implementation in erlang would be: flat([]) -> []; flat([[]|T]) -> flat(T); flat([[H1|T1]|T]) -> [H1|flat([T1|T])]; flat([H|T]) -> [H|flat(T)]. >flat([[1,2],3]). [1,2,3] >flat([[1, [2]], 3]). [1,[2],3] >flat([1]). [1] >flat([]). []. The only thing inconsistent is that as you pointed out several times there's no easy way to check the list is a ?proper list? without walking to the end of the list, > flat([1|2]). ** exception error: no function clause matching lists2:flat(2) but I think it?s ok to skip the ?proper list? checking and let dialyzer or programmer handle it as you suggested, and so maybe it?s good to add flat/1 to the lists module and make it a clear separation of append/1 and flat/1 which maybe a concept a lot of programmer already familiar with . mko > On 19/09/2019, at 8:29 PM, Richard O'Keefe wrote: > > "Not checking for whether something is a properly formed list (the whole > objection some have is that append/2 should be able to accept an > improper list as the second argument) but merely checking whether the > second argument is a list of any form at all." > > But what the heck is the point of *that*? > Why is it important for [a] ++ c to fail > but OK for [a] ++ [b|c] to succeed? > I'm sorry, but that makes no sense to me at all. > > > > > On Thu, 19 Sep 2019 at 18:58, zxq9 > wrote: > On 2019/09/19 10:33, Richard O'Keefe wrote: > > I'm sorry, but what O(1) runtime check are we talking about here? > > I don't know of any O(1) way to test "is this a proper list" in > > Erlang. > > Not checking for whether something is a properly formed list (the whole > objection some have is that append/2 should be able to accept an > improper list as the second argument) but merely checking whether the > second argument is a list of any form at all. > > A single is_list/1 check guarding the leading clause can do that. > > The discussion has gone through: > > A1: "The typespec says both arguments must be lists" > B1: "Use Dialyzer" > > A2: "The check should be dynamic at runtime" > B2: "Checking for *proper* lists is too much overhead" > > A3: "We should accept improper lists as a second argument" > B3: "Then is_list/2 could check the 2nd arg on entry only" > > A4: "Some people use append/2 to *construct* improper lists" > B4: "Then why do we even have a typespec at all?" > > My position is that B2 and A3 are valid, and B3 is a reasonable > compromise that shouldn't break legacy code. I think A4 is unreasonably > inconsistent, though, and would wonder why we're even going to have a > typespec. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Sep 19 12:36:50 2019 From: zxq9@REDACTED (zxq9) Date: Thu, 19 Sep 2019 19:36:50 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> <5be6e738-f56d-759b-c2c9-1afddc470017@zxq9.com> Message-ID: On 2019/09/19 17:29, Richard O'Keefe wrote: > "Not checking for whether something is a properly formed list (the whole > objection some have is that append/2 should be able to accept an > improper list as the second argument) but merely checking whether the > second argument is a list of any form at all." > > But what the heck is the point of *that*? > Why is it important for? [a] ++ c to fail > but OK for [a] ++ [b|c] to succeed? > I'm sorry, but that makes no sense to me at all. You'd have to ask the OP. The only reason I can think of would be to make sure I didn't change something trivial in code somewhere and wind up passing a record or map to append/2 without realizing it (crash on the first run/test/whatever). But this is what I use Dialyzer for anyway. The point of the thread was the feasibility of making a runtime check to guarantee a crash if the arguments weren't lists instead of merely going with "garbage in, garbage out". I can see some sense to that, but wouldn't prioritize append/2 over all the other functions that expect lists and *might* receive junk -- I certainly wouldn't want to make ALL functions have runtime type checks! The only place I put dynamic checks in my own code is when I'm sanitizing data coming in from the outside. Everything else gets Dialyzed. -Craig From roger@REDACTED Thu Sep 19 19:33:23 2019 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 19 Sep 2019 18:33:23 +0100 Subject: [erlang-questions] Diagnosing gen_server call timeouts Message-ID: I've got a gen_server:call that -- very occasionally -- suffers from a timeout. Obviously this means that my gen_server is already busy doing something else. Is there a way that I can instrument gen_server so that it will log something if it takes too long to return from a callback? That is: my handle_call is timing out because, presumably, there's another handle_call (or handle_info, etc.) that's blocking. I'd like *that* to be logged if it takes longer than, say, 200 milliseconds to return. Or, I guess, if the message queue length is excessive at any point. Caveat: I've got ~20,000 of these gen_server processes, and this only happens intermittently in production, so I'm thinking that tracing *isn't* the right answer here. From siraaj@REDACTED Thu Sep 19 20:42:11 2019 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 19 Sep 2019 14:42:11 -0400 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: On Thu, Sep 19, 2019 at 1:33 PM Roger Lipscombe wrote: > > I've got a gen_server:call that -- very occasionally -- suffers from a > timeout. Obviously this means that my gen_server is already busy doing > something else. > > Is there a way that I can instrument gen_server so that it will log > something if it takes too long to return from a callback? That is: my > handle_call is timing out because, presumably, there's another > handle_call (or handle_info, etc.) that's blocking. I'd like *that* to > be logged if it takes longer than, say, 200 milliseconds to return. > > Or, I guess, if the message queue length is excessive at any point. > > Caveat: I've got ~20,000 of these gen_server processes, and this only > happens intermittently in production, so I'm thinking that tracing > *isn't* the right answer here. Idea from an ignorant (of built-in features) point of view: call handler begins by sends a `{call_started, ID, Other_Interesting_Info}` to a bookkeeping process, then, before replying, `{call_completed, ID}`. Bookkeeper thus always knows the current in-progress calls and the stats for completed ones, so it can now log whichever ones violated your ostensible SLA. This sounds awfully similar to just posting timer start/end messages to StatsD or some such, but is more flexible. From jesper.louis.andersen@REDACTED Thu Sep 19 20:56:03 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 19 Sep 2019 20:56:03 +0200 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: On Thu, Sep 19, 2019 at 7:33 PM Roger Lipscombe wrote: > > Is there a way that I can instrument gen_server so that it will log > something if it takes too long to return from a callback? That is: my > handle_call is timing out because, presumably, there's another > handle_call (or handle_info, etc.) that's blocking. I'd like *that* to > be logged if it takes longer than, say, 200 milliseconds to return. > > The calling process is terminated by means of an exit signal, by calling `exit(timeout)`. You can capture this with an exception handler and then use erlang:process_info/2 to capture the gen_servers state when this happens (or you can try sys:get_state if it does return at some point). Essentially you turn your calling process into the logger. This works insofar there are relatively few of these errors, and it can explain what the called process is sitting and doing while this happens. One thing to be aware of is that if you capture the timeout, then the real reply message might arrive late in your mailbox. You will have to handle this accordingly, should you opt to make the process continue. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Sep 20 02:45:09 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 20 Sep 2019 03:45:09 +0300 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: Copy paste from our video_frame.erl: send_frame(Stream, #video_frame{content = Content, codec = Codec, flavor = Flavor, dts = DTS} = Frame, SourceTag) when is_pid(Stream) -> Frame1 = case SourceTag of undefined -> Frame; _ -> Frame#video_frame{source = SourceTag} end, try gen_server:call(Stream, Frame1) catch exit:{normal, _} -> {error, stopped}; exit:{noproc, _} -> {error, stopped}; exit:{shutdown, _} -> {error, stopped}; exit:{timeout, _}:Stack -> Dict = case process_info(Stream, dictionary) of {dictionary, Dict_} -> Dict_; undefined -> [] end, RemoteStack = process_info(Stream, current_stacktrace), Name = case proplists:get_value(name, Dict, <<"dead stream">>) of N when is_binary(N) -> N; N when is_atom(N) -> N; N -> iolist_to_binary(io_lib:format("~p", [N])) end, Status = proplists:get_value(status, Dict), case Status of S when S == starting_transcoder -> ok; _ -> events:error("failed to send frame ~p,~p,~p,~p, source_tag=~p to ~s (~p) in status ~p\n~p\nremote stack:\n~p", [ Content, Codec, Flavor, round(DTS), Frame1#video_frame.source, Name, Stream, Status, Stack, RemoteStack]) end, % [io:format("~10.. s: ~p~n", [K,V]) || {K,V} <- process_info(Stream)] {error, timeout} end. this code is helping us for years. You should remove video specific stuff. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maximfca@REDACTED Fri Sep 20 05:18:06 2019 From: maximfca@REDACTED (Maxim Fedorov) Date: Thu, 19 Sep 2019 20:18:06 -0700 Subject: [erlang-questions] [ANN] power_shell 1.1.2 has been released In-Reply-To: References: Message-ID: There are two reasons to name is eval/3: 1. Clearly indicate evaluation rather than actual execution. 2. Avoid clash with erlang:apply/3. When power_shell is started as an application (e.g. application:start(power_shell) in Erlang shell), it becomes a part of the shell, so you can just type eval(mod, fun, [args]). As for the project name, it is purely historical, and has no relation to Microsoft. Maxim On Wed, 18 Sep 2019 at 21:06, Richard O'Keefe wrote: > > The example is power_shell:eval(dist_util, gen_challenge, []). > With the arguments being Module, Function, Arguments, why is > it not called power_shell:apply/3? > > Since I'm currently trying to learn PowerShell, I was initially > confused by the module name. > > On Thu, 19 Sep 2019 at 16:02, Maxim Fedorov wrote: >> >> power_shell is an application providing a convenient way to evaluate >> functions extracted from compiled *.beam file, or read from *.erl >> source. >> >> How do we use it at WhatsApp? >> >> Debugging on production machines: evaluating functions that are not >> exported. For example, dist_util.erl contains gen_challenge/0, but it >> is not accessible without making a change to OTP kernel application. >> Using power_shell:eval(dist_util, gen_challenge, []) yields the >> desired result. >> >> Common Tests: testing functions that are not exported. We want to test >> against production build, providing production API. >> >> Development: changing *.erl file and immediately evaluating functions >> directly from *.erl file (no need to c(mymodule) to compile and load >> updated code). >> >> Package is available on hex: >> https://hex.pm/packages/power_shell >> >> Source code is available on GitHub: >> https://github.com/WhatsApp/power_shell >> >> Enjoy! >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From raoknz@REDACTED Fri Sep 20 05:45:22 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Fri, 20 Sep 2019 15:45:22 +1200 Subject: [erlang-questions] [ANN] power_shell 1.1.2 has been released In-Reply-To: References: Message-ID: "1. Clearly indicate evaluation rather than actual execution." What does that mean? Execution is evaluation is execution. On Fri, 20 Sep 2019 at 15:20, Maxim Fedorov wrote: > There are two reasons to name is eval/3: > 1. Clearly indicate evaluation rather than actual execution. > 2. Avoid clash with erlang:apply/3. When power_shell is started as an > application (e.g. application:start(power_shell) in Erlang shell), it > becomes a part of the shell, so you can just type eval(mod, fun, > [args]). > > As for the project name, it is purely historical, and has no relation > to Microsoft. > > Maxim > > On Wed, 18 Sep 2019 at 21:06, Richard O'Keefe wrote: > > > > The example is power_shell:eval(dist_util, gen_challenge, []). > > With the arguments being Module, Function, Arguments, why is > > it not called power_shell:apply/3? > > > > Since I'm currently trying to learn PowerShell, I was initially > > confused by the module name. > > > > On Thu, 19 Sep 2019 at 16:02, Maxim Fedorov wrote: > >> > >> power_shell is an application providing a convenient way to evaluate > >> functions extracted from compiled *.beam file, or read from *.erl > >> source. > >> > >> How do we use it at WhatsApp? > >> > >> Debugging on production machines: evaluating functions that are not > >> exported. For example, dist_util.erl contains gen_challenge/0, but it > >> is not accessible without making a change to OTP kernel application. > >> Using power_shell:eval(dist_util, gen_challenge, []) yields the > >> desired result. > >> > >> Common Tests: testing functions that are not exported. We want to test > >> against production build, providing production API. > >> > >> Development: changing *.erl file and immediately evaluating functions > >> directly from *.erl file (no need to c(mymodule) to compile and load > >> updated code). > >> > >> Package is available on hex: > >> https://hex.pm/packages/power_shell > >> > >> Source code is available on GitHub: > >> https://github.com/WhatsApp/power_shell > >> > >> Enjoy! > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Sep 20 13:57:04 2019 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 20 Sep 2019 12:57:04 +0100 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: Er, so I neglected to mention that the gen_server is remote (same host, different node), so process_info doesn't work :( On Fri, 20 Sep 2019 at 01:45, Max Lapshin wrote: > > Copy paste from our video_frame.erl: > > > send_frame(Stream, #video_frame{content = Content, codec = Codec, flavor = Flavor, dts = DTS} = Frame, SourceTag) when is_pid(Stream) -> > Frame1 = case SourceTag of > undefined -> Frame; > _ -> Frame#video_frame{source = SourceTag} > end, > try gen_server:call(Stream, Frame1) > catch > exit:{normal, _} -> > {error, stopped}; > exit:{noproc, _} -> > {error, stopped}; > exit:{shutdown, _} -> > {error, stopped}; > exit:{timeout, _}:Stack -> > Dict = case process_info(Stream, dictionary) of > {dictionary, Dict_} -> Dict_; > undefined -> [] > end, > RemoteStack = process_info(Stream, current_stacktrace), > Name = case proplists:get_value(name, Dict, <<"dead stream">>) of > N when is_binary(N) -> N; > N when is_atom(N) -> N; > N -> iolist_to_binary(io_lib:format("~p", [N])) > end, > Status = proplists:get_value(status, Dict), > case Status of > S when S == starting_transcoder -> ok; > _ -> events:error("failed to send frame ~p,~p,~p,~p, source_tag=~p to ~s (~p) in status ~p\n~p\nremote stack:\n~p", [ > Content, Codec, Flavor, round(DTS), Frame1#video_frame.source, > Name, Stream, Status, Stack, RemoteStack]) > end, > % [io:format("~10.. s: ~p~n", [K,V]) || {K,V} <- process_info(Stream)] > {error, timeout} > end. > > > > this code is helping us for years. You should remove video specific stuff. From z@REDACTED Fri Sep 20 14:13:06 2019 From: z@REDACTED (Danil Zagoskin) Date: Fri, 20 Sep 2019 15:13:06 +0300 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: You can use rpc for that: rpc:call(node(RPid), erlang, process_info, [RPid, [dictionary, current_stacktrace]]). On Fri, Sep 20, 2019 at 2:57 PM Roger Lipscombe wrote: > Er, so I neglected to mention that the gen_server is remote (same > host, different node), so process_info doesn't work :( > > On Fri, 20 Sep 2019 at 01:45, Max Lapshin wrote: > > > > Copy paste from our video_frame.erl: > > > > > > send_frame(Stream, #video_frame{content = Content, codec = Codec, flavor > = Flavor, dts = DTS} = Frame, SourceTag) when is_pid(Stream) -> > > Frame1 = case SourceTag of > > undefined -> Frame; > > _ -> Frame#video_frame{source = SourceTag} > > end, > > try gen_server:call(Stream, Frame1) > > catch > > exit:{normal, _} -> > > {error, stopped}; > > exit:{noproc, _} -> > > {error, stopped}; > > exit:{shutdown, _} -> > > {error, stopped}; > > exit:{timeout, _}:Stack -> > > Dict = case process_info(Stream, dictionary) of > > {dictionary, Dict_} -> Dict_; > > undefined -> [] > > end, > > RemoteStack = process_info(Stream, current_stacktrace), > > Name = case proplists:get_value(name, Dict, <<"dead stream">>) of > > N when is_binary(N) -> N; > > N when is_atom(N) -> N; > > N -> iolist_to_binary(io_lib:format("~p", [N])) > > end, > > Status = proplists:get_value(status, Dict), > > case Status of > > S when S == starting_transcoder -> ok; > > _ -> events:error("failed to send frame ~p,~p,~p,~p, > source_tag=~p to ~s (~p) in status ~p\n~p\nremote stack:\n~p", [ > > Content, Codec, Flavor, round(DTS), > Frame1#video_frame.source, > > Name, Stream, Status, Stack, RemoteStack]) > > end, > > % [io:format("~10.. s: ~p~n", [K,V]) || {K,V} <- > process_info(Stream)] > > {error, timeout} > > end. > > > > > > > > this code is helping us for years. You should remove video specific > stuff. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowe@REDACTED Fri Sep 20 20:50:52 2019 From: achowe@REDACTED (Anthony Howe) Date: Fri, 20 Sep 2019 14:50:52 -0400 Subject: [erlang-questions] ASN.1 decode errors In-Reply-To: <07b593e3-5a4f-e27b-9e3c-27dbe9abf5af@snert.com> References: <07b593e3-5a4f-e27b-9e3c-27dbe9abf5af@snert.com> Message-ID: Thank you to those who contacted me concerning this problem. It appears that the issue was with the vendor's (lack of) documentation (and wrong ASN.1 spec. cited by them) not explaining that the GPRS CDRs were wrapped in a binary file format that was not part of the ASN.1 BER encoding. So the Erlang ASN.1 compiler was correct, but the input was offset. Anthony On 2019-09-12 10:29, Anthony Howe wrote: > I've written a couple ASN.1 BER and PER transcoders to JSON for LTE, > TAP3, NRTRDE files before. ( It was Erlang's open source ASN.1 support > that got me to learn the language in the first place. ) I'm trying to > write one now for GPRS charging records (ETSI TS 101 393 V7.4.0 > (2000-02)); I have something that builds, but does not decode the input > and just reports a vague Erlang ASN error. > > I figure the problem revolves around a mismatched set of ASN module > files (which are a frik'n pain to find, but I digress). Anyway, I > figure if I understood the error better it might point me finally in the > right direction. > > So the error is... > > $ ./_build/default/bin/gprsdump -r cdr_0f1 > cdr_0f1: processing... > runtime error: cdr_0f1: {badmatch, > {error, > {asn1, > {invalid_choice_tag, > {1, > <<0,168,161,129,165,128,1,19,131,8,48,36,128, > 1,113,0,0,63,164>>}}}}} > > Reset of stack omitted. > > My interpretation is that initial CallEventRecord choice 1 > (GGSNPDPRecord) is not decoding, but I have no idea what the binary is > suppose to be telling me, other than "You Are Here in the BER input". > > CallEventRecord ::= CHOICE > { > sgsnPDPRecord [0] SGSNPDPRecord, > ggsnPDPRecord [1] GGSNPDPRecord, > sgsnMMRecord [2] SGSNMMRecord, > sgsnSMORecord [3] SGSNSMORecord, > sgsnSMTRecord [4] SGSNSMTRecord > } > > Any clues to would be welcome. > > Right now its looking like I'll have to learn how to decode BER by hand > in order to understand the failure better. Maybe a generic BER to ASN > syntax decoder tool? > > -- Anthony C Howe SnertSoft achowe@REDACTED Twitter: SirWumpus BarricadeMX & Milters http://snert.com/ http://nanozen.info/ http://snertsoft.com/ From roger@REDACTED Fri Sep 20 23:49:30 2019 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 20 Sep 2019 22:49:30 +0100 Subject: [erlang-questions] Diagnosing gen_server call timeouts In-Reply-To: References: Message-ID: In the end, I used 'dbg'; here's the relevant snippet: AssertElapsed = fun(Pid, Call, Req, Then) -> Now = erlang:monotonic_time(), ElapsedMicros = erlang:convert_time_unit(Now - Then, native, microsecond), if ElapsedMicros > 5000 -> io:format("~s ~p(~p) took ~B us\n", [lists:flatten(ec_date:format("c", calendar:universal_time())), Call, Req, ElapsedMicros]); true -> ok end, {message_queue_len, QueueLen} = erlang:process_info(Pid, message_queue_len), if QueueLen > 5 -> io:format("~s ~p\n", [lists:flatten(ec_date:format("c", calendar:universal_time())), erlang:process_info(Pid, messages)]); true -> ok end end. %% TState :: map(Fun -> [{Time, Arg0}]). TState0 = #{}. TFun = fun(_Msg = {trace, _Pid, call, {Mod, Fun, [Arg0 | _]}}, TState) -> % push an entry timestamp. Call = {erlang:monotonic_time(), Arg0}, maps:update_with(Fun, fun(Calls) -> [Call | Calls] end, [Call], TState); (_Msg = {trace, _Pid, return_from, {Mod, Fun, _Arity}, _Result}, TState) -> % pop entry timestamp. [{Then, Arg0} | Calls] = maps:get(Fun, TState), AssertElapsed(Pid, Fun, Arg0, Then), maps:update(Fun, Calls, TState); (Msg, TState) -> io:format("~p\n", [Msg]), TState end. dbg:start(). dbg:tracer(process, {TFun, TState0}). dbg:tpl(Mod, handle_call, '_', dbg:fun2ms(fun(_) -> return_trace() end)). dbg:tpl(Mod, handle_info, '_', dbg:fun2ms(fun(_) -> return_trace() end)). dbg:p(Pid, c). In my particular case, handle_info was occasionally taking ~30 seconds, depending on the input, meaning that if a gen_server:call occurred immediately afterwards, there was a good chance it would time out. Aside: I *always* forget that maps:update_with/4 doesn't apply Fun to Init. On Thu, 19 Sep 2019 at 18:33, Roger Lipscombe wrote: > > I've got a gen_server:call that -- very occasionally -- suffers from a > timeout. Obviously this means that my gen_server is already busy doing > something else. > > Is there a way that I can instrument gen_server so that it will log > something if it takes too long to return from a callback? That is: my > handle_call is timing out because, presumably, there's another > handle_call (or handle_info, etc.) that's blocking. I'd like *that* to > be logged if it takes longer than, say, 200 milliseconds to return. > > Or, I guess, if the message queue length is excessive at any point. > > Caveat: I've got ~20,000 of these gen_server processes, and this only > happens intermittently in production, so I'm thinking that tracing > *isn't* the right answer here. From eric.pailleau@REDACTED Sat Sep 21 10:15:28 2019 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 21 Sep 2019 10:15:28 +0200 Subject: [erlang-questions] [ANN] geas 2.4.6 (Erlang 22.1) Message-ID: <3a54bcc4-0ddd-36c4-6052-5ab0f08dadee@wanadoo.fr> Hi Geas 2.4.6 has been released ! Geas is a tool detecting the runnable official Erlang release window for your project. Geas will tell you also : - what are the offending functions in the beam/source files that reduce the available window. - if some beam files are compiled native. - the installed patches and recommend patches that should be installed depending your code. Geas is available as a module, erlang.mk and rebar 2/3 plugins. Changelog : - Update for Erlang 22.1 detection and database https://github.com/crownedgrouse/geas https://github.com/crownedgrouse/geas/releases/tag/2.4.6 Cheers ! Eric From by@REDACTED Sat Sep 21 12:22:24 2019 From: by@REDACTED (by) Date: Sat, 21 Sep 2019 18:22:24 +0800 Subject: [erlang-questions] C Nodes Tutorial "erl_publish" error Message-ID: Hi, I am going through this tutorial: http://erlang.org/doc/tutorial/cnode.html Compile steps works, but when I try to run the example with below command: $ ./cserver 3456 I got this: erl_publish $ echo $? 1 Does anyone encounter this before? Based on the documentation, seems this issue comes from: if (erl_publish(port) == -1) erl_err_quit("erl_publish"); But what should I do to fix this? I run this on my Mac computer: macOS Sierra 10.12.6 Erlang version: Erlang/OTP 22.0.1 With kind regards, Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Sep 21 18:06:14 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 21 Sep 2019 19:06:14 +0300 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: Cannot compile flussonic with 22.1 https://bugs.erlang.org/browse/ERL-1050 -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sat Sep 21 19:09:41 2019 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 21 Sep 2019 18:09:41 +0100 Subject: [erlang-questions] Logging level of "SSL: Socket error" Message-ID: I thought I'd follow up on http://erlang.org/pipermail/erlang-questions/2018-July/095948.html, wherein (to summarise): Roger: the logging level of "SSL: Socket error" changed from info to error; causes lots of noise in logs. See https://github.com/erlang/otp/commit/6e28a7909c665cc316d657dda02a2b8655ecc5da#diff-5e580136b1f3094b464f9870a474ee67L972 Ingela: so it did; might need reviewing. This is still the case in OTP-22.1; see https://github.com/erlang/otp/blob/OTP-22.1/lib/ssl/src/ssl_connection.erl#L1503 So, following up: any thoughts about downgrading (or making configurable) the log level for this particular (class of) errors? From maximfca@REDACTED Sat Sep 21 21:57:03 2019 From: maximfca@REDACTED (Maxim Fedorov) Date: Sat, 21 Sep 2019 12:57:03 -0700 Subject: [erlang-questions] one-for-one supervisor with concurrent start/shutdown Message-ID: G'Day, OTP implements sequential start/shutdown procedure for rest_for_one and all_for_one supervisors, assuming there are dependencies between children. It requires startup/termination order. simple_one_for_one implements concurrent shutdown. Startup is sequential due to the dynamic nature of this strategy. one_for_one supervisor implements sequential startup and shutdown. Is there any way to do it concurrently? Children are supposed to be independent of each other. Concurrent shutdown is easy to add, it's already done for simple_one_for_one. But startup is implemented as MFA callback, with no message protocol. proc_lib has message protocol internally, but not all children may follow it. Would it make sense to have yet another restart strategy (I'd name it async_one_for_one) ? From soverdor@REDACTED Sun Sep 22 01:05:56 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Sat, 21 Sep 2019 16:05:56 -0700 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: I just tried 22.1 today. My program uses the ftp client. It gets stuck and consumes all of my memory while trying to execute the following command. User = ftp:user(Pid, "ftp", ""), I have to control-c to stop it. At his point all of the memory comes back. I have been having this problem since "21,3". Thanks, Sam Overdorf soverdor@REDACTED On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: > > Cannot compile flussonic with 22.1 > > https://bugs.erlang.org/browse/ERL-1050 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From by@REDACTED Sun Sep 22 10:58:30 2019 From: by@REDACTED (by) Date: Sun, 22 Sep 2019 16:58:30 +0800 Subject: [erlang-questions] C Nodes Tutorial "erl_publish" error In-Reply-To: References: Message-ID: <1B0ABC61-4546-4E76-8CF4-182118F8A725@meetlost.com> Hi, I fixed this by manually run epmd daemon as below: $ epmd -daemon With kind regards, Yao > ? 2019?9?21??18:22?by ??? > > Hi, > > I am going through this tutorial: http://erlang.org/doc/tutorial/cnode.html > > Compile steps works, but when I try to run the example with below command: > $ ./cserver 3456 > > I got this: > erl_publish > > $ echo $? > 1 > > Does anyone encounter this before? > Based on the documentation, seems this issue comes from: > if (erl_publish(port) == -1) > erl_err_quit("erl_publish"); > But what should I do to fix this? > > I run this on my Mac computer: macOS Sierra 10.12.6 > Erlang version: Erlang/OTP 22.0.1 > > With kind regards, > Yao > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Sep 23 08:00:32 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 23 Sep 2019 14:00:32 +0800 Subject: [erlang-questions] lists module functions for maps? Message-ID: Should we have a corollary to lists:keyfind/3 which works on lists of map()? keyfind(Key, MapList) -> Map | false ... and the rest: keydelete(Key, MapList1) -> MapList2 keymap(Fun, Key, MapList1) -> MapList2 keymember(Key, MapList) -> boolean() keymerge(Key, MapList1, MapList2) -> MapList3 keyreplace(Key, MapList1, NewMap) -> MapList2 keysort(Key, MapList1) -> MapList2 keystore(Key, MapList1, NewMap) -> MapList2 keytake(Key, MapList1) -> {value, Map, MapList2} | false -- -Vance From vances@REDACTED Mon Sep 23 08:40:47 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 23 Sep 2019 14:40:47 +0800 Subject: [erlang-questions] [ANN] snmp_collector Message-ID: The snmp application in OTP does include a manager however it is not very useful in real world use cases. For our purposes we started with it but evolved to writing our own manager while making it an incremental addition to the existing snmp application. Additions include SNMP engine ID discovery and NIFs for SNMPv3 crypto. The result is available in our snmp_collector application available here: https://github.com/sigscale/snmp-collector -- -Vance From bram.verburg@REDACTED Mon Sep 23 08:48:18 2019 From: bram.verburg@REDACTED (Bram Verburg) Date: Mon, 23 Sep 2019 09:48:18 +0300 Subject: [erlang-questions] ASN1 exclusive decode failure Message-ID: Hi, I'm trying to decode an OCSP basic response: BasicOCSPResponse ::= SEQUENCE { tbsResponseData ResponseData, signatureAlgorithm AlgorithmIdentifier, signature BIT STRING, certs [0] EXPLICIT SEQUENCE OF Certificate OPTIONAL } I would like the 'certs' element to be returned as-is, basically as a list of DER certificates which I may or may not decode later. I was hoping I could use 'exclusive_decode', with the following asn1config: {exclusive_decode,{ocsp,[ {decode_basic_ocsp_response,['BasicOCSPResponse',[{certs,parts}]]} ]}}. But this fails with the following exception: ** exception error: no function clause matching ocsp:decode_tag_and_length({asn1,"tag failure",16, {1,131072, <<48,130,4,232,48,130,4,228,48,130,3,204,160,3,2,1,2,2,9, 0,227,...>>, <<>>}}) (ocsp.erl, line 2051) in function ocsp:decode_primitive_incomplete/2 (ocsp.erl, line 2026) in call from ocsp:decode_constructed_incomplete/2 (ocsp.erl, line 1828) in call from ocsp:decode_constructed_incomplete/2 (ocsp.erl, line 1830) in call from ocsp:decode_constructed_incomplete/2 (ocsp.erl, line 1808) in call from ocsp:decode_incomplete2/5 (ocsp.erl, line 21848) in call from ocsp:decode_partial_incomplete/3 (ocsp.erl, line 1190) A full BasicOCSPResponse decode of the same binary works fine, returning a list of Certificate records in the certs element, so the source data is valid. Is there any reason why the exclusive_decode 'parts' action might not work for this particular SEQUENCE, perhaps because it is explicit + optional? Thanks, Bram -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Sep 23 10:08:56 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 23 Sep 2019 10:08:56 +0200 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> Message-ID: <20190923080856.GA62052@erix.ericsson.se> On Thu, Sep 19, 2019 at 06:54:29AM +0900, zxq9 wrote: > On 2019/09/18 21:00, Raimo Niskanen wrote: > > On Tue, Sep 17, 2019 at 11:25:11PM +0900, zxq9 wrote: > >> On 2019/09/17 22:13, Raimo Niskanen wrote: > >>> On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: > >>> No. > >>> > >>> lists:append/2 can be used and *is* used by existing code to construct > >>> improper lists. It would break backwards compatibility. > > > > ...to construct improper lists even as in appending a tail element (that is > > need not be a list) to a list... > > At least here we can put down a line and say "that's a violation of the > typespec", because it is: > http://erlang.org/doc/man/lists.html#append-2 You are right. I dug up this discussion: http://erlang.org/pipermail/erlang-questions/2013-March/073022.html And from that I draw these conclusions: * The type specs and documentation describe the intention of the API * If the arguments follow the type spec, so will the return value * If not, it will not, but there is no guarantee for e.g an exception * Code may be more lenient that its type spec So, if you use lists:append/1,2 to construct an improper list you will get a Dialyzer warning, even though the code works. We will not change the implementation of lists:append/1,2 for effeciency and legacy reasons. If anything should change it should be the type spec and documentation, but I see no good reason why. Now you get a warning from Dialyzer for "bad style" according to some definition of it, which many think is a good thing. Dialyzer only makes guarantees about code that follows the type specs. Code that does not follow the type specs might work anyway. I suspect this whole thread originates in a misunderstanding about what type specs specify and not... > > So if we want to continue to use append/2 to *create* improper lists > (which I've never once encountered anywhere in the last decade, but > maybe you've got some examples of super wizard code where this is > necessary?) then the typespec should be changed, otherwise I think it > isn't unreasonable to do at +O(1) cost at runtime what Dialyzer would > do, or just leave things as they are and actively discourage deliberate > construction of improper lists via append/2. > > I'm not particularly opposed to changing the typespec to say "List2 can > be whatever because who cares whether lists are lists or not? Let's > ditch type sanity because a handful of people feel like it! WOOO!" but I > *do* believe it is inconsistent to the extreme to say "Well, we're going > to have Dialyzer (and all users who read the documentation... har har > har! Joke's on them!) believe that both arguments must be lists!" and > then in unofficial -- but heavily referenced -- discussion have an OTP > developer say "well, append/2 is used by some people to deliberately > construct improper lists, so let's just ignore the discussion entirely > even though there is actually a way to efficiently check this at runtime". > > -Craig -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From zxq9@REDACTED Mon Sep 23 10:17:25 2019 From: zxq9@REDACTED (zxq9) Date: Mon, 23 Sep 2019 17:17:25 +0900 Subject: [erlang-questions] behavior of lists:append/1 In-Reply-To: <20190923080856.GA62052@erix.ericsson.se> References: <20190917131311.GA63922@erix.ericsson.se> <0922d654-e767-f978-083f-9ccdaa5ddb37@zxq9.com> <20190918120050.GA65939@erix.ericsson.se> <001a1007-61e3-c1b8-1686-8afa924084e5@zxq9.com> <20190923080856.GA62052@erix.ericsson.se> Message-ID: On 2019/09/23 17:08, Raimo Niskanen wrote: > On Thu, Sep 19, 2019 at 06:54:29AM +0900, zxq9 wrote: >> On 2019/09/18 21:00, Raimo Niskanen wrote: >>> On Tue, Sep 17, 2019 at 11:25:11PM +0900, zxq9 wrote: >>>> On 2019/09/17 22:13, Raimo Niskanen wrote: >>>>> On Mon, Sep 16, 2019 at 10:53:57PM +0900, zxq9 wrote: >>>>> No. >>>>> >>>>> lists:append/2 can be used and *is* used by existing code to construct >>>>> improper lists. It would break backwards compatibility. >>> >>> ...to construct improper lists even as in appending a tail element (that is >>> need not be a list) to a list... >> >> At least here we can put down a line and say "that's a violation of the >> typespec", because it is: >> http://erlang.org/doc/man/lists.html#append-2 > > You are right. > > I dug up this discussion: > http://erlang.org/pipermail/erlang-questions/2013-March/073022.html > > And from that I draw these conclusions: > * The type specs and documentation describe the intention of the API > * If the arguments follow the type spec, so will the return value > * If not, it will not, but there is no guarantee for e.g an exception > * Code may be more lenient that its type spec > > So, if you use lists:append/1,2 to construct an improper list you will get > a Dialyzer warning, even though the code works. > > We will not change the implementation of lists:append/1,2 for effeciency > and legacy reasons. If anything should change it should be the type spec > and documentation, but I see no good reason why. Now you get a warning > from Dialyzer for "bad style" according to some definition of it, which > many think is a good thing. > > Dialyzer only makes guarantees about code that follows the type specs. > Code that does not follow the type specs might work anyway. > > I suspect this whole thread originates in a misunderstanding about what > type specs specify and not... Makes sense. I tend to follow Dialyzer almost religiously on projects where I need hot upgrades, so am probably giving its returns too much weight. I'll bookmark your response to reference the next time this conversation pops up. Regards -Craig From ingela.andin@REDACTED Mon Sep 23 10:47:55 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 23 Sep 2019 10:47:55 +0200 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: Hi! Please report bugs here: https://bugs.erlang.org/secure/Dashboard.jspa And please include more details about the what FTP server you are trying to connect too. Reproducible bugs are much easier to find and fix. Regards Ingela - Erlang/OTP team - Ericsson AB Den s?n 22 sep. 2019 kl 01:06 skrev Sam Overdorf : > I just tried 22.1 today. > My program uses the ftp client. > It gets stuck and consumes all of my memory while trying to execute > the following command. > User = ftp:user(Pid, "ftp", ""), > I have to control-c to stop it. > At his point all of the memory comes back. > I have been having this problem since "21,3". > > Thanks, > Sam Overdorf > soverdor@REDACTED > > > > > On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: > > > > Cannot compile flussonic with 22.1 > > > > https://bugs.erlang.org/browse/ERL-1050 > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Sep 23 11:30:52 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 23 Sep 2019 11:30:52 +0200 Subject: [erlang-questions] Common Test repeat_until and exit status Message-ID: <7b783822-5b08-f286-762a-b25f9e26093e@ninenines.eu> Hello, When I use the test group property repeat_until_all_ok I still get an error status from ct_run (exit status 1) because the tests that failed and were repeated are still counted as errors. For example: Testing ninenines.gun.gun_SUITE: Starting test (with repeated test cases) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - gun_SUITE:shutdown_reason failed on line 399 Reason: timeout - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - gun_SUITE:retry_1 failed on line 324 Reason: timeout - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Testing ninenines.gun.gun_SUITE: *** FAILED test case 17 *** - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - gun_SUITE:retry_fun failed on line 343 Reason: shutdown_too_late - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Testing ninenines.gun.gun_SUITE: *** FAILED test case 18 *** Testing ninenines.gun.gun_SUITE: *** FAILED test case 21 *** - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - gun_SUITE:retry_1 failed on line 324 Reason: timeout - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Testing ninenines.gun.gun_SUITE: *** FAILED test case 46 *** Testing ninenines.gun.gun_SUITE: Stopping repeat operation {repeat_until_all_ok,98} Testing ninenines.gun.gun_SUITE: TEST COMPLETE, 83 ok, 4 failed of 87 test cases [...] make[1]: *** [erlang.mk:6452: ct] Error 1 Is there a way to repeat flaky tests without Common Test counting the test failures as errors? In the example above I want Common Test (ct_run) to return the exit status 0 because the tests were OK after a few tries. This would be very useful to quickly get tests green on environments that are more likely to have timing issues (in cases where it's the test that is the problem, not the application). For example on slower environments, or on Windows/FreeBSD where the network stack doesn't always behave the same as Linux. I spend far too much time on fixing timing issues in tests. Cheers, -- Lo?c Hoguin https://ninenines.eu From karolis.velicka@REDACTED Mon Sep 23 13:07:19 2019 From: karolis.velicka@REDACTED (Karl Velicka) Date: Mon, 23 Sep 2019 14:07:19 +0300 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: I'm afraid I can't see the equivalence in your example - I see a list of tuples as a list of values, whereas a list of maps is a list of containers. In my mind, a more apt comparison would be a list of maps vs a list of lists of tuples, and functions operating on the latter would strike me as an odd addition to stdlib. I'd be curious to know what your usecase for these functions is. Some of them can be trivially implemented: keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) end, MapList). (The example implementation there is inefficient but I hope you get the idea) Some others, like keysort(Key, MapList1) -> MapList2 don't quite make sense to me as the semantics are unclear. I think elaborating on the use cases would help your proposal. All the best, Karl On Mon, 23 Sep 2019 at 09:00, Vance Shipley wrote: > Should we have a corollary to lists:keyfind/3 which works on lists of > map()? > > keyfind(Key, MapList) -> Map | false > > ... and the rest: > > keydelete(Key, MapList1) -> MapList2 > keymap(Fun, Key, MapList1) -> MapList2 > keymember(Key, MapList) -> boolean() > keymerge(Key, MapList1, MapList2) -> MapList3 > keyreplace(Key, MapList1, NewMap) -> MapList2 > keysort(Key, MapList1) -> MapList2 > keystore(Key, MapList1, NewMap) -> MapList2 > keytake(Key, MapList1) -> {value, Map, MapList2} | false > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elbrujohalcon@REDACTED Mon Sep 23 13:15:46 2019 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Mon, 23 Sep 2019 08:15:46 -0300 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> There is some historical context at play here, Karl. Records (which are, in many occasions, the predecessors of maps) are tuples and therefore it?s not uncommon to find code that uses the functions listed by Vance to work with them. For instance? find_user(UserName, Users) -> lists:keyfind(UserName, #user.username, Users). If we move from records to maps as our means to implement users, then we need something like? find_user(UserName, Users) -> case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) of [] -> false; [User|_] -> User end. It would be nice if we don?t need to do that and instead can do? find_user(UserName, Users) -> lists:keyfind_for_maps(UserName, username, Users). Hope this helps :) Brujo Benavides > On 23 Sep 2019, at 08:07, Karl Velicka wrote: > > I'm afraid I can't see the equivalence in your example - I see a list of tuples as a list of values, whereas a list of maps is a list of containers. In my mind, a more apt comparison would be a list of maps vs a list of lists of tuples, and functions operating on the latter would strike me as an odd addition to stdlib. > > I'd be curious to know what your usecase for these functions is. > > Some of them can be trivially implemented: > > keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) end, MapList). > > (The example implementation there is inefficient but I hope you get the idea) > > Some others, like keysort(Key, MapList1) -> MapList2 don't quite make sense to me as the semantics are unclear. > > > I think elaborating on the use cases would help your proposal. > > All the best, > Karl > > On Mon, 23 Sep 2019 at 09:00, Vance Shipley > wrote: > Should we have a corollary to lists:keyfind/3 which works on lists of map()? > > keyfind(Key, MapList) -> Map | false > > ... and the rest: > > keydelete(Key, MapList1) -> MapList2 > keymap(Fun, Key, MapList1) -> MapList2 > keymember(Key, MapList) -> boolean() > keymerge(Key, MapList1, MapList2) -> MapList3 > keyreplace(Key, MapList1, NewMap) -> MapList2 > keysort(Key, MapList1) -> MapList2 > keystore(Key, MapList1, NewMap) -> MapList2 > keytake(Key, MapList1) -> {value, Map, MapList2} | false > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From karolis.velicka@REDACTED Mon Sep 23 13:32:54 2019 From: karolis.velicka@REDACTED (Karl Velicka) Date: Mon, 23 Sep 2019 14:32:54 +0300 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> References: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> Message-ID: Ah, of course - that makes sense. I would still like to argue that maps are a good substitute for only some kinds of records and I see both data structures as more orthogonal than predecessor-successor but this isn't the right place for it and I don't want to derail the discussion (plus, I'm sure I'd find myself out of my depth in that discussion rather quickly!). Thanks for pointing out the obvious that I have somehow missed. On Mon, 23 Sep 2019 at 14:15, Brujo Benavides wrote: > There is some *historical* context at play here, Karl. > Records (which are, in many occasions, the predecessors of maps) are > tuples and therefore it?s not uncommon to find code that uses the functions > listed by Vance to work with them. For instance? > > find_user(UserName, Users) -> > lists:keyfind(UserName, #user.username, Users). > > If we move from records to maps as our means to implement users, then we > need something like? > > find_user(UserName, Users) -> > case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) > of > [] -> false; > [User|_] -> User > end. > > It would be *nice* if we don?t need to do that and instead can do? > > find_user(UserName, Users) -> > lists:keyfind_for_maps(UserName, username, Users). > > Hope this helps :) > > ------------------------------ > *Brujo Benavides * > > > > On 23 Sep 2019, at 08:07, Karl Velicka wrote: > > I'm afraid I can't see the equivalence in your example - I see a list of > tuples as a list of values, whereas a list of maps is a list of containers. > In my mind, a more apt comparison would be a list of maps vs a list of > lists of tuples, and functions operating on the latter would strike me as > an odd addition to stdlib. > > I'd be curious to know what your usecase for these functions is. > > Some of them can be trivially implemented: > > keyfind(Key, MapList) -> hd(lists:filter(fun(Map) -> maps:is_key(Key, Map) > end, MapList). > > (The example implementation there is inefficient but I hope you get the > idea) > > Some others, like keysort(Key, MapList1) -> MapList2 don't quite make > sense to me as the semantics are unclear. > > > I think elaborating on the use cases would help your proposal. > > All the best, > Karl > > On Mon, 23 Sep 2019 at 09:00, Vance Shipley wrote: > >> Should we have a corollary to lists:keyfind/3 which works on lists of >> map()? >> >> keyfind(Key, MapList) -> Map | false >> >> ... and the rest: >> >> keydelete(Key, MapList1) -> MapList2 >> keymap(Fun, Key, MapList1) -> MapList2 >> keymember(Key, MapList) -> boolean() >> keymerge(Key, MapList1, MapList2) -> MapList3 >> keyreplace(Key, MapList1, NewMap) -> MapList2 >> keysort(Key, MapList1) -> MapList2 >> keystore(Key, MapList1, NewMap) -> MapList2 >> keytake(Key, MapList1) -> {value, Map, MapList2} | false >> >> >> -- >> -Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Mon Sep 23 13:44:50 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Mon, 23 Sep 2019 23:44:50 +1200 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: Keep the function names the same but put them in another module, say 'maps:'. On Mon, 23 Sep 2019 at 18:01, Vance Shipley wrote: > > Should we have a corollary to lists:keyfind/3 which works on lists of map()? > > keyfind(Key, MapList) -> Map | false > > ... and the rest: > > keydelete(Key, MapList1) -> MapList2 > keymap(Fun, Key, MapList1) -> MapList2 > keymember(Key, MapList) -> boolean() > keymerge(Key, MapList1, MapList2) -> MapList3 > keyreplace(Key, MapList1, NewMap) -> MapList2 > keysort(Key, MapList1) -> MapList2 > keystore(Key, MapList1, NewMap) -> MapList2 > keytake(Key, MapList1) -> {value, Map, MapList2} | false > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andreas.schultz@REDACTED Mon Sep 23 13:55:19 2019 From: andreas.schultz@REDACTED (Andreas Schultz) Date: Mon, 23 Sep 2019 13:55:19 +0200 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> References: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> Message-ID: Am Mo., 23. Sept. 2019 um 13:15 Uhr schrieb Brujo Benavides < elbrujohalcon@REDACTED>: > There is some *historical* context at play here, Karl. > Records (which are, in many occasions, the predecessors of maps) are > tuples and therefore it?s not uncommon to find code that uses the functions > listed by Vance to work with them. For instance? > > find_user(UserName, Users) -> > lists:keyfind(UserName, #user.username, Users). > > If we move from records to maps as our means to implement users, then we > need something like? > > find_user(UserName, Users) -> > case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, Users) > of > [] -> false; > [User|_] -> User > end. > > It would be *nice* if we don?t need to do that and instead can do? > > find_user(UserName, Users) -> > lists:keyfind_for_maps(UserName, username, Users). > Your example doesn't make sense to me. Why don't you simply use a map with the username as key? Maps are a Key-Value stores. If you need secondary indices, you can always maintain additional maps for them. Non-unique secondary keys can be implemented by using a maps as value. Or use an ets where you can match on additional fields. Adding a linear search like lists:keyfind to maps make no sense in my opinion. Talking about ets, adding a match like API to maps might make sense. If only to make it possible to replace ets with maps without having to change to much code. Regards Andreas > [...] -- Andreas Schultz -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Mon Sep 23 14:02:13 2019 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Mon, 23 Sep 2019 15:02:13 +0300 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: <0773F586-02E8-4BB5-87B6-DFF51D9057CF@gmail.com> Message-ID: You can use a match spec to filter the list ( http://erlang.org/doc/man/ets.html#match_spec_run-2). Pattern matching works well with lists of maps. That's in the case, if you need to keep your data in a list instead of map (eg. by username). Karolis On Mon, Sep 23, 2019 at 2:55 PM Andreas Schultz < andreas.schultz@REDACTED> wrote: > Am Mo., 23. Sept. 2019 um 13:15 Uhr schrieb Brujo Benavides < > elbrujohalcon@REDACTED>: > >> There is some *historical* context at play here, Karl. >> Records (which are, in many occasions, the predecessors of maps) are >> tuples and therefore it?s not uncommon to find code that uses the functions >> listed by Vance to work with them. For instance? >> >> find_user(UserName, Users) -> >> lists:keyfind(UserName, #user.username, Users). >> >> If we move from records to maps as our means to implement users, then we >> need something like? >> >> find_user(UserName, Users) -> >> case lists:dropwhile(fun(#{username := UN}) -> UN =/= UserName end, >> Users) of >> [] -> false; >> [User|_] -> User >> end. >> > > >> It would be *nice* if we don?t need to do that and instead can do? >> >> find_user(UserName, Users) -> >> lists:keyfind_for_maps(UserName, username, Users). >> > > Your example doesn't make sense to me. Why don't you simply use a map with > the username as key? > > Maps are a Key-Value stores. If you need secondary indices, you can always > maintain additional maps for them. Non-unique secondary keys can be > implemented by using a maps as value. Or use an ets where you can match on > additional fields. > Adding a linear search like lists:keyfind to maps make no sense in > my opinion. > > Talking about ets, adding a match like API to maps might make sense. If > only to make it possible to replace ets with maps without having to change > to much code. > > Regards > Andreas > >> > [...] > > -- > > Andreas Schultz > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Sep 23 14:18:30 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 23 Sep 2019 20:18:30 +0800 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: On Mon, Sep 23, 2019 at 7:45 PM Richard O'Keefe wrote: > Keep the function names the same but put them in another module, say 'maps:'. I would argue that as functions which operate on lists they belong in the lists module. -- -Vance From vances@REDACTED Mon Sep 23 14:36:44 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 23 Sep 2019 20:36:44 +0800 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: On Mon, Sep 23, 2019 at 7:07 PM Karl Velicka wrote: > I'm afraid I can't see the equivalence in your example Think small maps and big lists. > I'd be curious to know what your usecase for these functions is. As Brujo suggests many use cases for lists:keyfind/3 et. al. deal with lists of records. This is a very common pattern in our organization. As we use maps more we naturally want to have the simple efficient (BIF) implementation in the lists module. One example is with JSON where we have an object which contains an array of objects which we want to filter, sort, get, take, etc.. 1> #{id := Id, name := Name, characteristic := Chars} = zj:decode(Body), 1> length(Chars). 100 2> hd(Chars). #{id = 42, name = foo, type = bar, value = "hello"}. Sure I can always write simple funs to handle these but we have collected common patterns of operations on lists into the lists module and made it very fast. It seems quite obvious to add support for the new type of term which may be in those lists. -- -Vance From max.lapshin@REDACTED Mon Sep 23 16:23:33 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 23 Sep 2019 17:23:33 +0300 Subject: [erlang-questions] [ANN] snmp_collector In-Reply-To: References: Message-ID: Why do you need to write nif here? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Sep 23 16:29:59 2019 From: vances@REDACTED (Vance Shipley) Date: Mon, 23 Sep 2019 22:29:59 +0800 Subject: [erlang-questions] [ANN] snmp_collector In-Reply-To: References: Message-ID: We did reuse the snmp application's implementation at first but profiling showed us that 98% of the total time was spent doing the one million iterations required to generate a hash and was taking over 100ms each time. Moving to a NIF on the same hardware reduced it to less than 10ms. On Mon, Sep 23, 2019, 10:23 PM Max Lapshin wrote: > Why do you need to write nif here? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Sep 23 16:31:30 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 23 Sep 2019 17:31:30 +0300 Subject: [erlang-questions] [ANN] snmp_collector In-Reply-To: References: Message-ID: very interesting. I'm looking at your code and do not see what is the difference from standard crypto nif. Where is the difference? -------------- next part -------------- An HTML attachment was scrubbed... URL: From soverdor@REDACTED Tue Sep 24 00:16:25 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Mon, 23 Sep 2019 15:16:25 -0700 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: I looked in the debug info and I am seeing the ftp hang bug in all versions of OTP since 21.3. ERL-909. On Mon, Sep 23, 2019 at 1:48 AM Ingela Andin wrote: > > Hi! > > Please report bugs here: https://bugs.erlang.org/secure/Dashboard.jspa > > And please include more details about the what FTP server you are trying to connect too. Reproducible bugs are much easier to find and fix. > > Regards Ingela - Erlang/OTP team - Ericsson AB > > Den s?n 22 sep. 2019 kl 01:06 skrev Sam Overdorf : >> >> I just tried 22.1 today. >> My program uses the ftp client. >> It gets stuck and consumes all of my memory while trying to execute >> the following command. >> User = ftp:user(Pid, "ftp", ""), >> I have to control-c to stop it. >> At his point all of the memory comes back. >> I have been having this problem since "21,3". >> >> Thanks, >> Sam Overdorf >> soverdor@REDACTED >> >> >> >> >> On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: >> > >> > Cannot compile flussonic with 22.1 >> > >> > https://bugs.erlang.org/browse/ERL-1050 >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From soverdor@REDACTED Tue Sep 24 00:19:12 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Mon, 23 Sep 2019 15:19:12 -0700 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: I am connecting to "ftp.freebsd.org". Some times it works sometimes it does not work. OTP-21.2 seems to work just fine. Thanks, Sam soverdor@REDACTED On Mon, Sep 23, 2019 at 3:16 PM Sam Overdorf wrote: > > I looked in the debug info and I am seeing the ftp hang bug in all > versions of OTP since 21.3. ERL-909. > > > > > On Mon, Sep 23, 2019 at 1:48 AM Ingela Andin wrote: > > > > Hi! > > > > Please report bugs here: https://bugs.erlang.org/secure/Dashboard.jspa > > > > And please include more details about the what FTP server you are trying to connect too. Reproducible bugs are much easier to find and fix. > > > > Regards Ingela - Erlang/OTP team - Ericsson AB > > > > Den s?n 22 sep. 2019 kl 01:06 skrev Sam Overdorf : > >> > >> I just tried 22.1 today. > >> My program uses the ftp client. > >> It gets stuck and consumes all of my memory while trying to execute > >> the following command. > >> User = ftp:user(Pid, "ftp", ""), > >> I have to control-c to stop it. > >> At his point all of the memory comes back. > >> I have been having this problem since "21,3". > >> > >> Thanks, > >> Sam Overdorf > >> soverdor@REDACTED > >> > >> > >> > >> > >> On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: > >> > > >> > Cannot compile flussonic with 22.1 > >> > > >> > https://bugs.erlang.org/browse/ERL-1050 > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From soverdor@REDACTED Tue Sep 24 01:15:35 2019 From: soverdor@REDACTED (Sam Overdorf) Date: Mon, 23 Sep 2019 16:15:35 -0700 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> Message-ID: I forgot to mention that I am using Windows-7 64bit. Sam On Mon, Sep 23, 2019 at 3:19 PM Sam Overdorf wrote: > > I am connecting to "ftp.freebsd.org". > Some times it works sometimes it does not work. > OTP-21.2 seems to work just fine. > > Thanks, > Sam > soverdor@REDACTED > > > > > On Mon, Sep 23, 2019 at 3:16 PM Sam Overdorf wrote: > > > > I looked in the debug info and I am seeing the ftp hang bug in all > > versions of OTP since 21.3. ERL-909. > > > > > > > > > > On Mon, Sep 23, 2019 at 1:48 AM Ingela Andin wrote: > > > > > > Hi! > > > > > > Please report bugs here: https://bugs.erlang.org/secure/Dashboard.jspa > > > > > > And please include more details about the what FTP server you are trying to connect too. Reproducible bugs are much easier to find and fix. > > > > > > Regards Ingela - Erlang/OTP team - Ericsson AB > > > > > > Den s?n 22 sep. 2019 kl 01:06 skrev Sam Overdorf : > > >> > > >> I just tried 22.1 today. > > >> My program uses the ftp client. > > >> It gets stuck and consumes all of my memory while trying to execute > > >> the following command. > > >> User = ftp:user(Pid, "ftp", ""), > > >> I have to control-c to stop it. > > >> At his point all of the memory comes back. > > >> I have been having this problem since "21,3". > > >> > > >> Thanks, > > >> Sam Overdorf > > >> soverdor@REDACTED > > >> > > >> > > >> > > >> > > >> On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: > > >> > > > >> > Cannot compile flussonic with 22.1 > > >> > > > >> > https://bugs.erlang.org/browse/ERL-1050 > > >> > > > >> > _______________________________________________ > > >> > erlang-questions mailing list > > >> > erlang-questions@REDACTED > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > >> _______________________________________________ > > >> erlang-questions mailing list > > >> erlang-questions@REDACTED > > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions From raoknz@REDACTED Tue Sep 24 04:48:11 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Tue, 24 Sep 2019 14:48:11 +1200 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: I would argue that the focus of these functions is the maps, not the lists they happen to be inside. Perhaps the whole thing is inside out. Perhaps what's really needed is something that says - here is a list of maps and a key - give me a map of lists and a list If {M,R} = maps:transpose(Ms, K) then R is the filter of Ms by the condition "does not have key K" and M is a map from V to the filter of Ms by the condition "has V as value for key K". Only do the linear scan once. On Tue, 24 Sep 2019 at 00:18, Vance Shipley wrote: > > On Mon, Sep 23, 2019 at 7:45 PM Richard O'Keefe wrote: > > Keep the function names the same but put them in another module, say 'maps:'. > > I would argue that as functions which operate on lists they belong in > the lists module. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vasdeveloper@REDACTED Tue Sep 24 06:15:37 2019 From: vasdeveloper@REDACTED (Theepan) Date: Tue, 24 Sep 2019 09:45:37 +0530 Subject: [erlang-questions] Duplicate Entries in Mnesia In-Reply-To: References: Message-ID: Team, Duplicates are generated when fetched through the indexes. I think it is the index tables that have the duplicate entries. Do I need to do anything extra to ensure that the indexes are unique? Doesn't Mnesia take care of it? Best On Wed, Sep 18, 2019 at 3:38 AM Theepan wrote: > Hi There, > > Once in a while our Mnesia database tables (of default type : SET) find > duplicate records. The whole recod duplicates I mean, not just the key > duplicates. > > We are using dirty reads/writes very often to cater to speed requirements. > > Running on Erlang/OTP 18.0 > > Is thee are known issue regarding? > > Best, > Theepan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue Sep 24 07:32:55 2019 From: vances@REDACTED (Vance Shipley) Date: Tue, 24 Sep 2019 13:32:55 +0800 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe wrote: > I would argue that the focus of these functions is the maps, not the lists they happen to be inside. In my mind this is all about lists. Think small maps and big lists. That the terms are maps is a detail I have to deal with in processing my lists. I just want the lists module to be orthogonal. If we aren't going to move all the lists:key* functions to a 'records' module (please don't) let's simply add support for the new term type which "replaces" records. Your example is certainly all about maps but it doesn't help my use cases. > Perhaps the whole thing is inside out. Perhaps what's really needed > is something that says > - here is a list of maps and a key > - give me a map of lists and a list > If {M,R} = maps:transpose(Ms, K) > then R is the filter of Ms by the condition "does not have key K" > and M is a map from V to the filter of Ms by the condition "has V as > value for key K". > Only do the linear scan once. -- -Vance From hans.r.nilsson@REDACTED Tue Sep 24 09:48:03 2019 From: hans.r.nilsson@REDACTED (Hans Nilsson R) Date: Tue, 24 Sep 2019 07:48:03 +0000 Subject: [erlang-questions] Patch package OTP 22.1 released In-Reply-To: References: <7237de1eec3697dc07aeb59779372d9c07d91c28.camel@ericsson.com> , Message-ID: That hanging bug should have been fixed in OTP-22.1. Please file a bug report in https://bugs.erlang.org/secure/Dashboard.jspa so we can start to handle it. Or fix it yourself and do a Pull Request in github.com/otp/erlang ?? Regards,Hans Nilsson Erlang/OTP Team, Ericsson AB ________________________________ Fr?n: erlang-questions-bounces@REDACTED f?r Sam Overdorf Skickat: den 24 september 2019 00:16 Till: Ingela Andin Kopia: Erlang-Questions Questions ?mne: Re: [erlang-questions] Patch package OTP 22.1 released I looked in the debug info and I am seeing the ftp hang bug in all versions of OTP since 21.3. ERL-909. On Mon, Sep 23, 2019 at 1:48 AM Ingela Andin wrote: > > Hi! > > Please report bugs here: https://protect2.fireeye.com/url?k=c39c1e8e-9f15b114-c39c5e15-0cc47ad93e74-ecabdc8c7f3dfc9c&q=1&u=https%3A%2F%2Fbugs.erlang.org%2Fsecure%2FDashboard.jspa > > And please include more details about the what FTP server you are trying to connect too. Reproducible bugs are much easier to find and fix. > > Regards Ingela - Erlang/OTP team - Ericsson AB > > Den s?n 22 sep. 2019 kl 01:06 skrev Sam Overdorf : >> >> I just tried 22.1 today. >> My program uses the ftp client. >> It gets stuck and consumes all of my memory while trying to execute >> the following command. >> User = ftp:user(Pid, "ftp", ""), >> I have to control-c to stop it. >> At his point all of the memory comes back. >> I have been having this problem since "21,3". >> >> Thanks, >> Sam Overdorf >> soverdor@REDACTED >> >> >> >> >> On Sat, Sep 21, 2019 at 9:06 AM Max Lapshin wrote: >> > >> > Cannot compile flussonic with 22.1 >> > >> > https://protect2.fireeye.com/url?k=ecea006f-b063aff5-ecea40f4-0cc47ad93e74-c1ce7cb27875658a&q=1&u=https%3A%2F%2Fbugs.erlang.org%2Fbrowse%2FERL-1050 >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > https://protect2.fireeye.com/url?k=a7ef900b-fb663f91-a7efd090-0cc47ad93e74-280ffeb4f54ae6e3&q=1&u=http%3A%2F%2Ferlang.org%2Fmailman%2Flistinfo%2Ferlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> https://protect2.fireeye.com/url?k=ef7e7259-b3f7ddc3-ef7e32c2-0cc47ad93e74-8c929a7d0a01b6d7&q=1&u=http%3A%2F%2Ferlang.org%2Fmailman%2Flistinfo%2Ferlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > https://protect2.fireeye.com/url?k=f7cf6d98-ab46c202-f7cf2d03-0cc47ad93e74-d04f61fa4c5fc5c6&q=1&u=http%3A%2F%2Ferlang.org%2Fmailman%2Flistinfo%2Ferlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED https://protect2.fireeye.com/url?k=81c8be10-dd41118a-81c8fe8b-0cc47ad93e74-00ada7c14abbd53c&q=1&u=http%3A%2F%2Ferlang.org%2Fmailman%2Flistinfo%2Ferlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Tue Sep 24 13:26:43 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 24 Sep 2019 13:26:43 +0200 Subject: [erlang-questions] Logging level of "SSL: Socket error" In-Reply-To: References: Message-ID: Hi! This seem to be a misunderstanding we will change it back to info level. Regards Ingela Erlang/OTP team Den l?r 21 sep. 2019 kl 19:10 skrev Roger Lipscombe : > I thought I'd follow up on > http://erlang.org/pipermail/erlang-questions/2018-July/095948.html, > wherein (to summarise): > > Roger: the logging level of "SSL: Socket error" changed from info to > error; causes lots of noise in logs. See > > https://github.com/erlang/otp/commit/6e28a7909c665cc316d657dda02a2b8655ecc5da#diff-5e580136b1f3094b464f9870a474ee67L972 > > Ingela: so it did; might need reviewing. > > This is still the case in OTP-22.1; see > > https://github.com/erlang/otp/blob/OTP-22.1/lib/ssl/src/ssl_connection.erl#L1503 > > So, following up: any thoughts about downgrading (or making > configurable) the log level for this particular (class of) errors? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Tue Sep 24 14:10:55 2019 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Tue, 24 Sep 2019 22:10:55 +1000 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: References: Message-ID: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> If it's all about lists, you've got all you need: lists:search/2 or as previously mentioned lists:dropwhile. To me presence of lists:key... functions is a serious legacy which is justified only because there are tons of code using it. I'm glad we don't have all of the 'sofs' functions in 'lists' module. Of course it would be nice to have faster versions than handcrafted functions, but I'd like them to be introduced in a separate module, e.g. list_of_maps. On 24 September 2019 3:32:55 pm AEST, Vance Shipley wrote: >On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe >wrote: >> I would argue that the focus of these functions is the maps, not the >lists they happen to be inside. > >In my mind this is all about lists. Think small maps and big lists. >That the terms are maps is a detail I have to deal with in processing >my lists. I just want the lists module to be orthogonal. If we aren't >going to move all the lists:key* functions to a 'records' module >(please don't) let's simply add support for the new term type which >"replaces" records. > >Your example is certainly all about maps but it doesn't help my use >cases. > >> Perhaps the whole thing is inside out. Perhaps what's really needed >> is something that says >> - here is a list of maps and a key >> - give me a map of lists and a list >> If {M,R} = maps:transpose(Ms, K) >> then R is the filter of Ms by the condition "does not have key K" >> and M is a map from V to the filter of Ms by the condition "has V as >> value for key K". >> Only do the linear scan once. > > >-- > -Vance >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Kind regards, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From karolis.velicka@REDACTED Tue Sep 24 14:41:36 2019 From: karolis.velicka@REDACTED (Karl Velicka) Date: Tue, 24 Sep 2019 15:41:36 +0300 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> References: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> Message-ID: I agree with Dmitry - if the pattern is common enough then a new module for dealing with such structures seems like the best way forward. Invisible to those that don't need it but available and performant to those that do. Karl On Tue, 24 Sep 2019, 15:11 Dmitry Belyaev, wrote: > If it's all about lists, you've got all you need: lists:search/2 or as > previously mentioned lists:dropwhile. > > To me presence of lists:key... functions is a serious legacy which is > justified only because there are tons of code using it. I'm glad we don't > have all of the 'sofs' functions in 'lists' module. > Of course it would be nice to have faster versions than handcrafted > functions, but I'd like them to be introduced in a separate module, e.g. > list_of_maps. > > On 24 September 2019 3:32:55 pm AEST, Vance Shipley > wrote: >> >> On Tue, Sep 24, 2019 at 10:48 AM Richard O'Keefe wrote: >> >>> I would argue that the focus of these functions is the maps, not the lists they happen to be inside. >>> >> >> In my mind this is all about lists. Think small maps and big lists. >> That the terms are maps is a detail I have to deal with in processing >> my lists. I just want the lists module to be orthogonal. If we aren't >> going to move all the lists:key* functions to a 'records' module >> (please don't) let's simply add support for the new term type which >> "replaces" records. >> >> Your example is certainly all about maps but it doesn't help my use cases. >> >> Perhaps the whole thing is inside out. Perhaps what's really needed >>> is something that says >>> - here is a list of maps and a key >>> - give me a map of lists and a list >>> If {M,R} = maps:transpose(Ms, K) >>> then R is the filter of Ms by the condition "does not have key K" >>> and M is a map from V to the filter of Ms by the condition "has V as >>> value for key K". >>> Only do the linear scan once. >>> >> >> > -- > Kind regards, > Dmitry Belyaev > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue Sep 24 14:56:19 2019 From: vances@REDACTED (Vance Shipley) Date: Tue, 24 Sep 2019 20:56:19 +0800 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> References: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> Message-ID: On Tue, Sep 24, 2019, 8:11 PM Dmitry Belyaev wrote: > To me presence of lists:key... functions is a serious legacy which is > justified only because there are tons of code using it. > That is an argument I could accept, if not agree Of course it would be nice to have faster versions than handcrafted > functions, but I'd like them to be introduced in a separate module, e.g. > list_of_maps. > I would withdraw my suggestion at that point. Not because your approach is wrong, just because I don't want to make this a bigger issue than the simple question I raised. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Sep 24 16:03:34 2019 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 24 Sep 2019 15:03:34 +0100 Subject: [erlang-questions] Logging level of "SSL: Socket error" In-Reply-To: References: Message-ID: On Tue, 24 Sep 2019 at 12:26, Ingela Andin wrote: > This seem to be a misunderstanding we will change it back to info level. Thanks. From z@REDACTED Tue Sep 24 17:33:02 2019 From: z@REDACTED (Danil Zagoskin) Date: Tue, 24 Sep 2019 18:33:02 +0300 Subject: [erlang-questions] Sending file descriptors and sockets to another beam Message-ID: Hi! As you may know, OTP 22 has a new low-level socket API: http://erlang.org/doc/man/socket.html It allows to communicate over UNIX sockets, and UNIX sockets have sendmsg feature, which has an option to pass a file descriptor. I tried to pass the FD with a fresh OTP, and this didn't just work. After a bit of hacking I've made a patch for socket NIF and a demo. Patch: https://github.com/erlang/otp/pull/2400 Demo: https://gist.github.com/stolen/303c30d4edbb8835f9bec3fad0d75ede Example: in four different shells execute: - ./sendsock.escript proxy - ./sendsock.escript worker - telnet localhost 13456 - telnet localhost 13456 See the listen and established sockets are owned by different beam.smp processes: $ lsof -i:13456 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME beam.smp 82931 stolen 32u IPv4 0x3c2e6eba43bdf26f 0t0 TCP *:13456 (LISTEN) beam.smp 82952 stolen 32u IPv4 0x3c2e6eba47fa726f 0t0 TCP localhost:13456->localhost:62163 (ESTABLISHED) beam.smp 82952 stolen 33u IPv4 0x3c2e6eba41348e8f 0t0 TCP localhost:13456->localhost:62185 (ESTABLISHED) telnet 82976 stolen 5u IPv4 0x3c2e6eba4f19452f 0t0 TCP localhost:62163->localhost:13456 (ESTABLISHED) telnet 82993 stolen 5u IPv4 0x3c2e6eba4786052f 0t0 TCP localhost:62185->localhost:13456 (ESTABLISHED) What this may be useful for? * Hot upgrades with BEAM upgrade/restart * Load-balancing TCP load between unreliable workers without proxying * Single-connection distribution (see https://github.com/stolen/webdist) * Maybe anything else Have fun! -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Sep 24 17:44:09 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 24 Sep 2019 17:44:09 +0200 Subject: [erlang-questions] Sending file descriptors and sockets to another beam In-Reply-To: References: Message-ID: <0e29a570-2203-673f-dc80-ea7243aae568@ninenines.eu> That's pretty cool. Here's one very real use case using this. Evidently the same could be done with Erlang with this patch: https://blog.cloudflare.com/know-your-scm_rights/ On 24/09/2019 17:33, Danil Zagoskin wrote: > Hi! > > As you may know, OTP 22 has a new low-level socket API: > http://erlang.org/doc/man/socket.html > It allows to communicate over UNIX sockets, > and UNIX sockets have sendmsg feature, which has an option to pass a > file descriptor. > > I tried to pass the FD with a fresh OTP, and this didn't just work. > After a bit of hacking I've made a patch for socket NIF and a demo. > ? Patch: https://github.com/erlang/otp/pull/2400 > ? Demo: https://gist.github.com/stolen/303c30d4edbb8835f9bec3fad0d75ede > > Example: in four different shells execute: > > * |./sendsock.escript proxy| > * |./sendsock.escript worker| > * |telnet localhost 13456| > * |telnet localhost 13456| > > See the listen and established sockets are owned by different > |beam.smp|?processes: > > |$ lsof -i:13456 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME > beam.smp 82931 stolen 32u IPv4 0x3c2e6eba43bdf26f 0t0 TCP *:13456 > (LISTEN) beam.smp 82952 stolen 32u IPv4 0x3c2e6eba47fa726f 0t0 TCP > localhost:13456->localhost:62163 (ESTABLISHED) beam.smp 82952 stolen 33u > IPv4 0x3c2e6eba41348e8f 0t0 TCP localhost:13456->localhost:62185 > (ESTABLISHED) telnet 82976 stolen 5u IPv4 0x3c2e6eba4f19452f 0t0 TCP > localhost:62163->localhost:13456 (ESTABLISHED) telnet 82993 stolen 5u > IPv4 0x3c2e6eba4786052f 0t0 TCP localhost:62185->localhost:13456 > (ESTABLISHED)| > > > What this may be useful for? > ? * Hot upgrades with BEAM upgrade/restart > ? * Load-balancing TCP load between unreliable workers without proxying > ? * Single-connection distribution (see https://github.com/stolen/webdist) > ? * Maybe anything else > > Have fun! > > -- > Danil Zagoskin | z@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From t@REDACTED Tue Sep 24 17:56:28 2019 From: t@REDACTED (Tristan Sloughter) Date: Tue, 24 Sep 2019 09:56:28 -0600 Subject: [erlang-questions] =?utf-8?q?Sending_file_descriptors_and_sockets?= =?utf-8?q?_to_another=09beam?= In-Reply-To: References: Message-ID: Oh nice, maybe this is why I could never get the systemd socket activation to work! It relies on using the file descriptor 3 for the listen socket. On Tue, Sep 24, 2019, at 09:33, Danil Zagoskin wrote: > Hi! > > As you may know, OTP 22 has a new low-level socket API: http://erlang.org/doc/man/socket.html > It allows to communicate over UNIX sockets, > and UNIX sockets have sendmsg feature, which has an option to pass a file descriptor. > > I tried to pass the FD with a fresh OTP, and this didn't just work. > After a bit of hacking I've made a patch for socket NIF and a demo. > Patch: https://github.com/erlang/otp/pull/2400 > Demo: https://gist.github.com/stolen/303c30d4edbb8835f9bec3fad0d75ede > > Example: in four different shells execute: > * `./sendsock.escript proxy` > * `./sendsock.escript worker` > * `telnet localhost 13456` > * `telnet localhost 13456` > See the listen and established sockets are owned by different `beam.smp` processes: > `$ lsof -i:13456 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME beam.smp 82931 stolen 32u IPv4 0x3c2e6eba43bdf26f 0t0 TCP *:13456 (LISTEN) beam.smp 82952 stolen 32u IPv4 0x3c2e6eba47fa726f 0t0 TCP localhost:13456->localhost:62163 (ESTABLISHED) beam.smp 82952 stolen 33u IPv4 0x3c2e6eba41348e8f 0t0 TCP localhost:13456->localhost:62185 (ESTABLISHED) telnet 82976 stolen 5u IPv4 0x3c2e6eba4f19452f 0t0 TCP localhost:62163->localhost:13456 (ESTABLISHED) telnet 82993 stolen 5u IPv4 0x3c2e6eba4786052f 0t0 TCP localhost:62185->localhost:13456 (ESTABLISHED)` > > What this may be useful for? > * Hot upgrades with BEAM upgrade/restart > * Load-balancing TCP load between unreliable workers without proxying > * Single-connection distribution (see https://github.com/stolen/webdist) > * Maybe anything else > > Have fun! > > -- > Danil Zagoskin | z@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From toddg@REDACTED Tue Sep 24 21:03:16 2019 From: toddg@REDACTED (Todd) Date: Tue, 24 Sep 2019 19:03:16 +0000 Subject: [erlang-questions] Energy Analytics Project : Post 01 Message-ID: Hi everyone! First, thank you to everyone that responded to me both on-list and off-list about my 'Where is the Erlang Golden Path?' (see http://zwrob.com/posts/erlang-01/). It made my year to get a compliment from ROK :-). I'd like to follow up with what I've been working on all summer. I have a lot of thoughts about Erlang and such... but (spoiler) I'm not going to talk about any of that. Instead I'm going to invite you all to help me combat climate change with my new GPL3 project, the "Energy Analytics Project". Here's the first in a series of posts on it: http://zwrob.com/posts/eap01/ Here's the actual project (EAP): https://github.com/energy-analytics-project/energy-dashboard And here's my pitch... I emailed the Erlang list a few years ago and tried to get a thread going about how to deal with climate change. But let's face it. It's huge and demoralizing and just talking about this is depressing. Only a few people joined the conversation and it fizzled quickly. Fast forward to this spring. I quit my job and decided to learn Erlang. That was one of two goals. The other goal was to create a company focused on addressing climate change, and the first product was the EAP. BTW - that company exists as an LLC, but in reality, I haven't made a dime. I've been happily unemployed, holed up in my house, writing code for 10 hours a day all summer... at first working on Goal #1, learn Erlang. Along the way I discovered how much I *love* Erlang. I really do. But, I also realized it was the wrong language for this project. In fact, once I really learned Erlang, I realized that this was the wrong goal anyway. Goal #1 should have been, combat climate change...write the EAP. Timeline: * The first versions of EAP were bash scripts. * The next versions were in Erlang. I'll write about how cool this was soon. * The latest versions have all been in Python3 The problem I ran into is that my envisioned clients are data scientists not Erlang developers. I want them to be self sufficient, able to create and make changes to the pipelines etc. If everything was in Erlang, that would never happen and I would be stuck managing this project forever. Python, however, is the LinguaFranca of data science. And I know Python reasonably well. So I pivoted to Python3. Again, I have lots of observations from implementing the EAP in both languages, and I promise to write them up soon. In summary, even though my climate change email thread fizzled out a few years ago, now I have a project that has clear technical and domain needs. Here's what I'm looking for: * concept validation * are these useful data-sets? * is it useful to expose data as Sqlite3 databases? * notebooks * we need actual notebooks that answer real questions * we need real data scientists to help make solid Jupyter Notebook prototypes to base the notebooks to follow (I'm not a data scientist, I know only the most basic things about Pandas, and it's been a full-time effort getting all this put together... this is an area I definitely need help in) * data-sets * what are the most useful data sets that we currently have? * where can we source similar data from all the other Western States? * what additional data do we need in order to answer these regional questions? * domain knowledge * If you, or someone you know, is an energy sector domain expert, then please join (I'm also not an energy sector specialist) * documentation * tests At this point, I have 20 out of the 90+ data feeds published. I'm fully committed to getting the remaining 80 CAISO/OASIS feeds published and available. But I'm at 100% capacity just getting this project off the ground. All the stuff mentioned above, and anything I've forgotten about... well, that's why I need help. I also need to get back to paid work soon, so I won't be able to devote my full attention to this. ----------------------------------------------------------------------------- Anyone that would like to contribute to this project, please jump in: https://github.com/energy-analytics-project/energy-dashboard ----------------------------------------------------------------------------- Note: if any of you are in other countries and want to adopt this project for your own needs. Go for it. It's all GPL3. -Todd Todd Greenwood-Geer http://zwrob.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Sep 25 10:22:31 2019 From: zxq9@REDACTED (zxq9) Date: Wed, 25 Sep 2019 17:22:31 +0900 Subject: [erlang-questions] lists module functions for maps? In-Reply-To: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> References: <86868ADA-0788-48C0-8618-0FBD6CD02CB9@gmail.com> Message-ID: On 2019/09/24 21:10, Dmitry Belyaev wrote: > To me presence of lists:key... functions is a serious legacy which is > justified only because there are tons of code using it. Lists of records are used all over the place in new code as well and the lists:key*/N functions are the standard way of dealing with them. Some shops have simply forgotten that maps are NOT a replacement for records. The only strange thing about those functions is that they are in the lists module instead of some module written specifically for manipulating lists of records. Lists of maps fall into the same category. It is easy to see how lists of maps would manifest many places when dealing with data from external systems (JSON, XML, Python dicts, YAML, etc.). This probably doesn't belong in the lists module, but something available somewhere would reduce the noisiness of the code needed to be written for consumers of that kind of data. I was stuck at home today sick, so wrote this as a starting point/substitute/copypasta for anyone who might find it useful: https://gitlab.com/zxq9/lom/blob/master/src/lom.erl lom.erl exports [new/0, store/3, take/3, replace/3, delete/3, find/3, member/3, keymap/3, sort/2, merge/3] And as Vance noted earlier, is a very natural thing to pair with a JSON decoder. The typespec doesn't match Vance's proposal in every case but only because in some cases you must provide both the key's name and its value. -Craig From bchesneau@REDACTED Wed Sep 25 23:23:28 2019 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 25 Sep 2019 23:23:28 +0200 Subject: [erlang-questions] [ann] hackney 1.15.2 has been released Message-ID: Hi all, I just did a new minor release of hackney, version 1.15.2. Hackney is an HTTP client library for Erlang based applications. This release has some fixes and doc improvements. *Changes:* 1.15.2 - 2019-09-25 - doc: fix tes run example in readme - fix: hackney stream, send hackney_response before calling handle_error - fix: error remove ssl honor_cipher_order option - doc: document self-signed certificate usage - bump ssl_verify_fun to 1.1.5 - fix: don't use default pool if set to false - fix: hackney_headers_new:store/3 fix value appending to a list - fix: miscellaeous specs - doc: miscellaneous improvements Project repository: https://github.com/benoitc/hackney Hex package: https://hex.pm/packages/hackney Enjoy! Beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Thu Sep 26 00:14:15 2019 From: rvirding@REDACTED (Robert Virding) Date: Thu, 26 Sep 2019 00:14:15 +0200 Subject: [erlang-questions] maps:first In-Reply-To: References: <84721a64-2c7f-28e6-b453-2b34f73aa35f@t-online.de> <4b87f331-d2f5-cafd-13b8-e725b483d74e@t-online.de> Message-ID: I have a specific need for it. I need 2 functions working on maps which allow me to step over all the key/value pairs in a map without making an iterator. What I need is a maps:first(Map) and then a maps:next(Key, Map) which takes a map and a key and gives me the next key/value pair WITHOUT an iterator. All I have is the map and a key.and cannot keep an iterator. Yes, it could be done by generating an iterator each time and stepping down it but not efficiently. The order would be irrelevant just so long as I can be just to get all the keys as long as the map is not updated while stepping over it. If the map is updated then all bets are off. This is the only thing which stops me from using maps inside Luel to represent Lua tables and it is most annoying. Robert On Wed, 14 Aug 2019 at 09:52, Andreas Schultz < andreas.schultz@REDACTED> wrote: > Am Di., 13. Aug. 2019 um 20:18 Uhr schrieb Oliver Bollmann < > oliver.bollmann@REDACTED>: > >> Hi, >> >> i guess *we* do all the same, >> >> 1) processes with propery Y, i have about 1-10 million processes >> 2) gen_server to manage these processes in maps,dict,lists for example a >> pool with in,out,leave,delete >> 3) monitoring these processes >> >> Dilemma: >> >> If process P died i have to delete it from data structure using in 2) >> >> - maps, dict very fast; >> lists -> List -- [Item] slow! >> OK, i could use a sorted list and and the subtract would be faster, >> but insert is then slower! >> And subtract need an algorithm! >> >> - i do not want a special process, only one random or the first one from >> data structure using in 2) >> here is the place where i need *first_from_data_structure* >> lists very fast: first defines it itself -> [First|Rest] = List, >> but using maps or dict there is not first, at all! >> > > Are you sure a maps first is really what you want? Maps leys are not > ordered (beyond MAP_SMALL_MAP_LIMIT, which is 32), but they are also not > random. > > As long as your map has less elements than MAP_SMALL_MAP_LIMIT it will > behave like a ordered list, beyond that it will use a hash for ordering. > That means that for a given set of keys the ordering will be stable (but > not sorted). So when you take the first element of a map and put it back > again (based on key), it will be the new first element again. > > IMHO the only use case for you maps:first function is something that tries > to distribute new requests/elements across the elements of a map. Your > maps:first function coupled with the constant ordering of map keys would > lead to a distribution that is highly skewed towards a small number of keys > (or even just one key). > > Now, a maps:nth/2 function that works like list:nth/2 is another story. > You could use > > maps:nth(rand:uniform(maps:size(Map)), Map) > > to get a somewhat evenly distributed key/element from a map. > Implementing this function currently requires you to convert the map to a > list first. For large maps having a maps:nth/2 should yield better > performance. > > Regards > Andreas > > The question is: How can i get a random or first item from maps or dict? >> >> Hope that helps! >> >> Oliver >> >> >> On 13.08.19 15:41, Lukas Larsson wrote: >> >> >> >> On Tue, Aug 13, 2019 at 3:34 PM Oliver Bollmann < >> oliver.bollmann@REDACTED> wrote: >> >>> I often need maps:first(Map), is there plan for it? >>> >> >> No, there is currently no plan for it, as I cannot think of a use case >> when it would be useful, but that might just be poor imagination on my part. >> >> What do you use maps:first for? >> >> >>> My current implementation is: >>> >>> first(Map) -> >>> case maps:next(maps:iterator(Map)) of {Key,Val,_} -> {Key,Val}; >>> none -> none >>> end. >>> >>> Create the iterator always a new map? >>> >>> -- >>> Gr??e >>> Oliver Bollmann >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -- >> Gr??e >> Oliver Bollmann >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > > Andreas Schultz > > -- > > Principal Engineer > > t: +49 391 819099-224 > > ------------------------------- enabling your networks > ----------------------------- > > Travelping GmbH > > Roentgenstra?e 13 > > 39108 Magdeburg > > Germany > > t: +49 391 819099-0 > > f: +49 391 819099-299 > > e: info@REDACTED > > w: https://www.travelping.com/ > > Company registration: Amtsgericht Stendal Reg. No.: HRB 10578 > Geschaeftsfuehrer: Holger Winkelmann VAT ID: DE236673780 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Thu Sep 26 22:18:42 2019 From: zkessin@REDACTED (Zachary Kessin) Date: Thu, 26 Sep 2019 16:18:42 -0400 Subject: [erlang-questions] Adopting use Elvis Message-ID: HI Erlang Gang My company is in the process of adopting Elvis for our code base, so far we have it on a few small projects and are finding it useful. But I had two questions: 1. What rules have people found most or least useful 2. Are there any rules that people have developed that are not part of the main Elvis package that we might want to look at? Thanks Zach Zach Kessin ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Fri Sep 27 09:42:02 2019 From: marc@REDACTED (Marc Worrell) Date: Fri, 27 Sep 2019 09:42:02 +0200 Subject: [erlang-questions] Adopting use Elvis In-Reply-To: References: Message-ID: Hi Zach, In Zotonic we are using Elvis for simple checking. We tried to only catch blatant problems or style diversions. Most useful are: * Variable and function naming * Nesting level Our line length of 110 is a compromise. I am ok with longer lines, though others in our team like punch card line lengths. Our current config is here: https://github.com/zotonic/zotonic/blob/master/elvis.config Cheers, Marc > On 26 Sep 2019, at 22:18, Zachary Kessin wrote: > > HI Erlang Gang > > My company is in the process of adopting Elvis for our code base, so far we have it on a few small projects and are finding it useful. But I had two questions: > > What rules have people found most or least useful > Are there any rules that people have developed that are not part of the main Elvis package that we might want to look at? > > Thanks > Zach > > Zach Kessin > > ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Sep 27 10:49:29 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 27 Sep 2019 10:49:29 +0200 Subject: [erlang-questions] [ANN] Gun 2.0 pre-release 1 + Gun 1.3.1 Message-ID: <51e3e0c2-6f23-7fa0-c998-6e384ef5db50@ninenines.eu> Hello! Gun 2.0.0-pre.1 has been released! Gun is an HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP. First a heads up: while reviewing the release notes for the 2.0 pre-release I found that one of the bugs fixed could contribute to security vulnerabilities in certain systems (using Gun inside a proxy in specific environments) so I urge everyone to upgrade to at least Gun 1.3.1. See https://ninenines.eu/docs/en/gun/1.3/guide/migrating_from_1.3/ for details on this issue. If you can I would highly recommend using the 2.0 pre-release. It includes new features such as graceful shutdown of the connection, flow control, event handlers (with MANY events) and the HTTP/2 code and performance has greatly been improved. On the proxy side of things, Gun now supports tunneling TLS connections over TLS! Socks5 support has also been added and many improvements were made to the existing proxy support. Now Gun can connect using TCP/TLS through any combination and number of TCP/TLS HTTP/Socks5 proxies. Gun can also be used for arbitrary protocols. In this case Gun acts more like a socket. This can be used to make Gun tunnel data through proxies, or to use HTTP/1.1 Upgrade to speak to non-HTTP/Websocket protocols. The next pre-release will focus on adding HTTP/2 CONNECT support for tunneling through HTTP/2 proxies, and at the same time implement Websocket over HTTP/2 since that uses the same mechanism. See the announcement article and the migration guide for details! * https://ninenines.eu/articles/gun-2.0.0-pre.1/ * https://ninenines.eu/docs/en/gun/2.0/guide/migrating_from_1.3/ Cheers, -- Lo?c Hoguin https://ninenines.eu From elbrujohalcon@REDACTED Fri Sep 27 12:11:58 2019 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Fri, 27 Sep 2019 07:11:58 -0300 Subject: [erlang-questions] Adopting use Elvis In-Reply-To: References: Message-ID: I?m a bit biased here, as a maintainer of Elvis, but I can show you the current config we have for our public repos: This one is for? mero This one is for a much simpler library (spillway) Personally, my favorite rules are DRY and god_modules . That ones help me a lot with refactoring. But, I?m of course also very fond of this one :P Brujo Benavides > On 27 Sep 2019, at 04:42, Marc Worrell wrote: > > Hi Zach, > > In Zotonic we are using Elvis for simple checking. > > We tried to only catch blatant problems or style diversions. > Most useful are: > > * Variable and function naming > * Nesting level > > Our line length of 110 is a compromise. > I am ok with longer lines, though others in our team like punch card line lengths. > > Our current config is here: > > https://github.com/zotonic/zotonic/blob/master/elvis.config > > Cheers, > > Marc > > >> On 26 Sep 2019, at 22:18, Zachary Kessin > wrote: >> >> HI Erlang Gang >> >> My company is in the process of adopting Elvis for our code base, so far we have it on a few small projects and are finding it useful. But I had two questions: >> >> What rules have people found most or least useful >> Are there any rules that people have developed that are not part of the main Elvis package that we might want to look at? >> >> Thanks >> Zach >> >> Zach Kessin >> >> ? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Fri Sep 27 14:45:21 2019 From: otp@REDACTED (Erlang/OTP) Date: Fri, 27 Sep 2019 14:45:21 +0200 Subject: [erlang-questions] Patch Package OTP 22.1.1 Released Message-ID: <20190927124521.GA47323@erix.ericsson.se> Patch Package: OTP 22.1.1 Git Tag: OTP-22.1.1 Date: 2019-09-27 Trouble Report Id: OTP-16069, OTP-16092, OTP-16103, OTP-16107 Seq num: ERIERL-410, ERL-1049, ERL-1050 System: OTP Release: 22 Application: compiler-7.4.6, erts-10.5.1, snmp-5.4.1 Predecessor: OTP 22.1 Check out the git tag OTP-22.1.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.4.6 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.4.6 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16103 Application(s): compiler Related Id(s): ERL-1050 Fixed a bug in the bit-syntax optimization pass that could crash the compiler. Full runtime dependencies of compiler-7.4.6: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-10.5.1 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.5.1 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16069 Application(s): erts A terminating process sending distributed 'EXIT' or 'DOWN' signals while terminating could end up in a state where no progress at all was made. This was triggered by a distribution channel that the terminating process was sending on got busy. This bug has existed since ERTS version 10.4 (OTP 22.0). OTP-16107 Application(s): erts Related Id(s): ERL-1049 When communicating with a simultaneously exiting port via the erlang:port_*() BIFs one could sometimes get stray {Ref, What} messages. Where Ref was a reference and What usually were the atom badarg. Full runtime dependencies of erts-10.5.1: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- snmp-5.4.1 ------------------------------------------------------ --------------------------------------------------------------------- The snmp-5.4.1 application can be applied independently of other applications on a full OTP 22 installation. --- Improvements and New Features --- OTP-16092 Application(s): snmp Related Id(s): ERIERL-410 Made it possible to add 'extra socket options' to the (gen_udp) socket open call (for both manager and agent). A new option has been added, extra_sock_opts, which makes it possible for the user to add a list of extra socket options that will be appended to the other socket options for the open call. See the snmp application config man page (erl -man 6 snmp) or the "Configuring the application" chapter of the Users Guide for more info. Full runtime dependencies of snmp-5.4.1: crypto-3.3, erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From buday.gergely@REDACTED Sat Sep 28 15:45:12 2019 From: buday.gergely@REDACTED (buday.gergely@REDACTED) Date: Sat, 28 Sep 2019 13:45:12 +0000 (UTC) Subject: [erlang-questions] Experience with much memory Message-ID: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Hi there, are there experience reports on using Erlang/Elixir on systems with 200 GB of memory and more? If you have a first-hand account on that, please share it with me. My friends are doing that and said that only C++ was capable of handling that. - Gergely -------------- next part -------------- An HTML attachment was scrubbed... URL: From himars@REDACTED Sat Sep 28 16:45:58 2019 From: himars@REDACTED (Jack Tang) Date: Sat, 28 Sep 2019 22:45:58 +0800 Subject: [erlang-questions] Experience with much memory In-Reply-To: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> References: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Message-ID: It is complex question. You should locate who hold the memory first :) Below codes might help you to locate the process / ets ``` applications_memory() -> try application:which_applications() of Apps -> lists:map( fun({AppName, _, _}) -> Mem = application_memory(AppName), {AppName, Mem} end, Apps) catch _:Exception -> error_logger:error_msg("[~p] get application memory failed ~p", [?MODULE, Exception]), undefined end. application_memory(App) -> case application_controller:get_master(App) of Master when is_pid(Master) -> {Root, Name} = application_master:get_child(Master), {Memory, Children} = memory_info(Root), {Name, Memory div 1000000, Children}; _ -> undefined end. memory_info(Sup) -> Infos = supervisor:which_children(Sup), {M, E, ES} = lists:foldl( fun({Name, PId, Type, _}, {Total, Memories, MemorySets}) -> case Type of worker -> {memory, Memory} = process_info(PId, memory), NTotal = Total + Memory, case Name of undefined -> NMemorySets = case gb_sets:size(MemorySets) of Size when Size > 10 -> {_, NSets} = gb_sets:take_smallest(MemorySets), NSets; _ -> MemorySets end, NNMemorySets = gb_sets:insert( {undefined, Memory, PId}, NMemorySets), {NTotal, Memories, NNMemorySets}; Name -> {NTotal, [{Name, Memory, PId}|Memories], MemorySets} end; supervisor -> {Memory, Each} = memory_info(PId), NTotal = Total + Memory, {NTotal, [{Name, {Memory, Each}, PId}|Memories], MemorySets} end end, {0, [], gb_sets:new()}, Infos), NE = case E of [] -> gb_sets:to_list(ES); _ -> E end, NNE = lists:map( fun({N, Mem, PId}) -> case Mem of Mem when is_integer(Mem) -> {N, Mem div 1000000, undefined, PId}; {T, RestEach} -> {N, T div 1000000, PId, RestEach} end end, NE), NNNE = lists:reverse(lists:keysort(3, NNE)), {M, NNNE}. ets_memory() -> All = ets:all(), AllMems = lists:map( fun(Ets) -> Mem = ets:info(Ets, memory), {Ets, Mem} end, All), Mems = lists:sort( fun({_, M1}, {_, M2}) -> M1 > M2 end, AllMems), Sum = lists:sum(lists:map(fun({_, Mem}) -> Mem end, AllMems)), {Sum, Mems}. ``` Also I recommend https://github.com/ferd/recon/blob/master/src/recon_alloc.erl . On Sat, Sep 28, 2019 at 10:03 PM wrote: > Hi there, > > are there experience reports on using Erlang/Elixir on systems with 200 GB > of memory and more? > > If you have a first-hand account on that, please share it with me. > > My friends are doing that and said that only C++ was capable of handling > that. > > - Gergely > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jack Tang http://www.linkedin.com/in/jacktang -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Sat Sep 28 17:07:26 2019 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sat, 28 Sep 2019 17:07:26 +0200 Subject: [erlang-questions] Experience with much memory In-Reply-To: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> References: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Message-ID: We have been running Erlang/OTP on machines with up to 3 TB of RAM, for many years. The Beam VM has no problems with this; we have seen single Erlang processes within a Beam use a few hundred GB of temporary data and then deallocate it again, while the VMs can keep running for months on end. The problems at that scale typically show up in the OS kernel, and you may need to tune it to avoid long pauses in the OS memory management, and occasionally flush the page cache to defragment the RAM. /Richard Den l?r 28 sep. 2019 kl 16:03 skrev : > Hi there, > > are there experience reports on using Erlang/Elixir on systems with 200 GB > of memory and more? > > If you have a first-hand account on that, please share it with me. > > My friends are doing that and said that only C++ was capable of handling > that. > > - Gergely > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dszoboszlay@REDACTED Sat Sep 28 22:30:16 2019 From: dszoboszlay@REDACTED (=?UTF-8?Q?D=C3=A1niel_Szoboszlay?=) Date: Sat, 28 Sep 2019 22:30:16 +0200 Subject: [erlang-questions] Experience with much memory In-Reply-To: References: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Message-ID: I would like to add to Richard's points that you may have to tune the memory allocator settings of the Beam too above a few hundred GB-s. The default values may lead to some fragmentation that is becoming significant at this point, especially for long-term allocated memory (typically: ETS tables). I would advise not to use address order first fit strategies for example, but rather stick to the old default, best fit strategy. Using a super carrier may be beneficial too if you expect to use a lot of memory. Cheers, Daniel On Sat, 28 Sep 2019 at 17:07, Richard Carlsson wrote: > We have been running Erlang/OTP on machines with up to 3 TB of RAM, for > many years. The Beam VM has no problems with this; we have seen single > Erlang processes within a Beam use a few hundred GB of temporary data and > then deallocate it again, while the VMs can keep running for months on end. > The problems at that scale typically show up in the OS kernel, and you may > need to tune it to avoid long pauses in the OS memory management, and > occasionally flush the page cache to defragment the RAM. > > /Richard > > > Den l?r 28 sep. 2019 kl 16:03 skrev : > >> Hi there, >> >> are there experience reports on using Erlang/Elixir on systems with 200 >> GB of memory and more? >> >> If you have a first-hand account on that, please share it with me. >> >> My friends are doing that and said that only C++ was capable of handling >> that. >> >> - Gergely >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Sep 29 05:12:49 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 29 Sep 2019 06:12:49 +0300 Subject: [erlang-questions] Experience with much memory In-Reply-To: References: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Message-ID: Your friends are oversimplifying things of course. When you speak about 200GB or 3TB of RAM, you need to speak about things a bit deeper than "fast or not fast". Here is the beginning of long list of issues to discuss: 1) is your software NUMA aware? Many people experience strange performance degradations when they upgrade from 1CPU to 2CPU and double system memory. Numa unaware software starts migrating data between processors with serious penalty. Erlang is NUMA aware out of the box, but you can spoil things if you allow too much data migration. 2) how big is per-process memory? If you allocate 200GB of small integers into tuples and all of them in a single erlang process, then you will not be able even to do it in a reasonable amount of time, I suppose =) If you do things correctly and all your 200GB are in ets, binaries and spread across different processes, then it will be ok 3) how fast is memory allocation/deallocation done? If you just read 200GB file to memory and read data from it, then it is really easy. If you do complicated allocations/deallocations with complex memory connections, then your memory allocator should be fast enought to handle these allocations on 200GB of RAM. Erlang is fast enough for this. Program written on C++ will have lot of hidden bugs that are hard to find because valgrinding 200GB is a funny task. There are other questions about "is 200GB handling possible", these 3 are just what I could remember. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sun Sep 29 11:13:14 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 29 Sep 2019 11:13:14 +0200 Subject: [erlang-questions] Experience with much memory In-Reply-To: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> References: <9BB064E2598222A0.01535977-f863-40bf-aba9-ac3b702abba9@mail.outlook.com> Message-ID: On Sat, Sep 28, 2019 at 4:03 PM wrote: > > Hi there, > > are there experience reports on using Erlang/Elixir on systems with 200 GB of memory and more? > > If you have a first-hand account on that, please share it with me. The Erlang/OTP VM handles that and more without much trouble. We have been running Erlang on servers with multi-TB RAM for many years now. There are a few things to look out for: - Use a recent (say OTP-19 or above) version of the VM, older ones had integer truncation bugs that could lead to wrong results or segmentation faults when dealing with very large terms. - Disable transparent hugepages in the Linux kernel, that's been known to cause huge unpredictable pauses or even kernel crashes. (May or may now have been fixed now, but was certainly true in RHEL/CentOS 6.) - Page fragmentation is an issue, also causing occasional pauses when the Linux kernel goes busy. Watch buddyinfo, drop caches sometimes, and _maybe_ even compact memory. However, dropping caches hurts file system performance so don't do that unnecessarily. - Increase max_map_count. At TB sizes the VM can reach the default limit and the failures are _not_ easy to debug. From duncan.attard.01@REDACTED Mon Sep 30 12:46:17 2019 From: duncan.attard.01@REDACTED (Duncan Paul Attard) Date: Mon, 30 Sep 2019 12:46:17 +0200 Subject: [erlang-questions] Suspending Erlang Processes Message-ID: <5390547C-8ADF-4370-B702-0CF75BD1A91F@um.edu.mt> I am tracing an Erlang process, say, `P` by invoking the BIF `erlang:trace(Pid_P, true, [set_on_spawn, procs, send, 'receive'])` from some process. As per the Erlang docs, the latter process becomes the tracer for `P`, which I shall call `Trc_Q`. Suppose now, that process `P` spawns a new process `Q`. Since the flag `set_on_spawn` was specified in the call to `erlang:trace/3` above, `Q` will automatically be traced by `Trc_P` as well. --- I want to spawn a **new** tracer, `Trc_Q`, and transfer the ownership of tracing `Q` to it, so that the resulting configuration will be that of process `P` being traced by tracer `Trc_P`, `Q` by `Trc_Q`. However, Erlang permits **at most** one tracer per process, so I cannot achieve said configuration by invoking `erlang:trace(Pid_Q, true, ..)` from `Trc_Q`. The only way possible is to do it in two steps: 1. Tracer `Trc_Q` calls `erlang:trace(Pid_Q, false, ..)` to stop `Trc_P` from tracing `Q`; 2. `Trc_Q` calls `erlang:trace(Pid_Q, true, ..)` again to start tracing `Q`. In the time span between steps **1.** and **2.** above, it might be possible that trace events by process `Q` are **lost** because at that moment, there is no tracer attached. One way of mitigating this is to perform the following: 1. Suspend process `Q` by calling `erlang:suspend_process(Pid_Q)` from `Trc_Q` (note that as per Erlang docs, `Trc_Q` remains blocked until `Q` is eventually suspended by the VM); 2. `Trc_Q` calls `erlang:trace(Pid_Q, false, ..)` to stop `Trc_P` from tracing `Q`; 3. `Trc_Q` calls `erlang:trace(Pid_Q, true, ..)` again to start tracing `Q`; 4. Finally, `Trc_Q` calls `erlang:resume_process(Pid_Q)` so that `Q` can continue executing. From what I was able to find out, while `Q` is suspended, messages sent to it are queued, and when resumed, `Trc_Q` receives the `{trace, Pid_Q, receive, Msg}` trace events accordingly without any loss. I have one constraint which led me to look at suspend/resume process: I cannot modify the code of `P` or `Q`, so inserting `receive` expressions to block said processes is out of the question. However, I am hesitant to use suspend/resume, since the Erlang docs explicitly say that these are to be used for *debugging purposes only*. Any idea as to why this is the case? Thanks. Duncan.