From zxq9@REDACTED Fri Nov 1 08:31:45 2019 From: zxq9@REDACTED (zxq9) Date: Fri, 1 Nov 2019 16:31:45 +0900 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <63F2C658-515F-416E-822A-5445EFFB46DC@micic.co.za> References: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> <63F2C658-515F-416E-822A-5445EFFB46DC@micic.co.za> Message-ID: <664b5b33-1349-7865-2871-45587caada71@zxq9.com> On 2019/11/01 6:03, Valentin Micic wrote: >> On 31 Oct 2019, at 21:43, Richard O'Keefe wrote: >> >> You can represent all sorts of information as binaries, >> but the binary doesn't know. >> Your intentions are not part of the binary. ... >> The best we can hope for in a "global" term ordering is >> that it satisfy the axioms of a total order. That's a useful >> thing to have. >> >> Want something else? Program it. > Yes, indeed, and I?ve implied that much ? I do want to program it, but I also wanted to know what people think ? if one aspires to program for others, it seems reasonable to consider dominant logic on the issue. > > I started by thinking that less cannot be more, e.g. two octets cannot be less than three. > > Now I am increasingly convinced that one should never compare Binaries that are not of the same size. > However, I am not so sure what to do if, or rather when, one has to (compare binaries of unequal sizes). Not so fast. You MUST be able to compare things in *some* way or else you cannot establish a global order, and without that an entire universe of useful things is closed to you. Binaries are a primitive data type. Strings can NEVER be a primitive data type. With this in mind, you could useful things with custom types such as "binary string", "list string", "mixed string"... -type bstring :: {bstring, binary()}. -type lstring :: {lstring, string()}. -type mstring :: {mstring, iolist()}. Your types could also (whoa!) include meta that indicates the subtype of the string data -- not merely whether it encoded as UTF8, but what the collation rule is intended to be, whether the encoding is canonical and if so which form, etc. (In my region this is all quite a big deal...) -type bstring :: {bstring, encoding(), collation(), lang(), binary()}. Now you could REALLY do some useful sorting! But not on binaries. They are primitives. The fact we have a universal order across ALL primitives is great and something that goes unappreciated until you run into a problem where it is critical to have such an order and you lack it. This discussion stems from a failure to distinguish between a complex type (strings) and a language primitive. In most modern OOP languages strings are objects (whether immutable or not) and carry a fair amount of meta with them. When you rip the arms and legs off an object you are essentially left with a tuple that carries the core data itself along with useful descriptive meta. That's what is needed to resolve the string sorting problem: the string itself plus meta to describe its nature. There is nothing even remotely weird about the way Erlang sorting order works. That's a platform issue in Erlang. If we want arbitrary collation sorts over a true (complex) string type, we're going to need to write a library to get it. -Craig From raimo.niskanen@REDACTED Fri Nov 1 09:40:44 2019 From: raimo.niskanen@REDACTED (Raimo Niskanen) Date: Fri, 1 Nov 2019 08:40:44 +0000 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <9637BA32-FC3E-4517-93AE-AF8270C0496F@lelf.lu> References: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> <9637BA32-FC3E-4517-93AE-AF8270C0496F@lelf.lu> Message-ID: <332c933a2289ed0ffff5fccace396a926187c856.camel@ericsson.com> On Thu, 2019-10-31 at 20:20 +0400, Antonio Nikishaev wrote: > > On 31 Oct 2019, at 14:52, Raimo Niskanen < > > raimo.niskanen@REDACTED> wrote: > > > > > > As for now, Elixir uses binaries as their default representation > > for > > strings, so having them sort as strings is vital for Elixir. > > (Unicode has shot a big hole in that argument, but still true for > > latin-1) > > It has not. UTF-8 does preserve order. Ah, yes! That clever property of UTF-8 slipped my mind. So in both Erlang and Elixir; codepoint lists sorts the same way as their corresponding UTF-8 binaries (which is the default format for binaris as strings in both Erlang and Elixir). Not true for other Unicode binary encodings, though... / Raimo > > > > > -- lelf > From raoknz@REDACTED Fri Nov 1 12:08:24 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Sat, 2 Nov 2019 00:08:24 +1300 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <332c933a2289ed0ffff5fccace396a926187c856.camel@ericsson.com> References: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> <9637BA32-FC3E-4517-93AE-AF8270C0496F@lelf.lu> <332c933a2289ed0ffff5fccace396a926187c856.camel@ericsson.com> Message-ID: I note that not only do dictionaries and phone books use different collation orders, so (fairly obviously) do different languages, and the rules can be insanely complex. English and French, for example, both require up to four passes over a pair of strings to resolve the order, and French runs one of those passes backwards. (According to the draft standard I read about 10 years ago.) Things get really complicated in a bilingual country like Canada or New Zealand, where you have to process text in a mix of two languages. (I'm not talking about scholarly articles, but about everyday newspaper articles.) It gets worse, but enough already. The bottom line is that there is NO "right" way to compare binaries and there is NO "wrong" way either, as long as the axioms of a total order are respected. The way Erlang does it is one good way. "Compute the base-255 digits of pi to as many places as necessary, exclusive-or each binary with the appropriate digits, and compare the results using the existing algorithm" is another perfectly legal way. There are basically four requirements for Erlang-style term ordering: (1) the axioms of a total order must be satisfied (2) the implementation must be fast enough that programmers are happy to use term comparison without wondering whether they should program something (3) the rules must be spelled out so that programmers know what they are getting (4) if the language uses the same operators for term comparison and arithmetic comparison, the two must be compatible; if (like Prolog) term comparison (@<) and arithmetic comparison (<) can disagree. Just think of the fun you can have trying to figure out what to do with NaNs. There is another desideratum, which isn't quite as important: (+) the "native" representation for text should be ordered in a way that is unsurprising for programmers but MUST NOT be identical to locale collation order for two reasons: (A) term comparison must distinguish things that locale collation might identify (B) term comparison is for computers and is not meant to be locale-sensitive, while collation for users should be locale-sensitive. It might be worth comparing three different languages I use: Erlang: term comparison is a total order Smalltalk: x = y must return true or false consistently for ANY x and y but x < y need not work if x and y are of different kinds and for a given x (like nil) need not ever work for any y (even y = x). Haskell: comparison is supposed to be total (thanks,IEEE) for any type that supports it; whether a type supports comparison is detected at compile-time; how the comparison is done can be different for every type. On Fri, 1 Nov 2019 at 21:40, Raimo Niskanen wrote: > > On Thu, 2019-10-31 at 20:20 +0400, Antonio Nikishaev wrote: > > > On 31 Oct 2019, at 14:52, Raimo Niskanen < > > > raimo.niskanen@REDACTED> wrote: > > > > > > > > > As for now, Elixir uses binaries as their default representation > > > for > > > strings, so having them sort as strings is vital for Elixir. > > > (Unicode has shot a big hole in that argument, but still true for > > > latin-1) > > > > It has not. UTF-8 does preserve order. > > Ah, yes! That clever property of UTF-8 slipped my mind. > > So in both Erlang and Elixir; codepoint lists sorts the same way as > their corresponding UTF-8 binaries (which is the default format for > binaris as strings in both Erlang and Elixir). > > Not true for other Unicode binary encodings, though... > > / Raimo > > > > > > > > > > > > > -- lelf > > From raoknz@REDACTED Fri Nov 1 12:36:31 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Sat, 2 Nov 2019 00:36:31 +1300 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <46D4876E-C091-4A16-9A2B-D91DA65986AE@micic.co.za> References: <46D4876E-C091-4A16-9A2B-D91DA65986AE@micic.co.za> Message-ID: Historical note: back in, oh, 1978 or 1979, I came across a PL/I dialect which compared strings X Y thus: if X is longer than Y, X > Y if X is shorter than Y, X < Y otherwise, compare the characters. It was rather disconcerting to find that 'KETTLE' > 'CAT'. However, it was also disconcerting to realise that the normal PL/I rules are equally quirky: "If the strings are not the same length, pad the shorter on the right with blanks, and then compare." This means, for example, that you can find strings X Y such that X > X||Y (where || is the string concatenation operator in PL/I). Fortran does the same as PL/I here: first pad with blanks to the same length, then compare. And it has the same rather confusing result. It turned out that the dialect I mentioned at the start had changed the rules to get something easier to reason about. Of course, lexicographic order would have been easy to reason about too... I built a string library once that stored strings in big-endian order even on little-endian machines, and word-aligned them, so that you could compare 4 (or 8) bytes at a time. It turned out not to be as much faster as I had hoped. On Sat, 26 Oct 2019 at 03:37, Valentin Micic wrote: > > Hi all, > > Consider the following inequalities: > > (tsdb_1_1@REDACTED)2876> <<0,0,1>> < <<1,0,0>>. > true > > As well as: > > (tsdb_1_1@REDACTED)2878> [0,0,1] < [1,0,0]. > true > > The result (true) makes sense ? in both cases operands are of the same size, and the first (leftmost) element value of the left operand is less than first (again, leftmost) element value of the right operand. > > However: > > (tsdb_1_1@REDACTED)2877> <<0,0,1>> < <<1,0>>. > true > > and > > (tsdb_1_1@REDACTED)2879> [0,0,1] < [1,0]. > true > > This indicates that the actual length of the operands are not considered, for clearly, three octets cannot be less then two, right? > > This becomes even more confusing if you check how tuples are compared given the similar test-case: > > (tsdb_1_1@REDACTED)2886> {0,0,1} < {1,0,0}. > true > (tsdb_1_1@REDACTED)2887> {0,0,1} < {1,0}. > false > > Here, the number of elements in the tuple are clearly taken into the consideration. > > Then, if one converts all three pairs of operands to its (external) binary representation, e.g. > > Binary: > > (tsdb_1_1@REDACTED)2916> term_to_binary( <<0,0,1>> ). > <<131,109,0,0,0,3,0,0,1>> > (tsdb_1_1@REDACTED)2917> term_to_binary( <<1,0>> ). > <<131,109,0,0,0,2,1,0>> > > List: > > tsdb_1_1@REDACTED)2918> term_to_binary( [0,0,1] ). > <<131,107,0,3,0,0,1>> > (tsdb_1_1@REDACTED)2919> term_to_binary( [1,0] ). > <<131,107,0,2,1,0>> > > Tuple: > (tsdb_1_1@REDACTED)2920> term_to_binary( {0,0,1} ). > <<131,104,3,97,0,97,0,97,1>> > (tsdb_1_1@REDACTED)2921> term_to_binary( {1,0} ). > <<131,104,2,97,1,97,0>> > > One could see that the number of ?elements? are known and available for comparison (I?ve highlighted this number using bold, yellow font) for all three situations. > > And yet, the different approaches between binary and lists on one side, and tuples, on another, appear to be deliberate as binary and list types are following the approach used by C standard library memcmp function, whereas tuples do not, e.g. > > int memcmp(const void *s1, const void *s2, size_t n); > > In other words, operands are reduced to some common size ?n', and then compared as if they were equal in length. > > I do understand this restriction in C -- if not for ?n?, the argument sizes are not known, and without it the execution may end up in a catastrophic failure (e.g. segment violation, hmm.. which can still happen anyway if ?n? is inadequate). > > So, THE QUESTION: > > Why would it be wrong to consider inequalities for binary() data types the same way we do it for tuples ? number of ?elements? first and ?numeric values? of individual elements ? second? > > In my view, this is not only not wrong, but it would be more logical, and this is why. > > Look at the following comparison: > > (tsdb_1_1@REDACTED)2957> <<0,0,1>> < <<0,0,1,0>>. > true > > > Well, this make sense, for less-is-less (e.g. three octets are less than four octets) and even if one considers <<0,0,1>> as a 24-bit integer value, and <<0,0,1,0>> as a 32-bit integer, one may ?normalise? the 24-bit integer to its 32-bit equivalent by adding a leading zero, and the comparison expression would still return true: > > (tsdb_1_1@REDACTED)2961> <<0,0,0,1>> < <<0,0,1,0>>. > true > > Therefore, as expected, 1 < 256, and, thus, we may be forgiven if we have the same expectation when we write: > > (tsdb_1_1@REDACTED)2958> <<1>> < <<0,0,1,0>>. > > Right? Well, not so, because: > > (tsdb_1_1@REDACTED)2958> <<1>> < <<0,0,1,0>>. > false > > Thus, "less is more", and therefore 1 appears to be greater-than-or-equal-to 256. Somehow. > > This could never happen if one considered a number of elements (in this case number of octets) before attempting to compare the individual element values. > > But wait, the "less-is-more" may easily become its opposite ? ?more is less?, because: > > (tsdb_1_1@REDACTED)2973> <<32, 97>> < <<97>>. > true > > Yes, this approach may help us to sort a list of, say, surnames alphabetically, so we can have all the ?Ms? before the ?Ns?, regardless of how long the surname is, but is this actually logical for two arbitrary binaries? Seeing that we actually care about number of elements when we deal with tuples, why not extend the same approach to arbitrary binary values? > > Why do we presume that binaries are given as STRINGS when we compare them? When it comes to lists, the situation is even worse, because lists do, but then do not follow the STRING comparison semantic: > > (tsdb_1_1@REDACTED)2991> [32, 97] < [97]. > true > > *** But! *** > > (tsdb_1_1@REDACTED)2992> [{32, 97}] < [97]. > false > > (tsdb_1_1@REDACTED)2993> [{32, 97}] < [{97}]. > false > > And just to be clear, I do not expect Erlang to change. Nor do I care about how lists are handled (well, at least not for the moment). > > My main concern is semantic behind comparing two arbitrary binary() operands. For one of my projects I?d like to employ tuple-like semantic (.e.g. number of octets first, values later) when comparing two arbitrary binaries and thus avoid ?less-is-more? and ?more-is-less? paradoxes. > > Is there any reason why I shouldn?t? (Okay, it may be a bit slower, but one may argue this approach may be even a bit faster for a long-enough binaries. But what else?) > > At any rate, I would appreciate your reasoning on the topic. What resonates better with you? > > > Thanks in advance. > > V/ > > From joe@REDACTED Fri Nov 1 16:25:11 2019 From: joe@REDACTED (Joe Harrison) Date: Fri, 1 Nov 2019 15:25:11 +0000 Subject: Nobody is unsubscribed In-Reply-To: References: Message-ID: Thanks for doing all of this, regardless. There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant way that doesn't break some client's "From:" field, subject line, or "Reply:" button in some way, but this seems like the least bad option. I hope my emails make it through to the list now ^_^ OT: Be careful of organisations' web contact forms which ask for your email address. Sometimes their web servers generate an email from the form using your email address as the "From:" address, which will break a lot of DKIM/DMARC/SPF stuff. I know of at least one local authority (council) website in the UK which is guilty of this. - Joe On 26/10/2019 07:57, Raimo Niskanen wrote: > It is mainly "the big ones" that have been affected by stricter DMARC > policies. > > When a subscriber sending from e.g Yahoo gets received by Gmail then > Gmail rejects that message since Yahoo's DMARC policy says so (also vice > versa). So the list gets a bounce and eventually blocks the Gmail > subscriber, if enough in a row happens to send with strict DMARC policies. > > So for some it has worked, some gets an annoying list probe every now > and then, some do not get many posts, but the final nail in the coffin > was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > policy and at the same time told us to get our act together and stop > sending "unhygienic e-mail". > > All the best > / Raimo > > > Den fre 25 okt. 2019 16:58Chris Rempel > skrev: > > Not having the subject contain [erlang-questions] or some other > obvious indicator is quite unfortunate.? I guess many people were > affected by not being DMARC compliant?? It seems to have been > working just fine for quite some time... ie it "works for me" as it was. > ? > That said, thanks for maintaining the list, and keeping it going.? > It is a most useful resource. > ? > Chris > ? > *Sent:*?Friday, October 25, 2019 at 7:38 AM > *From:*?"Raimo Niskanen" > > *To:*?erlang-questions@REDACTED > *Subject:*?Re: Nobody is unsubscribed > To achieve DMARC compliance we have stopped changing the Subject: > field and no longer add the mailing list footer to the messages. > > This is because From: Subject: and mail body among other fields are > often DKIM signed, so if we should change them we would not pass DKIM > signature check and thereby not be DMARC compliant. > > Sorry for the inconvenience, we do not make the rules... > / Raimo Niskanen > > On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen > wrote: > > > > The reason we changed mailing list servers was to get better DMARC and > > DKIM compliance. This is a test post for us to inspect its headers... > > -- > > Raimo Niskanen > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From Andras.Bekes@REDACTED Fri Nov 1 19:22:18 2019 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Fri, 1 Nov 2019 18:22:18 +0000 Subject: [erlang-questions] Segfault with Erlang R22 In-Reply-To: References: <649d87fb8e0346f7ad0a039e483a46af@morganstanley.com> <50f54b6f7f9b4e88ae3d1e540c881ba2@morganstanley.com> Message-ID: Program terminated with signal 11, Segmentation fault. #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 5252 c_p->seq_trace_lastcnt = unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.212.el6_10.3.x86_64 (gdb) bt #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at beam/erl_process.c:8465 #2 0x000000000069262a in thr_wrapper (vtwd=) at pthread/ethread.c:118 #3 0x00002b81f80f7dd5 in _L_unlock_48 () from /lib64/libpthread.so.0 #4 0x00002b81f80f5eb3 in __find_thread_by_id () from /lib64/libpthread.so.0 #5 0x0000000000000000 in ?? () (gdb) I am not sure how to " print the c_p parameter via its raw value (print *(Process*)0x.....)". Where should I take the value 0x..... from? -----Original Message----- From: Mikael Pettersson [mailto:mikpelinux@REDACTED] Sent: Thursday, October 24, 2019 7:10 PM To: Bekes, Andras G (IST) Cc: Erlang Questions Subject: Re: [erlang-questions] Segfault with Erlang R22 On Thu, Oct 24, 2019 at 4:57 PM Bekes, Andras G wrote: > > Hi Mikael, > > I filed a bug report in the bug tracker: https://bugs.erlang.org/browse/ERL-1074 > > Unfortunately printing *c_p did not reveal anything: > > (gdb) print c_p > $1 = > (gdb) print *c_p > value has been optimized out > > What should be the next step? > I can reliably produce 5-10 core dumps per week in my test system. I'd try to get a backtrace (bt command in gdb) from the crashed thread, then maybe print the c_p parameter via its raw value (print *(Process*)0x.....) if gdb insists that the value is optimized out. /Mikael > > -----Original Message----- > From: Mikael Pettersson [mailto:mikpelinux@REDACTED] > Sent: Friday, October 18, 2019 8:03 PM > To: Bekes, Andras G (IST) > Cc: Erlang Questions > Subject: Re: [erlang-questions] Segfault with Erlang R22 > > On Fri, Oct 18, 2019 at 4:34 PM Bekes, Andras G > wrote: > > > > Hi All, > > > > > > > > After upgrading to Erlang R22, my software crashes the Erlang VM with Segmentation fault. > > > > It happens rarely, only after several days of test workload, so I can?t really reproduce. > > > > > > > > I made more than 10 core dumps so far, loaded them into gdb, and all of them died at these 2 crash points: > > > > > > > > Program terminated with signal 11, Segmentation fault. > > > > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2ade29280590) at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:4064 > > > > 4064 if (is_not_tuple(r(0))) { > > > > > > > > or > > > > > > > > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > > > > 5252 c_p->seq_trace_lastcnt = unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); > > > > > > > > The software is not doing any tracing when the crash happens, nor does it have any NIFs. > > I think you should open a bug report in the erlang bug tracker. > > The second crash site above is in the remove_message() function, in a > block where ERL_MESSAGE_TOKEN(msgp) > is neither NIL nor am_undefined, but SEQ_TRACE_TOKEN(c_p) is invalid > (not the expected 5-tuple). > > Maybe printing *c_p in gdb when that happens could shed some light. > > /Mikael > From curtis@REDACTED Sat Nov 2 01:12:57 2019 From: curtis@REDACTED (Curtis J Schofield) Date: Sat, 02 Nov 2019 00:12:57 +0000 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> Message-ID: <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Hi! Just curious if there is an update on out of order certs. The example has id0, id1, id2, id3 certs with id1 being the natural root of id2 who is the root of id3, who is the root of id0. We can correct the out of order problem by including id1,id2,id3 certs in our chain. It would be nice to hear from the erlang maintainers around what kind of "out of order" erlang can handle nicely and if there is planned support for our case! Thank you again, Curtis. Sent through [ProtonMail](https://protonmail.com) Encrypted Email Channel. ??????? Original Message ??????? On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield wrote: > Hi! Thank you. > > I included the root cert in the example. The root cert is id1 in cert chain - this is evident in the other file. > > It seems because the root cert is out of order - the cert chain is invalid - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 > > Thank you for your consideration! > > Sent from ProtonMail Mobile > > On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin wrote: > >> Hi! >> >> "Unknown CA" means that you did not have the ROOT certificate of the chian in your "trusted store" (cacerts option). >> If you do not own the ROOT certificate you can not trust the chain. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >> >>> Dear Erlang Questions: >>> >>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>> >>> In SSL 9.2 we have a root CA and an out of order cert chain >>> for host hooks.glip.com. >>> >>> When we try to verify peer with the out of order cert >>> chain we get 'Unknown CA'. >>> >>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>> >>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>> mention that other care may need to be taken to ensure compatibility. >>> >>> Reproduce error: >>> >>> https://github.com/robotarmy/out-of-order-ssl >>> >>> Thank you, >>> Curtis and Team DevEco >>> >>> Sent through ProtonMail Encrypted Email Channel. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Sat Nov 2 02:47:31 2019 From: raoknz@REDACTED (Richard O'Keefe) Date: Sat, 2 Nov 2019 14:47:31 +1300 Subject: Nobody is unsubscribed In-Reply-To: References: Message-ID: Does this apply to the EEPS list as well? On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > > Thanks for doing all of this, regardless. > > There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > way that doesn't break some client's "From:" field, subject line, or > "Reply:" button in some way, but this seems like the least bad option. > > I hope my emails make it through to the list now ^_^ > > OT: Be careful of organisations' web contact forms which ask for your > email address. Sometimes their web servers generate an email from the > form using your email address as the "From:" address, which will break a > lot of DKIM/DMARC/SPF stuff. > I know of at least one local authority (council) website in the UK which > is guilty of this. > > - Joe > > On 26/10/2019 07:57, Raimo Niskanen wrote: > > It is mainly "the big ones" that have been affected by stricter DMARC > > policies. > > > > When a subscriber sending from e.g Yahoo gets received by Gmail then > > Gmail rejects that message since Yahoo's DMARC policy says so (also vice > > versa). So the list gets a bounce and eventually blocks the Gmail > > subscriber, if enough in a row happens to send with strict DMARC policies. > > > > So for some it has worked, some gets an annoying list probe every now > > and then, some do not get many posts, but the final nail in the coffin > > was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > > policy and at the same time told us to get our act together and stop > > sending "unhygienic e-mail". > > > > All the best > > / Raimo > > > > > > Den fre 25 okt. 2019 16:58Chris Rempel > > skrev: > > > > Not having the subject contain [erlang-questions] or some other > > obvious indicator is quite unfortunate. I guess many people were > > affected by not being DMARC compliant? It seems to have been > > working just fine for quite some time... ie it "works for me" as it was. > > > > That said, thanks for maintaining the list, and keeping it going. > > It is a most useful resource. > > > > Chris > > > > *Sent:* Friday, October 25, 2019 at 7:38 AM > > *From:* "Raimo Niskanen" > > > > *To:* erlang-questions@REDACTED > > *Subject:* Re: Nobody is unsubscribed > > To achieve DMARC compliance we have stopped changing the Subject: > > field and no longer add the mailing list footer to the messages. > > > > This is because From: Subject: and mail body among other fields are > > often DKIM signed, so if we should change them we would not pass DKIM > > signature check and thereby not be DMARC compliant. > > > > Sorry for the inconvenience, we do not make the rules... > > / Raimo Niskanen > > > > On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen > > wrote: > > > > > > The reason we changed mailing list servers was to get better DMARC and > > > DKIM compliance. This is a test post for us to inspect its headers... > > > -- > > > Raimo Niskanen > > > From ratmapper@REDACTED Sat Nov 2 09:14:59 2019 From: ratmapper@REDACTED (Raimo Niskanen) Date: Sat, 2 Nov 2019 09:14:59 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: Message-ID: Yes it does. It applies to all mailing lists. Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. Best regards / Raimo Niskanen Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > Does this apply to the EEPS list as well? > > On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > > > > Thanks for doing all of this, regardless. > > > > There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > > way that doesn't break some client's "From:" field, subject line, or > > "Reply:" button in some way, but this seems like the least bad option. > > > > I hope my emails make it through to the list now ^_^ > > > > OT: Be careful of organisations' web contact forms which ask for your > > email address. Sometimes their web servers generate an email from the > > form using your email address as the "From:" address, which will break a > > lot of DKIM/DMARC/SPF stuff. > > I know of at least one local authority (council) website in the UK which > > is guilty of this. > > > > - Joe > > > > On 26/10/2019 07:57, Raimo Niskanen wrote: > > > It is mainly "the big ones" that have been affected by stricter DMARC > > > policies. > > > > > > When a subscriber sending from e.g Yahoo gets received by Gmail then > > > Gmail rejects that message since Yahoo's DMARC policy says so (also > vice > > > versa). So the list gets a bounce and eventually blocks the Gmail > > > subscriber, if enough in a row happens to send with strict DMARC > policies. > > > > > > So for some it has worked, some gets an annoying list probe every now > > > and then, some do not get many posts, but the final nail in the coffin > > > was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > > > policy and at the same time told us to get our act together and stop > > > sending "unhygienic e-mail". > > > > > > All the best > > > / Raimo > > > > > > > > > Den fre 25 okt. 2019 16:58Chris Rempel > > > skrev: > > > > > > Not having the subject contain [erlang-questions] or some other > > > obvious indicator is quite unfortunate. I guess many people were > > > affected by not being DMARC compliant? It seems to have been > > > working just fine for quite some time... ie it "works for me" as > it was. > > > > > > That said, thanks for maintaining the list, and keeping it going. > > > It is a most useful resource. > > > > > > Chris > > > > > > *Sent:* Friday, October 25, 2019 at 7:38 AM > > > *From:* "Raimo Niskanen" > > > > > > *To:* erlang-questions@REDACTED erlang-questions@REDACTED> > > > *Subject:* Re: Nobody is unsubscribed > > > To achieve DMARC compliance we have stopped changing the Subject: > > > field and no longer add the mailing list footer to the messages. > > > > > > This is because From: Subject: and mail body among other fields are > > > often DKIM signed, so if we should change them we would not pass > DKIM > > > signature check and thereby not be DMARC compliant. > > > > > > Sorry for the inconvenience, we do not make the rules... > > > / Raimo Niskanen > > > > > > On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen < > ratmapper@REDACTED > > > > wrote: > > > > > > > > The reason we changed mailing list servers was to get better > DMARC and > > > > DKIM compliance. This is a test post for us to inspect its > headers... > > > > -- > > > > Raimo Niskanen > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eckard.brauer@REDACTED Sat Nov 2 10:01:00 2019 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Sat, 2 Nov 2019 10:01:00 +0100 Subject: [erlang-questions] Segfault with Erlang R22 In-Reply-To: References: <649d87fb8e0346f7ad0a039e483a46af@morganstanley.com> <50f54b6f7f9b4e88ae3d1e540c881ba2@morganstanley.com> Message-ID: <20191102100047.296cb65d@willy.fritz.box> It's a few years ago, but IIRC either "print *c_p" or "print *((Process*) c_p)". Problem would probably be that the processor already left the stack frame where c_p is valid. You can do "info stack" at this point and select the frame with "frame <#>" to try it again. If you're a little familiar with assembly language, you can even have a look at "disassemble
" or "disassemble function" to get an idea of where values are at what point in the instruction/processing flow - sometimes this helps too. I'd investigate starting with frame 2 here, as all frames below are already in libpthread. Hope that helps a bit... Eckard Am Fri, 1 Nov 2019 18:22:18 +0000 schrieb "Bekes, Andras G" : > Program terminated with signal 11, Segmentation fault. > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > > 5252 c_p->seq_trace_lastcnt = > unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); Missing separate > debuginfos, use: debuginfo-install glibc-2.12-1.212.el6_10.3.x86_64 > (gdb) bt > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at > beam/erl_process.c:8465 > #2 0x000000000069262a in thr_wrapper (vtwd=) > at pthread/ethread.c:118 > #3 0x00002b81f80f7dd5 in _L_unlock_48 () from /lib64/libpthread.so.0 > #4 0x00002b81f80f5eb3 in __find_thread_by_id () from > /lib64/libpthread.so.0 > #5 0x0000000000000000 in ?? () > (gdb) > > I am not sure how to " print the c_p parameter via its raw value > (print *(Process*)0x.....)". Where should I take the value 0x..... > from? > > -----Original Message----- > From: Mikael Pettersson [mailto:mikpelinux@REDACTED] > Sent: Thursday, October 24, 2019 7:10 PM > To: Bekes, Andras G (IST) > Cc: Erlang Questions > Subject: Re: [erlang-questions] Segfault with Erlang R22 > > On Thu, Oct 24, 2019 at 4:57 PM Bekes, Andras G > wrote: > [...] > > I'd try to get a backtrace (bt command in gdb) from the crashed > thread, then maybe print > the c_p parameter via its raw value (print *(Process*)0x.....) if gdb > insists that the value > is optimized out. > > /Mikael > > [...] > [...] > [...] > -- Wir haften nicht f?r die korrekte Funktion der in dieser eMail enthaltenen Viren. We are not liable for correct function of the viruses in this email! :) From beastie@REDACTED Sat Nov 2 15:17:06 2019 From: beastie@REDACTED (Mark Reynolds) Date: Sat, 02 Nov 2019 15:17:06 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hey, I confirm that out of order certs does not seems to be fixed, and it fails with 'Unknown CA' error: iex(2)> :hackney.get("https://social.fluffel.io") {:error, {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} the only issue with this server TLS certificates is the chain order (CA is Letsencrypt): https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: > Hi! > > Just curious if there is an update on out of order certs. > > The example has id0, id1, id2, id3 certs with id1 being the natural > root of id2 who is the root of id3, who is the root of id0. > > We can correct the out of order problem by including id1,id2,id3 certs > in our chain. > > It would be nice to hear from the erlang maintainers around what kind of > "out of order" erlang can handle nicely and if there is planned support for > our case! > > Thank you again, > > Curtis. > > > Sent through ProtonMail Encrypted Email Channel. > > > ??????? Original Message ??????? > On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield wrote: > >> Hi! Thank you. >> >> >> I included the root cert in the example. The root cert is id1 in cert chain - this is evident in the other file. >> >> It seems because the root cert is out of order - the cert chain is invalid - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 >> >> >> Thank you for your consideration! >> >> >> Sent from ProtonMail Mobile >> >> >> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin wrote: >>> >>> Hi! >>> >>> "Unknown CA" means that you did not have the ROOT certificate of the chian in your "trusted store" (cacerts option). >>> If you do not own the ROOT certificate you can not trust the chain. >>> >>> Regards Ingela Erlang/OTP Team - Ericsson AB >>> >>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >>>> Dear Erlang Questions: >>>> >>>> >>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>> >>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>> for host hooks.glip.com. >>>> >>>> When we try to verify peer with the out of order cert >>>> chain we get 'Unknown CA'. >>>> >>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>> >>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>> mention that other care may need to be taken to ensure compatibility. >>>> >>>> Reproduce error: >>>> >>>> https://github.com/robotarmy/out-of-order-ssl >>>> >>>> Thank you, >>>> Curtis and Team DevEco >>>> >>>> >>>> >>>> >>>> Sent through ProtonMail Encrypted Email Channel. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Mon Nov 4 10:47:03 2019 From: hello@REDACTED (Adam Lindberg) Date: Mon, 4 Nov 2019 10:47:03 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: Message-ID: Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? Cheers, Adam > On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > > Yes it does. It applies to all mailing lists. > > Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > > Best regards > / Raimo Niskanen > > Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > Does this apply to the EEPS list as well? > > On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > > > > Thanks for doing all of this, regardless. > > > > There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > > way that doesn't break some client's "From:" field, subject line, or > > "Reply:" button in some way, but this seems like the least bad option. > > > > I hope my emails make it through to the list now ^_^ > > > > OT: Be careful of organisations' web contact forms which ask for your > > email address. Sometimes their web servers generate an email from the > > form using your email address as the "From:" address, which will break a > > lot of DKIM/DMARC/SPF stuff. > > I know of at least one local authority (council) website in the UK which > > is guilty of this. > > > > - Joe > > > > On 26/10/2019 07:57, Raimo Niskanen wrote: > > > It is mainly "the big ones" that have been affected by stricter DMARC > > > policies. > > > > > > When a subscriber sending from e.g Yahoo gets received by Gmail then > > > Gmail rejects that message since Yahoo's DMARC policy says so (also vice > > > versa). So the list gets a bounce and eventually blocks the Gmail > > > subscriber, if enough in a row happens to send with strict DMARC policies. > > > > > > So for some it has worked, some gets an annoying list probe every now > > > and then, some do not get many posts, but the final nail in the coffin > > > was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > > > policy and at the same time told us to get our act together and stop > > > sending "unhygienic e-mail". > > > > > > All the best > > > / Raimo > > > > > > > > > Den fre 25 okt. 2019 16:58Chris Rempel > > > skrev: > > > > > > Not having the subject contain [erlang-questions] or some other > > > obvious indicator is quite unfortunate. I guess many people were > > > affected by not being DMARC compliant? It seems to have been > > > working just fine for quite some time... ie it "works for me" as it was. > > > > > > That said, thanks for maintaining the list, and keeping it going. > > > It is a most useful resource. > > > > > > Chris > > > > > > *Sent:* Friday, October 25, 2019 at 7:38 AM > > > *From:* "Raimo Niskanen" > > > > > > *To:* erlang-questions@REDACTED > > > *Subject:* Re: Nobody is unsubscribed > > > To achieve DMARC compliance we have stopped changing the Subject: > > > field and no longer add the mailing list footer to the messages. > > > > > > This is because From: Subject: and mail body among other fields are > > > often DKIM signed, so if we should change them we would not pass DKIM > > > signature check and thereby not be DMARC compliant. > > > > > > Sorry for the inconvenience, we do not make the rules... > > > / Raimo Niskanen > > > > > > On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen > > > wrote: > > > > > > > > The reason we changed mailing list servers was to get better DMARC and > > > > DKIM compliance. This is a test post for us to inspect its headers... > > > > -- > > > > Raimo Niskanen > > > > > From raimo+erlang-questions@REDACTED Mon Nov 4 11:34:27 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 4 Nov 2019 11:34:27 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: Message-ID: <20191104103427.GA81766@erix.ericsson.se> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: > Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? HTTPS has been active for www.erlang.org and bugs.erlang.org for years. The recent web server upgrade enabled it for erlang.org as well; we are working on it... Best regards / Raimo > > Cheers, > Adam > > > On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > > > > Yes it does. It applies to all mailing lists. > > > > Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > > > > Best regards > > / Raimo Niskanen > > > > Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > > Does this apply to the EEPS list as well? > > > > On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > > > > > > Thanks for doing all of this, regardless. > > > > > > There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > > > way that doesn't break some client's "From:" field, subject line, or > > > "Reply:" button in some way, but this seems like the least bad option. > > > > > > I hope my emails make it through to the list now ^_^ > > > > > > OT: Be careful of organisations' web contact forms which ask for your > > > email address. Sometimes their web servers generate an email from the > > > form using your email address as the "From:" address, which will break a > > > lot of DKIM/DMARC/SPF stuff. > > > I know of at least one local authority (council) website in the UK which > > > is guilty of this. > > > > > > - Joe > > > > > > On 26/10/2019 07:57, Raimo Niskanen wrote: > > > > It is mainly "the big ones" that have been affected by stricter DMARC > > > > policies. > > > > > > > > When a subscriber sending from e.g Yahoo gets received by Gmail then > > > > Gmail rejects that message since Yahoo's DMARC policy says so (also vice > > > > versa). So the list gets a bounce and eventually blocks the Gmail > > > > subscriber, if enough in a row happens to send with strict DMARC policies. > > > > > > > > So for some it has worked, some gets an annoying list probe every now > > > > and then, some do not get many posts, but the final nail in the coffin > > > > was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > > > > policy and at the same time told us to get our act together and stop > > > > sending "unhygienic e-mail". > > > > > > > > All the best > > > > / Raimo > > > > > > > > > > > > Den fre 25 okt. 2019 16:58Chris Rempel > > > > skrev: > > > > > > > > Not having the subject contain [erlang-questions] or some other > > > > obvious indicator is quite unfortunate. I guess many people were > > > > affected by not being DMARC compliant? It seems to have been > > > > working just fine for quite some time... ie it "works for me" as it was. > > > > > > > > That said, thanks for maintaining the list, and keeping it going. > > > > It is a most useful resource. > > > > > > > > Chris > > > > > > > > *Sent:* Friday, October 25, 2019 at 7:38 AM > > > > *From:* "Raimo Niskanen" > > > > > > > > *To:* erlang-questions@REDACTED > > > > *Subject:* Re: Nobody is unsubscribed > > > > To achieve DMARC compliance we have stopped changing the Subject: > > > > field and no longer add the mailing list footer to the messages. > > > > > > > > This is because From: Subject: and mail body among other fields are > > > > often DKIM signed, so if we should change them we would not pass DKIM > > > > signature check and thereby not be DMARC compliant. > > > > > > > > Sorry for the inconvenience, we do not make the rules... > > > > / Raimo Niskanen > > > > > > > > On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen > > > > wrote: > > > > > > > > > > The reason we changed mailing list servers was to get better DMARC and > > > > > DKIM compliance. This is a test post for us to inspect its headers... > > > > > -- > > > > > Raimo Niskanen > > > > > > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Mon Nov 4 11:53:16 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 4 Nov 2019 11:53:16 +0100 Subject: Nobody is unsubscribed In-Reply-To: <20191104103427.GA81766@erix.ericsson.se> References: <20191104103427.GA81766@erix.ericsson.se> Message-ID: <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> For erlang.org itself there's two problems currently: no automatic redirection from http to https; And this: Your connection is not private This server could not prove that it is erlang.org; its security certificate is from www2.erlang.org. This may be caused by a misconfiguration or an attacker intercepting your connection. NET::ERR_CERT_COMMON_NAME_INVALID Subject: www2.erlang.org Issuer: DigiCert SHA2 Secure Server CA Expires on: Oct 22, 2021 Current date: Nov 4, 2019 Keep up the good work. On 04/11/2019 11:34, Raimo Niskanen wrote: > On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? > > HTTPS has been active for www.erlang.org and bugs.erlang.org for years. > The recent web server upgrade enabled it for erlang.org as well; > we are working on it... > > Best regards > / Raimo > > >> >> Cheers, >> Adam >> >>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>> >>> Yes it does. It applies to all mailing lists. >>> >>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>> >>> Best regards >>> / Raimo Niskanen >>> >>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>> Does this apply to the EEPS list as well? >>> >>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>> >>>> Thanks for doing all of this, regardless. >>>> >>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>> way that doesn't break some client's "From:" field, subject line, or >>>> "Reply:" button in some way, but this seems like the least bad option. >>>> >>>> I hope my emails make it through to the list now ^_^ >>>> >>>> OT: Be careful of organisations' web contact forms which ask for your >>>> email address. Sometimes their web servers generate an email from the >>>> form using your email address as the "From:" address, which will break a >>>> lot of DKIM/DMARC/SPF stuff. >>>> I know of at least one local authority (council) website in the UK which >>>> is guilty of this. >>>> >>>> - Joe >>>> >>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>> policies. >>>>> >>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>> >>>>> So for some it has worked, some gets an annoying list probe every now >>>>> and then, some do not get many posts, but the final nail in the coffin >>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>> policy and at the same time told us to get our act together and stop >>>>> sending "unhygienic e-mail". >>>>> >>>>> All the best >>>>> / Raimo >>>>> >>>>> >>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>> > skrev: >>>>> >>>>> Not having the subject contain [erlang-questions] or some other >>>>> obvious indicator is quite unfortunate. I guess many people were >>>>> affected by not being DMARC compliant? It seems to have been >>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>> >>>>> That said, thanks for maintaining the list, and keeping it going. >>>>> It is a most useful resource. >>>>> >>>>> Chris >>>>> >>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>> *From:* "Raimo Niskanen" >>>> > >>>>> *To:* erlang-questions@REDACTED >>>>> *Subject:* Re: Nobody is unsubscribed >>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>> field and no longer add the mailing list footer to the messages. >>>>> >>>>> This is because From: Subject: and mail body among other fields are >>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>> signature check and thereby not be DMARC compliant. >>>>> >>>>> Sorry for the inconvenience, we do not make the rules... >>>>> / Raimo Niskanen >>>>> >>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>> > wrote: >>>>> > >>>>> > The reason we changed mailing list servers was to get better DMARC and >>>>> > DKIM compliance. This is a test post for us to inspect its headers... >>>>> > -- >>>>> > Raimo Niskanen >>>>> >>>> >> > -- Lo?c Hoguin https://ninenines.eu From by@REDACTED Mon Nov 4 12:45:41 2019 From: by@REDACTED (by) Date: Mon, 4 Nov 2019 19:45:41 +0800 Subject: Nobody is unsubscribed In-Reply-To: <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> References: <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> Message-ID: Exactly! For mailing list web page, I can visit http and https via below links respectively: http://erlang.org/mailman/listinfo/erlang-questions https://www2.erlang.org/mailman/listinfo/erlang-questions Yao > ? 2019?11?4??18:53?Lo?c Hoguin ??? > > ?For erlang.org itself there's two problems currently: no automatic redirection from http to https; > > And this: > > Your connection is not private > This server could not prove that it is erlang.org; its security certificate is from www2.erlang.org. This may be caused by a misconfiguration or an attacker intercepting your connection. > > NET::ERR_CERT_COMMON_NAME_INVALID > Subject: www2.erlang.org > > Issuer: DigiCert SHA2 Secure Server CA > > Expires on: Oct 22, 2021 > > Current date: Nov 4, 2019 > > Keep up the good work. > >> On 04/11/2019 11:34, Raimo Niskanen wrote: >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? >> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. >> The recent web server upgrade enabled it for erlang.org as well; >> we are working on it... >> Best regards >> / Raimo >>> >>> Cheers, >>> Adam >>> >>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>>> >>>> Yes it does. It applies to all mailing lists. >>>> >>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>>> >>>> Best regards >>>> / Raimo Niskanen >>>> >>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>>> Does this apply to the EEPS list as well? >>>> >>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>>> >>>>> Thanks for doing all of this, regardless. >>>>> >>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>>> way that doesn't break some client's "From:" field, subject line, or >>>>> "Reply:" button in some way, but this seems like the least bad option. >>>>> >>>>> I hope my emails make it through to the list now ^_^ >>>>> >>>>> OT: Be careful of organisations' web contact forms which ask for your >>>>> email address. Sometimes their web servers generate an email from the >>>>> form using your email address as the "From:" address, which will break a >>>>> lot of DKIM/DMARC/SPF stuff. >>>>> I know of at least one local authority (council) website in the UK which >>>>> is guilty of this. >>>>> >>>>> - Joe >>>>> >>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>>> policies. >>>>>> >>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>>> >>>>>> So for some it has worked, some gets an annoying list probe every now >>>>>> and then, some do not get many posts, but the final nail in the coffin >>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>>> policy and at the same time told us to get our act together and stop >>>>>> sending "unhygienic e-mail". >>>>>> >>>>>> All the best >>>>>> / Raimo >>>>>> >>>>>> >>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>> > skrev: >>>>>> >>>>>> Not having the subject contain [erlang-questions] or some other >>>>>> obvious indicator is quite unfortunate. I guess many people were >>>>>> affected by not being DMARC compliant? It seems to have been >>>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>>> >>>>>> That said, thanks for maintaining the list, and keeping it going. >>>>>> It is a most useful resource. >>>>>> >>>>>> Chris >>>>>> >>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>>> *From:* "Raimo Niskanen" >>>>> > >>>>>> *To:* erlang-questions@REDACTED >>>>>> *Subject:* Re: Nobody is unsubscribed >>>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>>> field and no longer add the mailing list footer to the messages. >>>>>> >>>>>> This is because From: Subject: and mail body among other fields are >>>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>>> signature check and thereby not be DMARC compliant. >>>>>> >>>>>> Sorry for the inconvenience, we do not make the rules... >>>>>> / Raimo Niskanen >>>>>> >>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>> > wrote: >>>>>> > >>>>>> > The reason we changed mailing list servers was to get better DMARC and >>>>>> > DKIM compliance. This is a test post for us to inspect its headers... >>>>>> > -- >>>>>> > Raimo Niskanen >>>>>> >>>>> >>> > > -- > Lo?c Hoguin > https://ninenines.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From marthin@REDACTED Mon Nov 4 13:08:51 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Mon, 04 Nov 2019 14:08:51 +0200 Subject: Solving the right problem Message-ID: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Hi everyone, Please pardon my long absences from this awesome (mature) community and my limited knowledge which is likely outdated as well. I?ve known since 1996 when I was first told (in confidence by an Ericsson Radio Systems liaison) about Erlang that it would have to play a role when I eventually get to implementing the system I?ve been working on designing since 1991. That big day is drawing near, so now I?d like to reaffirm my high level understanding of what the language is suited for. I reckon the problem I?m looking to address is intrinsically concurrent and if I can design the solution just right I just might be able to avoid introducing sequential code and choke points to create an dynamic (learning, responsive to conditions) distributed server capable of using all or most of its resources for any mixture of trivial, simple, complex and massive service requests whether it?s coming from a few clients or billions of them. Essentially as illustrated in the diagram below: I?d like to ask your advice and check some assumptions if I may impose. Is my conviction that Erlang (and OTP) is ideally if not best suited to addressing this type of problem objectively justified or likely based on loyalty and familiarity? Is my aspiration to scale the same code from one server to potentially hundreds of thousands of cores spread across the globe hopelessly romantic or somewhere within the realm of possibility? Assuming the network remains HTTP/HTTPS based, would Erlang?s inets module allow the code accepting new requests to interact with and control load balancing hardware to ensure each such request is served on the best available server, or will I need custom load balancers of my own for that? Still assuming HTTP/HTTPS will inets allow me to break up request processing across multiple processes and threads based on incremental scanning of the requests themselves? Are there lessons from previous (or current) successes and/or failures to achieve similar results to learn from available in the public domain like maybe from ejabberd or Yaws? ?(I?m not attempting to reinvent any wheels or address a general purpose need like Yaws et al. Internet and web protocols may be involved but I have a singular focus on delivering user-specific perspectives of a single large dataset to a custom client app.) ? Given my two additional objectives of eventually:- ending up with a boringly simple system which elegantly encapsulates all these complex requirements, and open-sourcing the entire system once it?s beyond reach of those with nefarious intentions, would anybody like to get involved in helping design and implement this project or even take the lead on it? Thank you in advance for your kind consideration. Warm regards, Marthin Laubscher -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 64024 bytes Desc: not available URL: From raimo+erlang-questions@REDACTED Mon Nov 4 13:44:42 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 4 Nov 2019 13:44:42 +0100 Subject: Nobody is unsubscribed In-Reply-To: <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> Message-ID: <20191104124442.GA28874@erix.ericsson.se> On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: > For erlang.org itself there's two problems currently: no automatic > redirection from http to https; That seems to be the industry standard now, but I would like content to be accessible without having to use https. The redirect for http://erlang.org and https://erlang.org goes to $scheme://www.erlang.org, which redirects to https://www.erlang.org. Unfortunately the redirects back from e.g https://www.erlang.org/doc changes to http://erlang.org/doc because https for erlang.org did not work until 10 minutes ago. Would it be sufficient to make those redirects from www.erlang.org to erlang.org not change from https to http? That, and the answer 20 lines down...? > > And this: > > Your connection is not private > This server could not prove that it is erlang.org; its security > certificate is from www2.erlang.org. This may be caused by a > misconfiguration or an attacker intercepting your connection. > > NET::ERR_CERT_COMMON_NAME_INVALID > Subject: www2.erlang.org > > Issuer: DigiCert SHA2 Secure Server CA > > Expires on: Oct 22, 2021 > > Current date: Nov 4, 2019 A new certificate is in place, so this should be fixed. / Raimo > > Keep up the good work. > > On 04/11/2019 11:34, Raimo Niskanen wrote: > > On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: > >> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? > > > > HTTPS has been active for www.erlang.org and bugs.erlang.org for years. > > The recent web server upgrade enabled it for erlang.org as well; > > we are working on it... > > > > Best regards > > / Raimo > > > > > >> > >> Cheers, > >> Adam > >> > >>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > >>> > >>> Yes it does. It applies to all mailing lists. > >>> > >>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > >>> > >>> Best regards > >>> / Raimo Niskanen > >>> > >>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > >>> Does this apply to the EEPS list as well? > >>> > >>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > >>>> > >>>> Thanks for doing all of this, regardless. > >>>> > >>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > >>>> way that doesn't break some client's "From:" field, subject line, or > >>>> "Reply:" button in some way, but this seems like the least bad option. > >>>> > >>>> I hope my emails make it through to the list now ^_^ > >>>> > >>>> OT: Be careful of organisations' web contact forms which ask for your > >>>> email address. Sometimes their web servers generate an email from the > >>>> form using your email address as the "From:" address, which will break a > >>>> lot of DKIM/DMARC/SPF stuff. > >>>> I know of at least one local authority (council) website in the UK which > >>>> is guilty of this. > >>>> > >>>> - Joe > >>>> > >>>> On 26/10/2019 07:57, Raimo Niskanen wrote: > >>>>> It is mainly "the big ones" that have been affected by stricter DMARC > >>>>> policies. > >>>>> > >>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then > >>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice > >>>>> versa). So the list gets a bounce and eventually blocks the Gmail > >>>>> subscriber, if enough in a row happens to send with strict DMARC policies. > >>>>> > >>>>> So for some it has worked, some gets an annoying list probe every now > >>>>> and then, some do not get many posts, but the final nail in the coffin > >>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > >>>>> policy and at the same time told us to get our act together and stop > >>>>> sending "unhygienic e-mail". > >>>>> > >>>>> All the best > >>>>> / Raimo > >>>>> > >>>>> > >>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>> > skrev: > >>>>> > >>>>> Not having the subject contain [erlang-questions] or some other > >>>>> obvious indicator is quite unfortunate. I guess many people were > >>>>> affected by not being DMARC compliant? It seems to have been > >>>>> working just fine for quite some time... ie it "works for me" as it was. > >>>>> > >>>>> That said, thanks for maintaining the list, and keeping it going. > >>>>> It is a most useful resource. > >>>>> > >>>>> Chris > >>>>> > >>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM > >>>>> *From:* "Raimo Niskanen" >>>>> > > >>>>> *To:* erlang-questions@REDACTED > >>>>> *Subject:* Re: Nobody is unsubscribed > >>>>> To achieve DMARC compliance we have stopped changing the Subject: > >>>>> field and no longer add the mailing list footer to the messages. > >>>>> > >>>>> This is because From: Subject: and mail body among other fields are > >>>>> often DKIM signed, so if we should change them we would not pass DKIM > >>>>> signature check and thereby not be DMARC compliant. > >>>>> > >>>>> Sorry for the inconvenience, we do not make the rules... > >>>>> / Raimo Niskanen > >>>>> > >>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>> > wrote: > >>>>> > > >>>>> > The reason we changed mailing list servers was to get better DMARC and > >>>>> > DKIM compliance. This is a test post for us to inspect its headers... > >>>>> > -- > >>>>> > Raimo Niskanen > >>>>> > >>>> > >> > > > > -- > Lo?c Hoguin > https://ninenines.eu -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From by@REDACTED Mon Nov 4 14:30:08 2019 From: by@REDACTED (by) Date: Mon, 4 Nov 2019 21:30:08 +0800 Subject: Configure Emacs for Erlang development Message-ID: <7F9F1238-2507-4843-B381-32FD214E3481@meetlost.com> Hi, I followed the guide with link (From Adopting Erlang): https://adoptingerlang.org/docs/development/setup/ Seems "ivy-erlang-complete" is not properly configured. Here is some environment information: Erlang version: Erlang/OTP 22.1.4 Emacs version: GNU Emacs 25.3.1 And here is my related ~/.emacs configuration for Erlang: ==== ;; for ivy-erlang-complete (use-package ivy-erlang-complete :ensure t) ;; for delight (use-package delight :ensure t) ;; for flycheck (use-package flycheck :ensure t :delight :config (global-flycheck-mode)) ;; for erlang (use-package erlang :load-path (?//lib/erlang/lib/tools-3.2.1/emacs") :hook (after-save . ivy-erlang-complete-reparse) :custom (ivy-erlang-complete-erlang-root "//lib/erlang") :config (ivy-erlang-complete-init) :mode (("\\.erl?$" . erlang-mode) ("rebar\\.config$" . erlang-mode) ("relx\\.config$" . erlang-mode) ("sys\\.config\\.src$" . erlang-mode) ("sys\\.config$" . erlang-mode) ("\\.config\\.src?$" . erlang-mode) ("\\.config\\.script?$" . erlang-mode) ("\\.hrl?$" . erlang-mode) ("\\.app?$" . erlang-mode) ("\\.app.src?$" . erlang-mode) ("\\Emakefile" . erlang-mode))) ==== When I restart my emacs with above configuration, I got error message as below: ==== Error (use-package): erlang/:config: Symbol?s function definition is void: ivy-erlang-complete-setup-flycheck ==== I have no clue about what happened. An example ~/.emacs file related to Erlang would be very helpful. I searched it with Google, but seems no typical configuration demo/example exist for configuring Emacs with Erlang. By the way, I want to ask: what kind of editor configuration do you use to develop Erlang every day? Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Mon Nov 4 15:54:04 2019 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Mon, 04 Nov 2019 09:54:04 -0500 Subject: Configure Emacs for Erlang development In-Reply-To: <7F9F1238-2507-4843-B381-32FD214E3481@meetlost.com> References: <7F9F1238-2507-4843-B381-32FD214E3481@meetlost.com> Message-ID: > By the way, I want to ask: what kind of editor configuration do you use to develop Erlang every day? Although I always mean to get better at Emacs and fully customize it to perfection, I never have. In lieu of personalization, I use Spacemacs (http://spacemacs.org/#) with the 'erlang' layer: http://spacemacs.org/layers/+lang/erlang/README.html The only thing I need for Erlang support is the 'erlang' in the configuration-layers list of my .spacemacs file, which starts like this: (defun dotspacemacs/layers () "Configuration Layers declaration. You should not put any user code in this function besides modifying the variable values." (setq-default dotspacemacs-distribution 'spacemacs dotspacemacs-enable-lazy-installation 'unused dotspacemacs-ask-for-lazy-installation t dotspacemacs-configuration-layer-path '() dotspacemacs-configuration-layers '( go lua ansible common-lisp csv docker emacs-lisp erlang git helm html javascript latex markdown org python ruby rust shell-scripts spell-checking systemd terraform yaml ) ;; etc ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Mon Nov 4 16:16:25 2019 From: t@REDACTED (Tristan Sloughter) Date: Mon, 04 Nov 2019 08:16:25 -0700 Subject: Configure Emacs for Erlang development In-Reply-To: <7F9F1238-2507-4843-B381-32FD214E3481@meetlost.com> References: <7F9F1238-2507-4843-B381-32FD214E3481@meetlost.com> Message-ID: <844bd1a8-4507-4ec3-872c-4db301c7c353@www.fastmail.com> Hey, that is from my config in adoptingerlang but I don't have it currently separated out and published. Though I did do that at one point to test this and hopefully can find that and I will upload that file and link to it from adoptingerlang. I don't have a good answer for you right now, sadly when I see errors like that I tend to just keep refershing the Emacs and package registry and updating the packages... sometimes having to uninstall and reinstall packages and eventually it works. Emacs package management is not the best.. hehe, as far as I know you can't specify a version of a package and can't lock dependencies. Tristan On Mon, Nov 4, 2019, at 06:30, by wrote: > Hi, > > I followed the guide with link (From Adopting Erlang): https://adoptingerlang.org/docs/development/setup/ > Seems "ivy-erlang-complete" is not properly configured. > > Here is some environment information: > Erlang version: Erlang/OTP 22.1.4 > Emacs version: GNU Emacs 25.3.1 > > And here is my related ~/.emacs configuration for Erlang: > ==== > ;; for ivy-erlang-complete > (use-package ivy-erlang-complete > :ensure t) > > ;; for delight > (use-package delight > :ensure t) > > ;; for flycheck > (use-package flycheck > :ensure t > :delight > :config (global-flycheck-mode)) > > ;; for erlang > (use-package erlang > :load-path (?//lib/erlang/lib/tools-3.2.1/emacs") > :hook (after-save . ivy-erlang-complete-reparse) > :custom (ivy-erlang-complete-erlang-root "//lib/erlang") > :config (ivy-erlang-complete-init) > :mode (("\\.erl?$" . erlang-mode) > ("rebar\\.config$" . erlang-mode) > ("relx\\.config$" . erlang-mode) > ("sys\\.config\\.src$" . erlang-mode) > ("sys\\.config$" . erlang-mode) > ("\\.config\\.src?$" . erlang-mode) > ("\\.config\\.script?$" . erlang-mode) > ("\\.hrl?$" . erlang-mode) > ("\\.app?$" . erlang-mode) > ("\\.app.src?$" . erlang-mode) > ("\\Emakefile" . erlang-mode))) > ==== > > When I restart my emacs with above configuration, I got error message as below: > ==== > Error (use-package): erlang/:config: Symbol?s function definition is void: ivy-erlang-complete-setup-flycheck > ==== > I have no clue about what happened. > > An example ~/.emacs file related to Erlang would be very helpful. > > I searched it with Google, but seems no typical configuration demo/example exist for configuring Emacs with Erlang. > > By the way, I want to ask: what kind of editor configuration do you use to develop Erlang every day? > > Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From by@REDACTED Mon Nov 4 16:44:05 2019 From: by@REDACTED (by) Date: Mon, 4 Nov 2019 23:44:05 +0800 Subject: Configure Emacs for Erlang development In-Reply-To: <844bd1a8-4507-4ec3-872c-4db301c7c353@www.fastmail.com> References: <844bd1a8-4507-4ec3-872c-4db301c7c353@www.fastmail.com> Message-ID: Yes, same configuration may behave differently because of different package versions. A working package does not mean it will keep working in future updates. The documentation and change log become significant in this case. Yao >> ? 2019?11?4??23:16?Tristan Sloughter ??? > ? > Hey, that is from my config in adoptingerlang but I don't have it currently separated out and published. Though I did do that at one point to test this and hopefully can find that and I will upload that file and link to it from adoptingerlang. > > I don't have a good answer for you right now, sadly when I see errors like that I tend to just keep refershing the Emacs and package registry and updating the packages... sometimes having to uninstall and reinstall packages and eventually it works. > > Emacs package management is not the best.. hehe, as far as I know you can't specify a version of a package and can't lock dependencies. > > Tristan > >> On Mon, Nov 4, 2019, at 06:30, by wrote: >> Hi, >> >> I followed the guide with link (From Adopting Erlang): https://adoptingerlang.org/docs/development/setup/ >> Seems "ivy-erlang-complete" is not properly configured. >> >> Here is some environment information: >> Erlang version: Erlang/OTP 22.1.4 >> Emacs version: GNU Emacs 25.3.1 >> >> And here is my related ~/.emacs configuration for Erlang: >> ==== >> ;; for ivy-erlang-complete >> (use-package ivy-erlang-complete >> :ensure t) >> >> ;; for delight >> (use-package delight >> :ensure t) >> >> ;; for flycheck >> (use-package flycheck >> :ensure t >> :delight >> :config (global-flycheck-mode)) >> >> ;; for erlang >> (use-package erlang >> :load-path (?//lib/erlang/lib/tools-3.2.1/emacs") >> :hook (after-save . ivy-erlang-complete-reparse) >> :custom (ivy-erlang-complete-erlang-root "//lib/erlang") >> :config (ivy-erlang-complete-init) >> :mode (("\\.erl?$" . erlang-mode) >> ("rebar\\.config$" . erlang-mode) >> ("relx\\.config$" . erlang-mode) >> ("sys\\.config\\.src$" . erlang-mode) >> ("sys\\.config$" . erlang-mode) >> ("\\.config\\.src?$" . erlang-mode) >> ("\\.config\\.script?$" . erlang-mode) >> ("\\.hrl?$" . erlang-mode) >> ("\\.app?$" . erlang-mode) >> ("\\.app.src?$" . erlang-mode) >> ("\\Emakefile" . erlang-mode))) >> ==== >> >> When I restart my emacs with above configuration, I got error message as below: >> ==== >> Error (use-package): erlang/:config: Symbol?s function definition is void: ivy-erlang-complete-setup-flycheck >> ==== >> I have no clue about what happened. >> >> An example ~/.emacs file related to Erlang would be very helpful. >> >> I searched it with Google, but seems no typical configuration demo/example exist for configuring Emacs with Erlang. >> >> By the way, I want to ask: what kind of editor configuration do you use to develop Erlang every day? >> >> Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Mon Nov 4 17:15:44 2019 From: hello@REDACTED (Adam Lindberg) Date: Mon, 4 Nov 2019 17:15:44 +0100 Subject: Nobody is unsubscribed In-Reply-To: <20191104124442.GA28874@erix.ericsson.se> References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191104124442.GA28874@erix.ericsson.se> Message-ID: <6F8397E9-0DC2-4A6E-80A1-8B7A6E0DB4B5@alind.io> Have to admit I only tried typing https://erlang.org and noticed the errors. Cheers, Adam > On 4. Nov 2019, at 13:44, Raimo Niskanen wrote: > > On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: >> For erlang.org itself there's two problems currently: no automatic >> redirection from http to https; > > That seems to be the industry standard now, but I would like content to be > accessible without having to use https. > > The redirect for http://erlang.org and https://erlang.org goes to > $scheme://www.erlang.org, which redirects to https://www.erlang.org. > > Unfortunately the redirects back from e.g https://www.erlang.org/doc > changes to http://erlang.org/doc because https for erlang.org did not work > until 10 minutes ago. > > Would it be sufficient to make those redirects from www.erlang.org to > erlang.org not change from https to http? > > That, and the answer 20 lines down...? > >> >> And this: >> >> Your connection is not private >> This server could not prove that it is erlang.org; its security >> certificate is from www2.erlang.org. This may be caused by a >> misconfiguration or an attacker intercepting your connection. >> >> NET::ERR_CERT_COMMON_NAME_INVALID >> Subject: www2.erlang.org >> >> Issuer: DigiCert SHA2 Secure Server CA >> >> Expires on: Oct 22, 2021 >> >> Current date: Nov 4, 2019 > > A new certificate is in place, so this should be fixed. > > / Raimo > > >> >> Keep up the good work. >> >> On 04/11/2019 11:34, Raimo Niskanen wrote: >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? >>> >>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. >>> The recent web server upgrade enabled it for erlang.org as well; >>> we are working on it... >>> >>> Best regards >>> / Raimo >>> >>> >>>> >>>> Cheers, >>>> Adam >>>> >>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>>>> >>>>> Yes it does. It applies to all mailing lists. >>>>> >>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>>>> >>>>> Best regards >>>>> / Raimo Niskanen >>>>> >>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>>>> Does this apply to the EEPS list as well? >>>>> >>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>>>> >>>>>> Thanks for doing all of this, regardless. >>>>>> >>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>>>> way that doesn't break some client's "From:" field, subject line, or >>>>>> "Reply:" button in some way, but this seems like the least bad option. >>>>>> >>>>>> I hope my emails make it through to the list now ^_^ >>>>>> >>>>>> OT: Be careful of organisations' web contact forms which ask for your >>>>>> email address. Sometimes their web servers generate an email from the >>>>>> form using your email address as the "From:" address, which will break a >>>>>> lot of DKIM/DMARC/SPF stuff. >>>>>> I know of at least one local authority (council) website in the UK which >>>>>> is guilty of this. >>>>>> >>>>>> - Joe >>>>>> >>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>>>> policies. >>>>>>> >>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>>>> >>>>>>> So for some it has worked, some gets an annoying list probe every now >>>>>>> and then, some do not get many posts, but the final nail in the coffin >>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>>>> policy and at the same time told us to get our act together and stop >>>>>>> sending "unhygienic e-mail". >>>>>>> >>>>>>> All the best >>>>>>> / Raimo >>>>>>> >>>>>>> >>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>> > skrev: >>>>>>> >>>>>>> Not having the subject contain [erlang-questions] or some other >>>>>>> obvious indicator is quite unfortunate. I guess many people were >>>>>>> affected by not being DMARC compliant? It seems to have been >>>>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>>>> >>>>>>> That said, thanks for maintaining the list, and keeping it going. >>>>>>> It is a most useful resource. >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>>>> *From:* "Raimo Niskanen" >>>>>> > >>>>>>> *To:* erlang-questions@REDACTED >>>>>>> *Subject:* Re: Nobody is unsubscribed >>>>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>>>> field and no longer add the mailing list footer to the messages. >>>>>>> >>>>>>> This is because From: Subject: and mail body among other fields are >>>>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>>>> signature check and thereby not be DMARC compliant. >>>>>>> >>>>>>> Sorry for the inconvenience, we do not make the rules... >>>>>>> / Raimo Niskanen >>>>>>> >>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>> > wrote: >>>>>>>> >>>>>>>> The reason we changed mailing list servers was to get better DMARC and >>>>>>>> DKIM compliance. This is a test post for us to inspect its headers... >>>>>>>> -- >>>>>>>> Raimo Niskanen >>>>>>> >>>>>> >>>> >>> >> >> -- >> Lo?c Hoguin >> https://ninenines.eu > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Mon Nov 4 17:30:30 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 4 Nov 2019 17:30:30 +0100 Subject: Nobody is unsubscribed In-Reply-To: <20191104124442.GA28874@erix.ericsson.se> References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191104124442.GA28874@erix.ericsson.se> Message-ID: On 04/11/2019 13:44, Raimo Niskanen wrote: > On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: >> For erlang.org itself there's two problems currently: no automatic >> redirection from http to https; > > That seems to be the industry standard now, but I would like content to be > accessible without having to use https. Redirection is generally not great because you get redirected every time you go through via http. There's HSTS that gets us one step further by telling browsers to remember they have to use HTTPS instead of HTTP, so the initial HTTP call isn't made. > The redirect for http://erlang.org and https://erlang.org goes to > $scheme://www.erlang.org, which redirects to https://www.erlang.org. > > Unfortunately the redirects back from e.g https://www.erlang.org/doc > changes to http://erlang.org/doc because https for erlang.org did not work > until 10 minutes ago. And redirection tends to lead to this issues. > Would it be sufficient to make those redirects from www.erlang.org to > erlang.org not change from https to http? You definitely shouldn't downgrade if possible. I am wondering however if you want to leave *browsers* able to access the site via plain HTTP, or clients in general (including things like curl for example). A policy like HSTS is only used by clients that understand it (so mostly browsers) so maybe this is what you want to setup. Browsers would always go through HTTPS; other clients would be able to use both HTTP and HTTPS. Cheers, > That, and the answer 20 lines down...? > >> >> And this: >> >> Your connection is not private >> This server could not prove that it is erlang.org; its security >> certificate is from www2.erlang.org. This may be caused by a >> misconfiguration or an attacker intercepting your connection. >> >> NET::ERR_CERT_COMMON_NAME_INVALID >> Subject: www2.erlang.org >> >> Issuer: DigiCert SHA2 Secure Server CA >> >> Expires on: Oct 22, 2021 >> >> Current date: Nov 4, 2019 > > A new certificate is in place, so this should be fixed. > > / Raimo > > >> >> Keep up the good work. >> >> On 04/11/2019 11:34, Raimo Niskanen wrote: >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? >>> >>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. >>> The recent web server upgrade enabled it for erlang.org as well; >>> we are working on it... >>> >>> Best regards >>> / Raimo >>> >>> >>>> >>>> Cheers, >>>> Adam >>>> >>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>>>> >>>>> Yes it does. It applies to all mailing lists. >>>>> >>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>>>> >>>>> Best regards >>>>> / Raimo Niskanen >>>>> >>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>>>> Does this apply to the EEPS list as well? >>>>> >>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>>>> >>>>>> Thanks for doing all of this, regardless. >>>>>> >>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>>>> way that doesn't break some client's "From:" field, subject line, or >>>>>> "Reply:" button in some way, but this seems like the least bad option. >>>>>> >>>>>> I hope my emails make it through to the list now ^_^ >>>>>> >>>>>> OT: Be careful of organisations' web contact forms which ask for your >>>>>> email address. Sometimes their web servers generate an email from the >>>>>> form using your email address as the "From:" address, which will break a >>>>>> lot of DKIM/DMARC/SPF stuff. >>>>>> I know of at least one local authority (council) website in the UK which >>>>>> is guilty of this. >>>>>> >>>>>> - Joe >>>>>> >>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>>>> policies. >>>>>>> >>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>>>> >>>>>>> So for some it has worked, some gets an annoying list probe every now >>>>>>> and then, some do not get many posts, but the final nail in the coffin >>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>>>> policy and at the same time told us to get our act together and stop >>>>>>> sending "unhygienic e-mail". >>>>>>> >>>>>>> All the best >>>>>>> / Raimo >>>>>>> >>>>>>> >>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>> > skrev: >>>>>>> >>>>>>> Not having the subject contain [erlang-questions] or some other >>>>>>> obvious indicator is quite unfortunate. I guess many people were >>>>>>> affected by not being DMARC compliant? It seems to have been >>>>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>>>> >>>>>>> That said, thanks for maintaining the list, and keeping it going. >>>>>>> It is a most useful resource. >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>>>> *From:* "Raimo Niskanen" >>>>>> > >>>>>>> *To:* erlang-questions@REDACTED >>>>>>> *Subject:* Re: Nobody is unsubscribed >>>>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>>>> field and no longer add the mailing list footer to the messages. >>>>>>> >>>>>>> This is because From: Subject: and mail body among other fields are >>>>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>>>> signature check and thereby not be DMARC compliant. >>>>>>> >>>>>>> Sorry for the inconvenience, we do not make the rules... >>>>>>> / Raimo Niskanen >>>>>>> >>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>> > wrote: >>>>>>> > >>>>>>> > The reason we changed mailing list servers was to get better DMARC and >>>>>>> > DKIM compliance. This is a test post for us to inspect its headers... >>>>>>> > -- >>>>>>> > Raimo Niskanen >>>>>>> >>>>>> >>>> >>> >> >> -- >> Lo?c Hoguin >> https://ninenines.eu > -- Lo?c Hoguin https://ninenines.eu From beastie@REDACTED Mon Nov 4 21:00:12 2019 From: beastie@REDACTED (Mark Reynolds) Date: Mon, 04 Nov 2019 21:00:12 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191104124442.GA28874@erix.ericsson.se> Message-ID: Using HSTS without a http to https redirection is against the RFC (6797): >If an HSTS Host receives an HTTP request message over a non-secure transport, it SHOULD send an HTTP response message containing a status code indicating a permanent redirect, such as status code 301 Also, it's a requirement for inculsion into the HSTS preload list: > In order to be accepted to the HSTS preload list through this form, your site must satisfy the following set of requirements: [?] > 2- Redirect from HTTP to HTTPS on the same host, if you are listening on port 80. On Mon, Nov 4, 2019, at 17:30, Lo?c Hoguin wrote: > On 04/11/2019 13:44, Raimo Niskanen wrote: > > On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: > >> For erlang.org itself there's two problems currently: no automatic > >> redirection from http to https; > > > > That seems to be the industry standard now, but I would like content to be > > accessible without having to use https. > > Redirection is generally not great because you get redirected every time > you go through via http. There's HSTS that gets us one step further by > telling browsers to remember they have to use HTTPS instead of HTTP, so > the initial HTTP call isn't made. > > > The redirect for http://erlang.org and https://erlang.org goes to > > $scheme://www.erlang.org, which redirects to https://www.erlang.org. > > > > Unfortunately the redirects back from e.g https://www.erlang.org/doc > > changes to http://erlang.org/doc because https for erlang.org did not work > > until 10 minutes ago. > > And redirection tends to lead to this issues. > > > Would it be sufficient to make those redirects from www.erlang.org to > > erlang.org not change from https to http? > > You definitely shouldn't downgrade if possible. I am wondering however > if you want to leave *browsers* able to access the site via plain HTTP, > or clients in general (including things like curl for example). A policy > like HSTS is only used by clients that understand it (so mostly > browsers) so maybe this is what you want to setup. Browsers would always > go through HTTPS; other clients would be able to use both HTTP and HTTPS. > > Cheers, > > > That, and the answer 20 lines down...? > > > >> > >> And this: > >> > >> Your connection is not private > >> This server could not prove that it is erlang.org; its security > >> certificate is from www2.erlang.org. This may be caused by a > >> misconfiguration or an attacker intercepting your connection. > >> > >> NET::ERR_CERT_COMMON_NAME_INVALID > >> Subject: www2.erlang.org > >> > >> Issuer: DigiCert SHA2 Secure Server CA > >> > >> Expires on: Oct 22, 2021 > >> > >> Current date: Nov 4, 2019 > > > > A new certificate is in place, so this should be fixed. > > > > / Raimo > > > > > >> > >> Keep up the good work. > >> > >> On 04/11/2019 11:34, Raimo Niskanen wrote: > >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: > >>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? > >>> > >>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. > >>> The recent web server upgrade enabled it for erlang.org as well; > >>> we are working on it... > >>> > >>> Best regards > >>> / Raimo > >>> > >>> > >>>> > >>>> Cheers, > >>>> Adam > >>>> > >>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > >>>>> > >>>>> Yes it does. It applies to all mailing lists. > >>>>> > >>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > >>>>> > >>>>> Best regards > >>>>> / Raimo Niskanen > >>>>> > >>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > >>>>> Does this apply to the EEPS list as well? > >>>>> > >>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > >>>>>> > >>>>>> Thanks for doing all of this, regardless. > >>>>>> > >>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > >>>>>> way that doesn't break some client's "From:" field, subject line, or > >>>>>> "Reply:" button in some way, but this seems like the least bad option. > >>>>>> > >>>>>> I hope my emails make it through to the list now ^_^ > >>>>>> > >>>>>> OT: Be careful of organisations' web contact forms which ask for your > >>>>>> email address. Sometimes their web servers generate an email from the > >>>>>> form using your email address as the "From:" address, which will break a > >>>>>> lot of DKIM/DMARC/SPF stuff. > >>>>>> I know of at least one local authority (council) website in the UK which > >>>>>> is guilty of this. > >>>>>> > >>>>>> - Joe > >>>>>> > >>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: > >>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC > >>>>>>> policies. > >>>>>>> > >>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then > >>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice > >>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail > >>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. > >>>>>>> > >>>>>>> So for some it has worked, some gets an annoying list probe every now > >>>>>>> and then, some do not get many posts, but the final nail in the coffin > >>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > >>>>>>> policy and at the same time told us to get our act together and stop > >>>>>>> sending "unhygienic e-mail". > >>>>>>> > >>>>>>> All the best > >>>>>>> / Raimo > >>>>>>> > >>>>>>> > >>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>>> > skrev: > >>>>>>> > >>>>>>> Not having the subject contain [erlang-questions] or some other > >>>>>>> obvious indicator is quite unfortunate. I guess many people were > >>>>>>> affected by not being DMARC compliant? It seems to have been > >>>>>>> working just fine for quite some time... ie it "works for me" as it was. > >>>>>>> > >>>>>>> That said, thanks for maintaining the list, and keeping it going. > >>>>>>> It is a most useful resource. > >>>>>>> > >>>>>>> Chris > >>>>>>> > >>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM > >>>>>>> *From:* "Raimo Niskanen" >>>>>>> > > >>>>>>> *To:* erlang-questions@REDACTED > >>>>>>> *Subject:* Re: Nobody is unsubscribed > >>>>>>> To achieve DMARC compliance we have stopped changing the Subject: > >>>>>>> field and no longer add the mailing list footer to the messages. > >>>>>>> > >>>>>>> This is because From: Subject: and mail body among other fields are > >>>>>>> often DKIM signed, so if we should change them we would not pass DKIM > >>>>>>> signature check and thereby not be DMARC compliant. > >>>>>>> > >>>>>>> Sorry for the inconvenience, we do not make the rules... > >>>>>>> / Raimo Niskanen > >>>>>>> > >>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>>> > wrote: > >>>>>>> > > >>>>>>> > The reason we changed mailing list servers was to get better DMARC and > >>>>>>> > DKIM compliance. This is a test post for us to inspect its headers... > >>>>>>> > -- > >>>>>>> > Raimo Niskanen > >>>>>>> > >>>>>> > >>>> > >>> > >> > >> -- > >> Lo?c Hoguin > >> https://ninenines.eu > > > > -- > Lo?c Hoguin > https://ninenines.eu > From essen@REDACTED Mon Nov 4 22:15:53 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 4 Nov 2019 22:15:53 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191104124442.GA28874@erix.ericsson.se> Message-ID: <4ab31213-0965-ff1f-389c-fe483aecf030@ninenines.eu> Yes it's a requirement for the preload list, but it's not required in the RFC, it's just a SHOULD[1]. So erlang.org can definitely be setup like I mentioned. Not perfect but could be the most appropriate solution considering Raimo wants content to be available via plain HTTP. [1]SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course. On 04/11/2019 21:00, Mark Reynolds wrote: > Using HSTS without a http to https redirection is against the RFC (6797): > >> If an HSTS Host receives an HTTP request message over a non-secure transport, it SHOULD send an HTTP response message containing a status code indicating a permanent redirect, such as status code 301 > > Also, it's a requirement for inculsion into the HSTS preload list: > >> In order to be accepted to the HSTS preload list through this form, your site must satisfy the following set of requirements: > [?] >> 2- Redirect from HTTP to HTTPS on the same host, if you are listening on port 80. > > On Mon, Nov 4, 2019, at 17:30, Lo?c Hoguin wrote: >> On 04/11/2019 13:44, Raimo Niskanen wrote: >>> On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: >>>> For erlang.org itself there's two problems currently: no automatic >>>> redirection from http to https; >>> >>> That seems to be the industry standard now, but I would like content to be >>> accessible without having to use https. >> >> Redirection is generally not great because you get redirected every time >> you go through via http. There's HSTS that gets us one step further by >> telling browsers to remember they have to use HTTPS instead of HTTP, so >> the initial HTTP call isn't made. >> >>> The redirect for http://erlang.org and https://erlang.org goes to >>> $scheme://www.erlang.org, which redirects to https://www.erlang.org. >>> >>> Unfortunately the redirects back from e.g https://www.erlang.org/doc >>> changes to http://erlang.org/doc because https for erlang.org did not work >>> until 10 minutes ago. >> >> And redirection tends to lead to this issues. >> >>> Would it be sufficient to make those redirects from www.erlang.org to >>> erlang.org not change from https to http? >> >> You definitely shouldn't downgrade if possible. I am wondering however >> if you want to leave *browsers* able to access the site via plain HTTP, >> or clients in general (including things like curl for example). A policy >> like HSTS is only used by clients that understand it (so mostly >> browsers) so maybe this is what you want to setup. Browsers would always >> go through HTTPS; other clients would be able to use both HTTP and HTTPS. >> >> Cheers, >> >>> That, and the answer 20 lines down...? >>> >>>> >>>> And this: >>>> >>>> Your connection is not private >>>> This server could not prove that it is erlang.org; its security >>>> certificate is from www2.erlang.org. This may be caused by a >>>> misconfiguration or an attacker intercepting your connection. >>>> >>>> NET::ERR_CERT_COMMON_NAME_INVALID >>>> Subject: www2.erlang.org >>>> >>>> Issuer: DigiCert SHA2 Secure Server CA >>>> >>>> Expires on: Oct 22, 2021 >>>> >>>> Current date: Nov 4, 2019 >>> >>> A new certificate is in place, so this should be fixed. >>> >>> / Raimo >>> >>> >>>> >>>> Keep up the good work. >>>> >>>> On 04/11/2019 11:34, Raimo Niskanen wrote: >>>>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >>>>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? >>>>> >>>>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. >>>>> The recent web server upgrade enabled it for erlang.org as well; >>>>> we are working on it... >>>>> >>>>> Best regards >>>>> / Raimo >>>>> >>>>> >>>>>> >>>>>> Cheers, >>>>>> Adam >>>>>> >>>>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>>>>>> >>>>>>> Yes it does. It applies to all mailing lists. >>>>>>> >>>>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>>>>>> >>>>>>> Best regards >>>>>>> / Raimo Niskanen >>>>>>> >>>>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>>>>>> Does this apply to the EEPS list as well? >>>>>>> >>>>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>>>>>> >>>>>>>> Thanks for doing all of this, regardless. >>>>>>>> >>>>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>>>>>> way that doesn't break some client's "From:" field, subject line, or >>>>>>>> "Reply:" button in some way, but this seems like the least bad option. >>>>>>>> >>>>>>>> I hope my emails make it through to the list now ^_^ >>>>>>>> >>>>>>>> OT: Be careful of organisations' web contact forms which ask for your >>>>>>>> email address. Sometimes their web servers generate an email from the >>>>>>>> form using your email address as the "From:" address, which will break a >>>>>>>> lot of DKIM/DMARC/SPF stuff. >>>>>>>> I know of at least one local authority (council) website in the UK which >>>>>>>> is guilty of this. >>>>>>>> >>>>>>>> - Joe >>>>>>>> >>>>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>>>>>> policies. >>>>>>>>> >>>>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>>>>>> >>>>>>>>> So for some it has worked, some gets an annoying list probe every now >>>>>>>>> and then, some do not get many posts, but the final nail in the coffin >>>>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>>>>>> policy and at the same time told us to get our act together and stop >>>>>>>>> sending "unhygienic e-mail". >>>>>>>>> >>>>>>>>> All the best >>>>>>>>> / Raimo >>>>>>>>> >>>>>>>>> >>>>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>>>> > skrev: >>>>>>>>> >>>>>>>>> Not having the subject contain [erlang-questions] or some other >>>>>>>>> obvious indicator is quite unfortunate. I guess many people were >>>>>>>>> affected by not being DMARC compliant? It seems to have been >>>>>>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>>>>>> >>>>>>>>> That said, thanks for maintaining the list, and keeping it going. >>>>>>>>> It is a most useful resource. >>>>>>>>> >>>>>>>>> Chris >>>>>>>>> >>>>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>>>>>> *From:* "Raimo Niskanen" >>>>>>>> > >>>>>>>>> *To:* erlang-questions@REDACTED >>>>>>>>> *Subject:* Re: Nobody is unsubscribed >>>>>>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>>>>>> field and no longer add the mailing list footer to the messages. >>>>>>>>> >>>>>>>>> This is because From: Subject: and mail body among other fields are >>>>>>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>>>>>> signature check and thereby not be DMARC compliant. >>>>>>>>> >>>>>>>>> Sorry for the inconvenience, we do not make the rules... >>>>>>>>> / Raimo Niskanen >>>>>>>>> >>>>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>>>> > wrote: >>>>>>>>> > >>>>>>>>> > The reason we changed mailing list servers was to get better DMARC and >>>>>>>>> > DKIM compliance. This is a test post for us to inspect its headers... >>>>>>>>> > -- >>>>>>>>> > Raimo Niskanen >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> Lo?c Hoguin >>>> https://ninenines.eu >>> >> >> -- >> Lo?c Hoguin >> https://ninenines.eu >> -- Lo?c Hoguin https://ninenines.eu From okaprinarjaya@REDACTED Tue Nov 5 05:59:20 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 5 Nov 2019 11:59:20 +0700 Subject: Solving the right problem In-Reply-To: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: I recommended you to use Erlang. Erlang can save you from nowadays complex microservice architecture. If you not use Erlang, it will become so so so fckn complex architecture, too many external tools to wired and manage. Erlang is all in one toolbox. You need Redis? Erlang have ETS and or Mnesia, You need Consul (services discovery) in order all of your micro-services able to communicate to each other? Oh come on.. Erlang is distributed by default. you can use rpc:call() or spawn() process in other server, or send message RemoteNodePid ! {Your, Message} ALL tools nowadays are mimicking Erlang's features Pada tanggal Sen, 4 Nov 2019 pukul 19.10 Marthin Laubscher < marthin@REDACTED> menulis: > Hi everyone, > > > > Please pardon my long absences from this awesome (mature) community and my > limited knowledge which is likely outdated as well. I?ve known since 1996 > when I was first told (in confidence by an Ericsson Radio Systems liaison) > about Erlang that it would have to play a role when I eventually get to > implementing the system I?ve been working on designing since 1991. That big > day is drawing near, so now I?d like to reaffirm my high level > understanding of what the language is suited for. > > > > I reckon the problem I?m looking to address is intrinsically concurrent > and if I can design the solution just right I just might be able to avoid > introducing sequential code and choke points to create an dynamic > (learning, responsive to conditions) distributed server capable of using > all or most of its resources for any mixture of trivial, simple, complex > and massive service requests whether it?s coming from a few clients or > billions of them. Essentially as illustrated in the diagram below: > > > > > > I?d like to ask your advice and check some assumptions if I may impose. > > > > 1. Is my conviction that Erlang (and OTP) is ideally if not best > suited to addressing this type of problem objectively justified or likely > based on loyalty and familiarity? > 2. Is my aspiration to scale the same code from one server to > potentially hundreds of thousands of cores spread across the globe > hopelessly romantic or somewhere within the realm of possibility? > 3. Assuming the network remains HTTP/HTTPS based, would Erlang?s inets > module allow the code accepting new requests to interact with and control > load balancing hardware to ensure each such request is served on the best > available server, or will I need custom load balancers of my own for that? > 4. Still assuming HTTP/HTTPS will inets allow me to break up request > processing across multiple processes and threads based on incremental > scanning of the requests themselves? > 5. Are there lessons from previous (or current) successes and/or > failures to achieve similar results to learn from available in the public > domain like maybe from ejabberd or Yaws? (I?m not attempting to reinvent > any wheels or address a general purpose need like Yaws et al. Internet and > web protocols may be involved but I have a singular focus on delivering > user-specific perspectives of a single large dataset to a custom client > app.) > > > > Given my two additional objectives of eventually:- > > 1. ending up with a boringly simple system which elegantly > encapsulates all these complex requirements, and > 2. open-sourcing the entire system once it?s beyond reach of those > with nefarious intentions, > > would anybody like to get involved in helping design and implement this > project or even take the lead on it? > > > > Thank you in advance for your kind consideration. > > > > Warm regards, > > > > Marthin Laubscher > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 64024 bytes Desc: not available URL: From okaprinarjaya@REDACTED Tue Nov 5 06:04:41 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 5 Nov 2019 12:04:41 +0700 Subject: Solving the right problem In-Reply-To: References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: And, if your team feels so desperate to code in Erlang, because of its weird syntax, you can use Elixir to utilise ALL of Erlang's power in a more friendly and easy-to-understand syntax Pada tanggal Sel, 5 Nov 2019 pukul 11.59 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > I recommended you to use Erlang. > > Erlang can save you from nowadays complex microservice architecture. If > you not use Erlang, it will become so so so fckn complex architecture, too > many external tools to wired and manage. > Erlang is all in one toolbox. > You need Redis? Erlang have ETS and or Mnesia, > You need Consul (services discovery) in order all of your micro-services > able to communicate to each other? Oh come on.. Erlang is distributed by > default. you can use rpc:call() or spawn() process in other server, or > send message RemoteNodePid ! {Your, Message} > > ALL tools nowadays are mimicking Erlang's features > > > > Pada tanggal Sen, 4 Nov 2019 pukul 19.10 Marthin Laubscher < > marthin@REDACTED> menulis: > >> Hi everyone, >> >> >> >> Please pardon my long absences from this awesome (mature) community and >> my limited knowledge which is likely outdated as well. I?ve known since >> 1996 when I was first told (in confidence by an Ericsson Radio Systems >> liaison) about Erlang that it would have to play a role when I eventually >> get to implementing the system I?ve been working on designing since 1991. >> That big day is drawing near, so now I?d like to reaffirm my high level >> understanding of what the language is suited for. >> >> >> >> I reckon the problem I?m looking to address is intrinsically concurrent >> and if I can design the solution just right I just might be able to avoid >> introducing sequential code and choke points to create an dynamic >> (learning, responsive to conditions) distributed server capable of using >> all or most of its resources for any mixture of trivial, simple, complex >> and massive service requests whether it?s coming from a few clients or >> billions of them. Essentially as illustrated in the diagram below: >> >> >> >> >> >> I?d like to ask your advice and check some assumptions if I may impose. >> >> >> >> 1. Is my conviction that Erlang (and OTP) is ideally if not best >> suited to addressing this type of problem objectively justified or likely >> based on loyalty and familiarity? >> 2. Is my aspiration to scale the same code from one server to >> potentially hundreds of thousands of cores spread across the globe >> hopelessly romantic or somewhere within the realm of possibility? >> 3. Assuming the network remains HTTP/HTTPS based, would Erlang?s >> inets module allow the code accepting new requests to interact with and >> control load balancing hardware to ensure each such request is served on >> the best available server, or will I need custom load balancers of my own >> for that? >> 4. Still assuming HTTP/HTTPS will inets allow me to break up request >> processing across multiple processes and threads based on incremental >> scanning of the requests themselves? >> 5. Are there lessons from previous (or current) successes and/or >> failures to achieve similar results to learn from available in the public >> domain like maybe from ejabberd or Yaws? (I?m not attempting to reinvent >> any wheels or address a general purpose need like Yaws et al. Internet and >> web protocols may be involved but I have a singular focus on delivering >> user-specific perspectives of a single large dataset to a custom client >> app.) >> >> >> >> Given my two additional objectives of eventually:- >> >> 1. ending up with a boringly simple system which elegantly >> encapsulates all these complex requirements, and >> 2. open-sourcing the entire system once it?s beyond reach of those >> with nefarious intentions, >> >> would anybody like to get involved in helping design and implement this >> project or even take the lead on it? >> >> >> >> Thank you in advance for your kind consideration. >> >> >> >> Warm regards, >> >> >> >> Marthin Laubscher >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 64024 bytes Desc: not available URL: From marthin@REDACTED Tue Nov 5 14:59:00 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Tue, 05 Nov 2019 15:59:00 +0200 Subject: Solving the right problem In-Reply-To: References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: Thank you. Yes, Erlang has brilliant building blocks, as far as I can tell, but that is the problem ? it?s only as far as I can tell. I?ve never had a chance to test Erlang?s limits. I cannot expect people who built their business? core competencies around having learnt about those limits the hard (expensive way) to give that away for free, so I?d be grateful for any pointers and insights I can get to inform my design decisions. From: I Gusti Ngurah Oka Prinarjaya Date: Tuesday, 5 November, 2019 at 07:00 To: "marthin@REDACTED" Cc: Erlang Subject: Re: Solving the right problem I recommended you to use Erlang. Erlang can save you from nowadays complex microservice architecture. If you not use Erlang, it will become so so so fckn complex architecture, too many external tools to wired and manage. Erlang is all in one toolbox. You need Redis? Erlang have ETS and or Mnesia, You need Consul (services discovery) in order all of your micro-services able to communicate to each other? Oh come on.. Erlang is distributed by default. you can use rpc:call() or spawn() process in other server, or send message RemoteNodePid ! {Your, Message} ALL tools nowadays are mimicking Erlang's features Pada tanggal Sen, 4 Nov 2019 pukul 19.10 Marthin Laubscher menulis: Hi everyone, Please pardon my long absences from this awesome (mature) community and my limited knowledge which is likely outdated as well. I?ve known since 1996 when I was first told (in confidence by an Ericsson Radio Systems liaison) about Erlang that it would have to play a role when I eventually get to implementing the system I?ve been working on designing since 1991. That big day is drawing near, so now I?d like to reaffirm my high level understanding of what the language is suited for. I reckon the problem I?m looking to address is intrinsically concurrent and if I can design the solution just right I just might be able to avoid introducing sequential code and choke points to create an dynamic (learning, responsive to conditions) distributed server capable of using all or most of its resources for any mixture of trivial, simple, complex and massive service requests whether it?s coming from a few clients or billions of them. Essentially as illustrated in the diagram below: I?d like to ask your advice and check some assumptions if I may impose. Is my conviction that Erlang (and OTP) is ideally if not best suited to addressing this type of problem objectively justified or likely based on loyalty and familiarity? Is my aspiration to scale the same code from one server to potentially hundreds of thousands of cores spread across the globe hopelessly romantic or somewhere within the realm of possibility? Assuming the network remains HTTP/HTTPS based, would Erlang?s inets module allow the code accepting new requests to interact with and control load balancing hardware to ensure each such request is served on the best available server, or will I need custom load balancers of my own for that? Still assuming HTTP/HTTPS will inets allow me to break up request processing across multiple processes and threads based on incremental scanning of the requests themselves? Are there lessons from previous (or current) successes and/or failures to achieve similar results to learn from available in the public domain like maybe from ejabberd or Yaws? (I?m not attempting to reinvent any wheels or address a general purpose need like Yaws et al. Internet and web protocols may be involved but I have a singular focus on delivering user-specific perspectives of a single large dataset to a custom client app.) Given my two additional objectives of eventually:- ending up with a boringly simple system which elegantly encapsulates all these complex requirements, and open-sourcing the entire system once it?s beyond reach of those with nefarious intentions, would anybody like to get involved in helping design and implement this project or even take the lead on it? Thank you in advance for your kind consideration. Warm regards, Marthin Laubscher -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 64025 bytes Desc: not available URL: From by@REDACTED Tue Nov 5 15:08:26 2019 From: by@REDACTED (by) Date: Tue, 5 Nov 2019 22:08:26 +0800 Subject: Configure Emacs for Erlang development In-Reply-To: References: <844bd1a8-4507-4ec3-872c-4db301c7c353@www.fastmail.com> Message-ID: I got it! Here is my working Emacs configuration for Erlang development: ==== ;; for delight (use-package delight :ensure t) ;; for flycheck (use-package flycheck :ensure t :delight :config (global-flycheck-mode)) ;; for ivy-erlang-complete (use-package ivy-erlang-complete :ensure t) ;; for company-erlang: auto complete (add-hook 'erlang-mode-hook #'company-erlang-init) ;; for erlang (use-package erlang :load-path (?//lib/erlang/lib/tools-3.2.1/emacs") :hook (after-save . ivy-erlang-complete-reparse) :custom (ivy-erlang-complete-erlang-root ?//lib/erlang") :config (ivy-erlang-complete-init) :mode (("\\.erl?$" . erlang-mode) ("rebar\\.config$" . erlang-mode) ("relx\\.config$" . erlang-mode) ("sys\\.config\\.src$" . erlang-mode) ("sys\\.config$" . erlang-mode) ("\\.config\\.src?$" . erlang-mode) ("\\.config\\.script?$" . erlang-mode) ("\\.hrl?$" . erlang-mode) ("\\.app?$" . erlang-mode) ("\\.app.src?$" . erlang-mode) ("\\Emakefile" . erlang-mode))) ==== The trick is put ivy-erlang-complete after flycheck. A sequential issue :-) The auto completion works after I use gsed(GNU sed) instead of default sed(BSD sed) on macOS. Yao > ? 2019?11?4??23:44?by ??? > > Yes, same configuration may behave differently because of different package versions. > > A working package does not mean it will keep working in future updates. The documentation and change log become significant in this case. > > Yao > >> ? 2019?11?4??23:16?Tristan Sloughter ??? >> >> ? >> Hey, that is from my config in adoptingerlang but I don't have it currently separated out and published. Though I did do that at one point to test this and hopefully can find that and I will upload that file and link to it from adoptingerlang. >> >> I don't have a good answer for you right now, sadly when I see errors like that I tend to just keep refershing the Emacs and package registry and updating the packages... sometimes having to uninstall and reinstall packages and eventually it works. >> >> Emacs package management is not the best.. hehe, as far as I know you can't specify a version of a package and can't lock dependencies. >> >> Tristan >> >> On Mon, Nov 4, 2019, at 06:30, by wrote: >>> Hi, >>> >>> I followed the guide with link (From Adopting Erlang): https://adoptingerlang.org/docs/development/setup/ >>> Seems "ivy-erlang-complete" is not properly configured. >>> >>> Here is some environment information: >>> Erlang version: Erlang/OTP 22.1.4 >>> Emacs version: GNU Emacs 25.3.1 >>> >>> And here is my related ~/.emacs configuration for Erlang: >>> ==== >>> ;; for ivy-erlang-complete >>> (use-package ivy-erlang-complete >>> :ensure t) >>> >>> ;; for delight >>> (use-package delight >>> :ensure t) >>> >>> ;; for flycheck >>> (use-package flycheck >>> :ensure t >>> :delight >>> :config (global-flycheck-mode)) >>> >>> ;; for erlang >>> (use-package erlang >>> :load-path (?//lib/erlang/lib/tools-3.2.1/emacs") >>> :hook (after-save . ivy-erlang-complete-reparse) >>> :custom (ivy-erlang-complete-erlang-root "//lib/erlang") >>> :config (ivy-erlang-complete-init) >>> :mode (("\\.erl?$" . erlang-mode) >>> ("rebar\\.config$" . erlang-mode) >>> ("relx\\.config$" . erlang-mode) >>> ("sys\\.config\\.src$" . erlang-mode) >>> ("sys\\.config$" . erlang-mode) >>> ("\\.config\\.src?$" . erlang-mode) >>> ("\\.config\\.script?$" . erlang-mode) >>> ("\\.hrl?$" . erlang-mode) >>> ("\\.app?$" . erlang-mode) >>> ("\\.app.src?$" . erlang-mode) >>> ("\\Emakefile" . erlang-mode))) >>> ==== >>> >>> When I restart my emacs with above configuration, I got error message as below: >>> ==== >>> Error (use-package): erlang/:config: Symbol?s function definition is void: ivy-erlang-complete-setup-flycheck >>> ==== >>> I have no clue about what happened. >>> >>> An example ~/.emacs file related to Erlang would be very helpful. >>> >>> I searched it with Google, but seems no typical configuration demo/example exist for configuring Emacs with Erlang. >>> >>> By the way, I want to ask: what kind of editor configuration do you use to develop Erlang every day? >>> >>> Yao >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Nov 5 16:01:32 2019 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 5 Nov 2019 16:01:32 +0100 Subject: Solving the right problem In-Reply-To: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: Re. 3, you should definitely look into using existing solutions for HTTP/HTTPS load-balancing. This will work every bit as well with Erlang/inets as with any other technology. Re. 4, yes, and you're not limited to inets. Take a look e.g. at Cowboy [1] Re. 5, well, your description is vague enough that it's hard to answer, but you seem to be aiming for some form of SOA architecture. If you want to proceed quickly with prototyping and MVPs, you could implement a component architecture inside a single Erlang node and make some minimal preparations for being able to later break them apart into a larger network of services. A single Erlang node running on a decent cloud instance is likely to handle a fairly large number of devices without breaking a sweat, unless your applications are expected to be very computationally heavy. This way, you can defer the many messy issues of going full SOA from the beginning, and benefit from Erlang's outstanding "micro-SOA" capabilities. [1] https://github.com/ninenines/cowboy Den m?n 4 nov. 2019 kl 13:10 skrev Marthin Laubscher < marthin@REDACTED>: > Hi everyone, > > > > Please pardon my long absences from this awesome (mature) community and my > limited knowledge which is likely outdated as well. I?ve known since 1996 > when I was first told (in confidence by an Ericsson Radio Systems liaison) > about Erlang that it would have to play a role when I eventually get to > implementing the system I?ve been working on designing since 1991. That big > day is drawing near, so now I?d like to reaffirm my high level > understanding of what the language is suited for. > > > > I reckon the problem I?m looking to address is intrinsically concurrent > and if I can design the solution just right I just might be able to avoid > introducing sequential code and choke points to create an dynamic > (learning, responsive to conditions) distributed server capable of using > all or most of its resources for any mixture of trivial, simple, complex > and massive service requests whether it?s coming from a few clients or > billions of them. Essentially as illustrated in the diagram below: > > > > > > I?d like to ask your advice and check some assumptions if I may impose. > > > > 1. Is my conviction that Erlang (and OTP) is ideally if not best > suited to addressing this type of problem objectively justified or likely > based on loyalty and familiarity? > 2. Is my aspiration to scale the same code from one server to > potentially hundreds of thousands of cores spread across the globe > hopelessly romantic or somewhere within the realm of possibility? > 3. Assuming the network remains HTTP/HTTPS based, would Erlang?s inets > module allow the code accepting new requests to interact with and control > load balancing hardware to ensure each such request is served on the best > available server, or will I need custom load balancers of my own for that? > 4. Still assuming HTTP/HTTPS will inets allow me to break up request > processing across multiple processes and threads based on incremental > scanning of the requests themselves? > 5. Are there lessons from previous (or current) successes and/or > failures to achieve similar results to learn from available in the public > domain like maybe from ejabberd or Yaws? (I?m not attempting to reinvent > any wheels or address a general purpose need like Yaws et al. Internet and > web protocols may be involved but I have a singular focus on delivering > user-specific perspectives of a single large dataset to a custom client > app.) > > > > Given my two additional objectives of eventually:- > > 1. ending up with a boringly simple system which elegantly > encapsulates all these complex requirements, and > 2. open-sourcing the entire system once it?s beyond reach of those > with nefarious intentions, > > would anybody like to get involved in helping design and implement this > project or even take the lead on it? > > > > Thank you in advance for your kind consideration. > > > > Warm regards, > > > > Marthin Laubscher > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 64024 bytes Desc: not available URL: From raimo+erlang-questions@REDACTED Tue Nov 5 16:46:49 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 5 Nov 2019 16:46:49 +0100 Subject: Nobody is unsubscribed In-Reply-To: <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> Message-ID: <20191105154649.GA20777@erix.ericsson.se> Certificates are now in place, and redirects are "working", so if you go to http://erlang.org you end up at https://www.erlang.org/. Remains that many links back from https://www.erlang.org to erlang.org downgrade to http:. We have not (yet) implemented HTTP Strict Transport Security (HSTS) on erlang.org or any of its subdomains. Will that be frowned upon? If not I think we ore done for now (apart from hunting down all bad links mentioned above). Thank you for your feedback! / Raimo Niskanen On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: > For erlang.org itself there's two problems currently: no automatic > redirection from http to https; > > And this: > > Your connection is not private > This server could not prove that it is erlang.org; its security > certificate is from www2.erlang.org. This may be caused by a > misconfiguration or an attacker intercepting your connection. > > NET::ERR_CERT_COMMON_NAME_INVALID > Subject: www2.erlang.org > > Issuer: DigiCert SHA2 Secure Server CA > > Expires on: Oct 22, 2021 > > Current date: Nov 4, 2019 > > Keep up the good work. > > On 04/11/2019 11:34, Raimo Niskanen wrote: > > On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: > >> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? > > > > HTTPS has been active for www.erlang.org and bugs.erlang.org for years. > > The recent web server upgrade enabled it for erlang.org as well; > > we are working on it... > > > > Best regards > > / Raimo > > > > > >> > >> Cheers, > >> Adam > >> > >>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > >>> > >>> Yes it does. It applies to all mailing lists. > >>> > >>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > >>> > >>> Best regards > >>> / Raimo Niskanen > >>> > >>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > >>> Does this apply to the EEPS list as well? > >>> > >>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > >>>> > >>>> Thanks for doing all of this, regardless. > >>>> > >>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > >>>> way that doesn't break some client's "From:" field, subject line, or > >>>> "Reply:" button in some way, but this seems like the least bad option. > >>>> > >>>> I hope my emails make it through to the list now ^_^ > >>>> > >>>> OT: Be careful of organisations' web contact forms which ask for your > >>>> email address. Sometimes their web servers generate an email from the > >>>> form using your email address as the "From:" address, which will break a > >>>> lot of DKIM/DMARC/SPF stuff. > >>>> I know of at least one local authority (council) website in the UK which > >>>> is guilty of this. > >>>> > >>>> - Joe > >>>> > >>>> On 26/10/2019 07:57, Raimo Niskanen wrote: > >>>>> It is mainly "the big ones" that have been affected by stricter DMARC > >>>>> policies. > >>>>> > >>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then > >>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice > >>>>> versa). So the list gets a bounce and eventually blocks the Gmail > >>>>> subscriber, if enough in a row happens to send with strict DMARC policies. > >>>>> > >>>>> So for some it has worked, some gets an annoying list probe every now > >>>>> and then, some do not get many posts, but the final nail in the coffin > >>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > >>>>> policy and at the same time told us to get our act together and stop > >>>>> sending "unhygienic e-mail". > >>>>> > >>>>> All the best > >>>>> / Raimo > >>>>> > >>>>> > >>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>> > skrev: > >>>>> > >>>>> Not having the subject contain [erlang-questions] or some other > >>>>> obvious indicator is quite unfortunate. I guess many people were > >>>>> affected by not being DMARC compliant? It seems to have been > >>>>> working just fine for quite some time... ie it "works for me" as it was. > >>>>> > >>>>> That said, thanks for maintaining the list, and keeping it going. > >>>>> It is a most useful resource. > >>>>> > >>>>> Chris > >>>>> > >>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM > >>>>> *From:* "Raimo Niskanen" >>>>> > > >>>>> *To:* erlang-questions@REDACTED > >>>>> *Subject:* Re: Nobody is unsubscribed > >>>>> To achieve DMARC compliance we have stopped changing the Subject: > >>>>> field and no longer add the mailing list footer to the messages. > >>>>> > >>>>> This is because From: Subject: and mail body among other fields are > >>>>> often DKIM signed, so if we should change them we would not pass DKIM > >>>>> signature check and thereby not be DMARC compliant. > >>>>> > >>>>> Sorry for the inconvenience, we do not make the rules... > >>>>> / Raimo Niskanen > >>>>> > >>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>> > wrote: > >>>>> > > >>>>> > The reason we changed mailing list servers was to get better DMARC and > >>>>> > DKIM compliance. This is a test post for us to inspect its headers... > >>>>> > -- > >>>>> > Raimo Niskanen > >>>>> > >>>> > >> > > > > -- > Lo?c Hoguin > https://ninenines.eu -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From by@REDACTED Tue Nov 5 17:09:33 2019 From: by@REDACTED (by) Date: Wed, 6 Nov 2019 00:09:33 +0800 Subject: Nobody is unsubscribed In-Reply-To: <20191105154649.GA20777@erix.ericsson.se> References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191105154649.GA20777@erix.ericsson.se> Message-ID: Hi, What about making link https://www.erlang.org/mailman/listinfo/erlang-questions work? For now, link https://www2.erlang.org/mailman/listinfo/erlang-questions works. When a new Erlang user try to subscribe to this mailing list, he/she will input some information through http if link http://erlang.org/mailman/listinfo/erlang-questions is the final target. Yao > ? 2019?11?5??23:46?Raimo Niskanen ??? > > Certificates are now in place, and redirects are "working", so if you go to > http://erlang.org you end up at https://www.erlang.org/. > > Remains that many links back from https://www.erlang.org to erlang.org > downgrade to http:. > > We have not (yet) implemented HTTP Strict Transport Security (HSTS) > on erlang.org or any of its subdomains. > > Will that be frowned upon? > > If not I think we ore done for now (apart from hunting down all bad links > mentioned above). > > Thank you for your feedback! > / Raimo Niskanen > > > On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: >> For erlang.org itself there's two problems currently: no automatic >> redirection from http to https; >> >> And this: >> >> Your connection is not private >> This server could not prove that it is erlang.org; its security >> certificate is from www2.erlang.org. This may be caused by a >> misconfiguration or an attacker intercepting your connection. >> >> NET::ERR_CERT_COMMON_NAME_INVALID >> Subject: www2.erlang.org >> >> Issuer: DigiCert SHA2 Secure Server CA >> >> Expires on: Oct 22, 2021 >> >> Current date: Nov 4, 2019 >> >> Keep up the good work. >> >> On 04/11/2019 11:34, Raimo Niskanen wrote: >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: >>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? >>> >>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. >>> The recent web server upgrade enabled it for erlang.org as well; >>> we are working on it... >>> >>> Best regards >>> / Raimo >>> >>> >>>> >>>> Cheers, >>>> Adam >>>> >>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: >>>>> >>>>> Yes it does. It applies to all mailing lists. >>>>> >>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. >>>>> >>>>> Best regards >>>>> / Raimo Niskanen >>>>> >>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: >>>>> Does this apply to the EEPS list as well? >>>>> >>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: >>>>>> >>>>>> Thanks for doing all of this, regardless. >>>>>> >>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant >>>>>> way that doesn't break some client's "From:" field, subject line, or >>>>>> "Reply:" button in some way, but this seems like the least bad option. >>>>>> >>>>>> I hope my emails make it through to the list now ^_^ >>>>>> >>>>>> OT: Be careful of organisations' web contact forms which ask for your >>>>>> email address. Sometimes their web servers generate an email from the >>>>>> form using your email address as the "From:" address, which will break a >>>>>> lot of DKIM/DMARC/SPF stuff. >>>>>> I know of at least one local authority (council) website in the UK which >>>>>> is guilty of this. >>>>>> >>>>>> - Joe >>>>>> >>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: >>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC >>>>>>> policies. >>>>>>> >>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then >>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice >>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail >>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. >>>>>>> >>>>>>> So for some it has worked, some gets an annoying list probe every now >>>>>>> and then, some do not get many posts, but the final nail in the coffin >>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC >>>>>>> policy and at the same time told us to get our act together and stop >>>>>>> sending "unhygienic e-mail". >>>>>>> >>>>>>> All the best >>>>>>> / Raimo >>>>>>> >>>>>>> >>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>> > skrev: >>>>>>> >>>>>>> Not having the subject contain [erlang-questions] or some other >>>>>>> obvious indicator is quite unfortunate. I guess many people were >>>>>>> affected by not being DMARC compliant? It seems to have been >>>>>>> working just fine for quite some time... ie it "works for me" as it was. >>>>>>> >>>>>>> That said, thanks for maintaining the list, and keeping it going. >>>>>>> It is a most useful resource. >>>>>>> >>>>>>> Chris >>>>>>> >>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM >>>>>>> *From:* "Raimo Niskanen" >>>>>> > >>>>>>> *To:* erlang-questions@REDACTED >>>>>>> *Subject:* Re: Nobody is unsubscribed >>>>>>> To achieve DMARC compliance we have stopped changing the Subject: >>>>>>> field and no longer add the mailing list footer to the messages. >>>>>>> >>>>>>> This is because From: Subject: and mail body among other fields are >>>>>>> often DKIM signed, so if we should change them we would not pass DKIM >>>>>>> signature check and thereby not be DMARC compliant. >>>>>>> >>>>>>> Sorry for the inconvenience, we do not make the rules... >>>>>>> / Raimo Niskanen >>>>>>> >>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>> > wrote: >>>>>>>> >>>>>>>> The reason we changed mailing list servers was to get better DMARC and >>>>>>>> DKIM compliance. This is a test post for us to inspect its headers... >>>>>>>> -- >>>>>>>> Raimo Niskanen >>>>>>> >>>>>> >>>> >>> >> >> -- >> Lo?c Hoguin >> https://ninenines.eu > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Wed Nov 6 00:15:08 2019 From: james@REDACTED (james) Date: Tue, 5 Nov 2019 23:15:08 +0000 Subject: Solving the right problem In-Reply-To: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: <9f1a5343-2759-ddee-ae0d-aa2ace3d887b@mansionfamily.plus.com> My question would be - why HTTP/HTTPS? I know 'everybody loves stateless' but actually that's not always a good model. If you were to design based on gRPC and drop the connection when the user is not actively working, how many concurrent connections will you have, and how much latency can you wear? From g@REDACTED Wed Nov 6 02:04:33 2019 From: g@REDACTED (Guilherme Andrade) Date: Wed, 6 Nov 2019 01:04:33 +0000 Subject: [ANN] locus: Geolocation and ASN lookup of IP addresses In-Reply-To: References: Message-ID: Hello list, Locus 1.8.0, a geolocation library built on top of MaxMind GeoLite2, was released today. Added: - support for returning types other than map upon successful lookups Changed: - MMDB decoder, which was split into separate tree, data section and analysis modules - imported `stacktrace_compat` version [1.0.2 => 1.1.1] Removed: - support for OTP 18 Fixed: - incidents of `locus` managerial processes keeping references to old binaries, upon a database update, for a potentially unlimited time (OTP 20+ only) - broken logging of playground shell on OTP 21.1+ * Hex package: https://hex.pm/packages/locus/1.8.0 * Documentation: https://hexdocs.pm/locus/1.8.0/ * Source code (GitHub): https://github.com/g-andrade/locus/tree/1.8.0 * Source code (GitLab): https://gitlab.com/g-andrade/locus/tree/1.8.0 -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From marthin@REDACTED Wed Nov 6 09:18:04 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Wed, 06 Nov 2019 10:18:04 +0200 Subject: [Original] Re: Solving the right problem In-Reply-To: <9f1a5343-2759-ddee-ae0d-aa2ace3d887b@mansionfamily.plus.com> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> <9f1a5343-2759-ddee-ae0d-aa2ace3d887b@mansionfamily.plus.com> Message-ID: <0B4AD10F-C097-4B40-9AA3-1D8A4D544C24@lobeshare.co.za> Hi James, Nice questions. Thank you. HTTP(S) was assumed for historical reasons which might no longer be valid. Frankly, it was probably more about the ports which were the first to be opened for outgoing traffic by panic-struck workplaces, and the level of packet inspection that accompanied that than the specific properties of the protocol. Neither the problem nor the solutions I foresee are actually stateless by nature, that much is true. Tracking cursor states in the server allows for much more intelligent choices and proactive actions to ensure that each next bit of data is delivered to the client via the most opportune path from where it is mastered, cached or calculated. Having said that, I'd think maintaining sessions too low down the network stack might at some point start working against the type of cursor mobility I wish to have. It remains to be seen. Ideally the time it takes to set-up and tear down network level connections between individual servers and individual clients can also be factored into how the work is distributed across nodes with available computing and network capacity. Network latency is critically important but only from the perspective of how it impacts on user experience. The project's design approach aims to address this by progressively rolling out and using servers that are physically close (in network terms) to the clients they serve and then using smart pre-fetch logic to get the data there before the user needs it. It's impossible to make up for network latency using faster processing, but people take several orders of magnitude longer to consume a chunk of data than computers and that create the opportunity to "catch-up" on latency. I'd love to discuss alternatives if you'd like to offer some? Making the most of ready-made components is crucial, but not if they have made conflicting design decisions which end up keeping the project overall from attaining its goals. Marthin ?On 2019/11/06, 01:16, "james" wrote: My question would be - why HTTP/HTTPS? I know 'everybody loves stateless' but actually that's not always a good model. If you were to design based on gRPC and drop the connection when the user is not actively working, how many concurrent connections will you have, and how much latency can you wear? From tty.erlang@REDACTED Wed Nov 6 10:02:49 2019 From: tty.erlang@REDACTED (T Ty) Date: Wed, 6 Nov 2019 09:02:49 +0000 Subject: [Original] Re: Solving the right problem In-Reply-To: <0B4AD10F-C097-4B40-9AA3-1D8A4D544C24@lobeshare.co.za> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> <9f1a5343-2759-ddee-ae0d-aa2ace3d887b@mansionfamily.plus.com> <0B4AD10F-C097-4B40-9AA3-1D8A4D544C24@lobeshare.co.za> Message-ID: I would suggest you ignore the Erlang/Elixir mode of thought as a starting point. Your architecture is closely related to either XMPP or AMQP protocol and suggest you base it off these instead of heading straight down do a coding language. Of course I would be bias towards MongooseIM or RabbitMQ. This abstracts the entire communication question away and you can focus on the business logic. In XMPP, each client device and server would be XMPP clients and both would communicate using simple text based coded messages. You automatically get federation and presence notification. Store and forward is automatically built in. XMPP is also moving towards addressing IoT devices which might further simplify your architecture. In RabbitMQ each client/server would have a two queue (in/out) and all communications goes through these. Multiple clients can share a single queue pair if they are grouped together e.g. all clients within the same geo-location. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From marthin@REDACTED Wed Nov 6 10:49:22 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Wed, 06 Nov 2019 11:49:22 +0200 Subject: Solving the right problem In-Reply-To: References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: <40E2972A-6957-4C93-ACD3-E34687DEDB16@lobeshare.co.za> Thank you Ulf, I?m hesitant to label what I?m going for here with loaded terms such as SOA architecture or ?micro-SOA?. I?ve found in the many years I?ve worked in telco?s as a Systems Architect or Design Authority that we do better when we don?t overplay architectural choices as if they are goals in themselves or even as reusable solutions. Yes, for trivial problems with trivial solutions where the cost of a bespoke architecture would never be warranted it makes sense to squeeze the solution into an architecture that there or there about and deal with the little things that gets in the way as they come up. Been there, done that too. When, by contrast, the project?s value proposition is such that the implications on cost, TTM or otherwise, of a custom architecture is warranted by the benefits to viability, sustainability and profitability, then using a ?best fit? ready-made architecture tends to lead not to minor issues to overcome, but impedance mismatch level issues to contend with when one can least afford those ? while scaling and rolling out. I?m hoping to leverage my extremely narrow problem domain to arrive at the simplest possible design rules and principles which will achieve the time to market, performance and scalability I?m aiming for. If the result turns out to be applicable to more people?s problem domains than my own that would be cool, but I?m obliged to prioritise my own project above all else. I?m sorry about the vagueness in my question. I will endeavor to clarify more as questions like yours come up. The project I?m working aims to have massive impact from a simple design. At this juncture I should describe it as a giant global key-value store where the keys have many dimensions including space and time and payload which, if actually stored in a key-value store, would usually be a fraction of the size of the composite key. Therefore, and because the data is intrinsically relational (i.e. everything is related to everything either directly or indirectly, but yes, Relational as in the 5th Normal Form too), these intrinsic properties makes true key-value store often sub-optimal. The idea is to have the client request data based on a cursor or user/session specific context. Each client would maintain a logical connection with its closest (network wise) server where it makes the requests. The request is load-balanced at that server address to a host with available capacity and knowledge of the context. ?At that host, the request gets interpreted/processed by combining a) the context in which the request was made, such as 3D viewpoint and zoom level, b) the request itself, i.e. what data is being requested, c) how and where the raw or derived data can best be obtained from. ?This I?d hope to determine both recursively and without sequential code bottlenecks by splitting up the request and context matching across as many nodes as are required and available. The same then goes to how the request is fulfilled. By the client including data in the request about the user?s apparent momentum the matching process is also capable of anticipating what data and/or results might be requested next. Up to 100% of available resources may be spent on such pre-fetching, but it may not hold back the current request nor may it demand resources required for other priority requests. Simply put, every core in every data center and every provisioned network link should run at 100% all the time doing the most opportunistic work possible to provide each user fair and maximal use of the resources. To implement that simply and in a manner that can evolve naturally, I was hoping to find the ideal place in code terms for a function which takes a ?parsed? request and cursor data structure as parameters and returns a result structure which will be filled in in due course (not a completed result). The returned result would be handed back to an appointed client handler which would compile and send results back to the client as and when it comes in from wherever it had to be fetched. The client itself would be aware of the fact that not all data is present yet when it gets results, and would set up to be ?subscribed? to data arriving later. I know this is still very vague and I am sorry about that. But maybe there is enough in there to kick off the next level discussion about what I asked in the first place, which is to learn about past and/or present experiences using erlang to solve similar problems and what worked out well vs what didn?t. Thanks, Marthin From: Ulf Wiger Date: Tuesday, 5 November, 2019 at 17:03 To: "marthin@REDACTED" Cc: erlang-questions Subject: Re: Solving the right problem Re. 3, you should definitely look into using existing solutions for HTTP/HTTPS load-balancing. This will work every bit as well with Erlang/inets as with any other technology. Re. 4, yes, and you're not limited to inets. Take a look e.g. at Cowboy [1] Re. 5, well, your description is vague enough that it's hard to answer, but you seem to be aiming for some form of SOA architecture. If you want to proceed quickly with prototyping and MVPs, you could implement a component architecture inside a single Erlang node and make some minimal preparations for being able to later break them apart into a larger network of services. A single Erlang node running on a decent cloud instance is likely to handle a fairly large number of devices without breaking a sweat, unless your applications are expected to be very computationally heavy. This way, you can defer the many messy issues of going full SOA from the beginning, and benefit from Erlang's outstanding "micro-SOA" capabilities. [1] https://github.com/ninenines/cowboy Den m?n 4 nov. 2019 kl 13:10 skrev Marthin Laubscher : Hi everyone, Please pardon my long absences from this awesome (mature) community and my limited knowledge which is likely outdated as well. I?ve known since 1996 when I was first told (in confidence by an Ericsson Radio Systems liaison) about Erlang that it would have to play a role when I eventually get to implementing the system I?ve been working on designing since 1991. That big day is drawing near, so now I?d like to reaffirm my high level understanding of what the language is suited for. I reckon the problem I?m looking to address is intrinsically concurrent and if I can design the solution just right I just might be able to avoid introducing sequential code and choke points to create an dynamic (learning, responsive to conditions) distributed server capable of using all or most of its resources for any mixture of trivial, simple, complex and massive service requests whether it?s coming from a few clients or billions of them. Essentially as illustrated in the diagram below: I?d like to ask your advice and check some assumptions if I may impose. Is my conviction that Erlang (and OTP) is ideally if not best suited to addressing this type of problem objectively justified or likely based on loyalty and familiarity? Is my aspiration to scale the same code from one server to potentially hundreds of thousands of cores spread across the globe hopelessly romantic or somewhere within the realm of possibility? Assuming the network remains HTTP/HTTPS based, would Erlang?s inets module allow the code accepting new requests to interact with and control load balancing hardware to ensure each such request is served on the best available server, or will I need custom load balancers of my own for that? Still assuming HTTP/HTTPS will inets allow me to break up request processing across multiple processes and threads based on incremental scanning of the requests themselves? Are there lessons from previous (or current) successes and/or failures to achieve similar results to learn from available in the public domain like maybe from ejabberd or Yaws? (I?m not attempting to reinvent any wheels or address a general purpose need like Yaws et al. Internet and web protocols may be involved but I have a singular focus on delivering user-specific perspectives of a single large dataset to a custom client app.) Given my two additional objectives of eventually:- ending up with a boringly simple system which elegantly encapsulates all these complex requirements, and open-sourcing the entire system once it?s beyond reach of those with nefarious intentions, would anybody like to get involved in helping design and implement this project or even take the lead on it? Thank you in advance for your kind consideration. Warm regards, Marthin Laubscher -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 64025 bytes Desc: not available URL: From oliver.bollmann@REDACTED Wed Nov 6 10:58:22 2019 From: oliver.bollmann@REDACTED (Oliver Bollmann) Date: Wed, 6 Nov 2019 10:58:22 +0100 Subject: Handshake -> psk_key_exchange_modes Message-ID: <43916d50-7857-1055-3969-c9bc8ff24fb7@t-online.de> Hi, 0) Erlang/OTP 22 [erts-10.5.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [hipe] 1) client_server:start(). -> Port 2) Open browser: https://localhost:Port using Safari,Chrome 3) exception error: no function clause matching ssl_handshake:extension_value({psk_key_exchange_modes,[psk_dhe_ke]}) (ssl_handshake.erl, line 1492) ???? in function? maps:map_1/2 (maps.erl, line 252) ???? in call from maps:map_1/2 (maps.erl, line 252) ???? in call from maps:map/2 (maps.erl, line 243) ???? in call from ssl_connection:handshake/2 (ssl_connection.erl, line 127) ???? in call from client_server:start/0 (client_server.erl, line 42) Any Hints? -- Gr??e Oliver Bollmann -------------- next part -------------- %% %% %CopyrightBegin% %% %% Copyright Ericsson AB 2003-2018. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. %% You may obtain a copy of the License at %% %% http://www.apache.org/licenses/LICENSE-2.0 %% %% Unless required by applicable law or agreed to in writing, software %% distributed under the License is distributed on an "AS IS" BASIS, %% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. %% See the License for the specific language governing permissions and %% limitations under the License. %% %% %CopyrightEnd% %% %%% Purpose: Example of SSL client and server using example certificates. -module(client_server). -export([start/0, init_connect/1]). start() -> %% Start ssl application {ok, StartedApps} = application:ensure_all_started(ssl), %% Let the current process be the server that listens and accepts %% Listen {ok, LSock} = ssl:listen(0, mk_opts(listen)), {ok, {_, LPort}} = ssl:sockname(LSock), io:fwrite("Listen: port = ~w.~n", [LPort]), %% Spawn the client process that connects to the server %spawn(?MODULE, init_connect, [LPort]), %% Accept {ok, ASock} = ssl:transport_accept(LSock), {ok, HsSocket, _Ex} = ssl:handshake(ASock), {ok, SslSocket} = ssl:handshake_continue(HsSocket, mk_opts(listen)), io:fwrite("Accept: accepted.~n"), {ok, Cert} = ssl:peercert(SslSocket), io:fwrite("Accept: peer cert:~n~p~n", [public_key:pkix_decode_cert(Cert, otp)]), io:fwrite("Accept: sending \"hello\".~n"), ssl:send(SslSocket, "hello"), {error, closed} = ssl:recv(SslSocket, 0), io:fwrite("Accept: detected closed.~n"), ssl:close(SslSocket), io:fwrite("Listen: closing and terminating.~n"), ssl:close(LSock), lists:foreach(fun application:stop/1, lists:reverse(StartedApps)). %% Client connect init_connect(LPort) -> {ok, Host} = inet:gethostname(), {ok, CSock} = ssl:connect(Host, LPort, mk_opts(connect)), io:fwrite("Connect: connected.~n"), {ok, Cert} = ssl:peercert(CSock), io:fwrite("Connect: peer cert:~n~p~n", [public_key:pkix_decode_cert(Cert, otp)]), {ok, Data} = ssl:recv(CSock, 0), io:fwrite("Connect: got data: ~p~n", [Data]), io:fwrite("Connect: closing and terminating.~n"), ssl:close(CSock). mk_opts(listen) -> mk_opts("server"); mk_opts(connect) -> mk_opts("client"); mk_opts(Role) -> Dir = filename:join([code:lib_dir(ssl), "examples", "certs", "etc"]), [{active, false}, {verify, 2}, {depth, 2}, {handshake, hello}, {server_name_indication, disable}, {cacertfile, filename:join([Dir, Role, "cacerts.pem"])}, {certfile, filename:join([Dir, Role, "cert.pem"])}, {keyfile, filename:join([Dir, Role, "key.pem"])}]. From otp@REDACTED Wed Nov 6 11:31:11 2019 From: otp@REDACTED (Erlang/OTP) Date: Wed, 6 Nov 2019 11:31:11 +0100 (CET) Subject: Patch Package OTP 22.1.6 Released Message-ID: <20191106103111.17FA925485C@hel.cslab.ericsson.net> Patch Package: OTP 22.1.6 Git Tag: OTP-22.1.6 Date: 2019-11-06 Trouble Report Id: OTP-16219, OTP-16228, OTP-16241, OTP-16242 Seq num: ERIERL-427, ERL-1076, ERL-1078 System: OTP Release: 22 Application: compiler-7.4.8, crypto-4.6.2, erts-10.5.4, snmp-5.4.3 Predecessor: OTP 22.1.5 Check out the git tag OTP-22.1.6, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.4.8 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.4.8 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16219 Application(s): compiler, erts Related Id(s): ERL-1076 The compiler could do an unsafe optimization of receives, which would cause a receive to only scan part of the message queue. This bug fix in the compiler fixes a bug in the socket module. Full runtime dependencies of compiler-7.4.8: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- crypto-4.6.2 ---------------------------------------------------- --------------------------------------------------------------------- The crypto-4.6.2 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16242 Application(s): crypto Related Id(s): ERL-1078 The AEAD tag was not previously checked on decrypt with chacha20_poly1305 Full runtime dependencies of crypto-4.6.2: erts-9.0, kernel-5.3, stdlib-3.4 --------------------------------------------------------------------- --- erts-10.5.4 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.5.4 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16219 Application(s): compiler, erts Related Id(s): ERL-1076 The compiler could do an unsafe optimization of receives, which would cause a receive to only scan part of the message queue. This bug fix in the compiler fixes a bug in the socket module. OTP-16241 Application(s): erts Related Id(s): ERL-1076, OTP-16219 Fix bug where the receive marker used by the runtime to do the receive queue optimization could be incorrectly set. The symptom of this would be that a message that should match in a receive never matches. The bug requires the OTP-22 compiler and multiple selective receives to trigger. See OTP-16219 for details about the bug fix in the compiler. Full runtime dependencies of erts-10.5.4: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- snmp-5.4.3 ------------------------------------------------------ --------------------------------------------------------------------- The snmp-5.4.3 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16228 Application(s): snmp Related Id(s): ERIERL-427 Agent discovery cleanup. If there is no receiver of INFORM then #state.reqs in snmpa_net_if keeps on growing for DISCOVERY. Full runtime dependencies of snmp-5.4.3: crypto-3.3, erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, stdlib-2.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From raimo+erlang-questions@REDACTED Wed Nov 6 11:33:06 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 6 Nov 2019 11:33:06 +0100 Subject: Nobody is unsubscribed In-Reply-To: References: <20191104103427.GA81766@erix.ericsson.se> <97c4a6bf-e402-b853-5caf-9e6f20a7fd1e@ninenines.eu> <20191105154649.GA20777@erix.ericsson.se> Message-ID: <20191106103306.GA80084@erix.ericsson.se> On Wed, Nov 06, 2019 at 12:09:33AM +0800, by wrote: > Hi, > > What about making link https://www.erlang.org/mailman/listinfo/erlang-questions work? > > For now, link https://www2.erlang.org/mailman/listinfo/erlang-questions works. > > When a new Erlang user try to subscribe to this mailing list, he/she will input some information through http if link http://erlang.org/mailman/listinfo/erlang-questions is the final target. Yes. These are some of the content links to fix. We also have some general web server redirects to fix. / Raimo > > Yao > > > ? 2019?11?5??23:46?Raimo Niskanen ??? > > > > Certificates are now in place, and redirects are "working", so if you go to > > http://erlang.org you end up at https://www.erlang.org/. > > > > Remains that many links back from https://www.erlang.org to erlang.org > > downgrade to http:. > > > > We have not (yet) implemented HTTP Strict Transport Security (HSTS) > > on erlang.org or any of its subdomains. > > > > Will that be frowned upon? > > > > If not I think we ore done for now (apart from hunting down all bad links > > mentioned above). > > > > Thank you for your feedback! > > / Raimo Niskanen > > > > > > On Mon, Nov 04, 2019 at 11:53:16AM +0100, Lo?c Hoguin wrote: > >> For erlang.org itself there's two problems currently: no automatic > >> redirection from http to https; > >> > >> And this: > >> > >> Your connection is not private > >> This server could not prove that it is erlang.org; its security > >> certificate is from www2.erlang.org. This may be caused by a > >> misconfiguration or an attacker intercepting your connection. > >> > >> NET::ERR_CERT_COMMON_NAME_INVALID > >> Subject: www2.erlang.org > >> > >> Issuer: DigiCert SHA2 Secure Server CA > >> > >> Expires on: Oct 22, 2021 > >> > >> Current date: Nov 4, 2019 > >> > >> Keep up the good work. > >> > >> On 04/11/2019 11:34, Raimo Niskanen wrote: > >>> On Mon, Nov 04, 2019 at 10:47:03AM +0100, Adam Lindberg wrote: > >>>> Speaking of servers and domains, when is HTTPS coming to erlang.org and it?s sub-domains? > >>> > >>> HTTPS has been active for www.erlang.org and bugs.erlang.org for years. > >>> The recent web server upgrade enabled it for erlang.org as well; > >>> we are working on it... > >>> > >>> Best regards > >>> / Raimo > >>> > >>> > >>>> > >>>> Cheers, > >>>> Adam > >>>> > >>>>> On 2. Nov 2019, at 09:14, Raimo Niskanen wrote: > >>>>> > >>>>> Yes it does. It applies to all mailing lists. > >>>>> > >>>>> Ericsson has got its eyes on mailing lists at erlang.org since it owns the domain. > >>>>> > >>>>> Best regards > >>>>> / Raimo Niskanen > >>>>> > >>>>> Den l?r 2 nov. 2019 02:47Richard O'Keefe skrev: > >>>>> Does this apply to the EEPS list as well? > >>>>> > >>>>> On Sat, 2 Nov 2019 at 04:25, Joe Harrison wrote: > >>>>>> > >>>>>> Thanks for doing all of this, regardless. > >>>>>> > >>>>>> There's no perfect way to do mailing lists in a DMARC/DKIM/SPF compliant > >>>>>> way that doesn't break some client's "From:" field, subject line, or > >>>>>> "Reply:" button in some way, but this seems like the least bad option. > >>>>>> > >>>>>> I hope my emails make it through to the list now ^_^ > >>>>>> > >>>>>> OT: Be careful of organisations' web contact forms which ask for your > >>>>>> email address. Sometimes their web servers generate an email from the > >>>>>> form using your email address as the "From:" address, which will break a > >>>>>> lot of DKIM/DMARC/SPF stuff. > >>>>>> I know of at least one local authority (council) website in the UK which > >>>>>> is guilty of this. > >>>>>> > >>>>>> - Joe > >>>>>> > >>>>>> On 26/10/2019 07:57, Raimo Niskanen wrote: > >>>>>>> It is mainly "the big ones" that have been affected by stricter DMARC > >>>>>>> policies. > >>>>>>> > >>>>>>> When a subscriber sending from e.g Yahoo gets received by Gmail then > >>>>>>> Gmail rejects that message since Yahoo's DMARC policy says so (also vice > >>>>>>> versa). So the list gets a bounce and eventually blocks the Gmail > >>>>>>> subscriber, if enough in a row happens to send with strict DMARC policies. > >>>>>>> > >>>>>>> So for some it has worked, some gets an annoying list probe every now > >>>>>>> and then, some do not get many posts, but the final nail in the coffin > >>>>>>> was Ericsson (Erlang/OTP's home corporation) that tightened its DMARC > >>>>>>> policy and at the same time told us to get our act together and stop > >>>>>>> sending "unhygienic e-mail". > >>>>>>> > >>>>>>> All the best > >>>>>>> / Raimo > >>>>>>> > >>>>>>> > >>>>>>> Den fre 25 okt. 2019 16:58Chris Rempel >>>>>>> > skrev: > >>>>>>> > >>>>>>> Not having the subject contain [erlang-questions] or some other > >>>>>>> obvious indicator is quite unfortunate. I guess many people were > >>>>>>> affected by not being DMARC compliant? It seems to have been > >>>>>>> working just fine for quite some time... ie it "works for me" as it was. > >>>>>>> > >>>>>>> That said, thanks for maintaining the list, and keeping it going. > >>>>>>> It is a most useful resource. > >>>>>>> > >>>>>>> Chris > >>>>>>> > >>>>>>> *Sent:* Friday, October 25, 2019 at 7:38 AM > >>>>>>> *From:* "Raimo Niskanen" >>>>>>> > > >>>>>>> *To:* erlang-questions@REDACTED > >>>>>>> *Subject:* Re: Nobody is unsubscribed > >>>>>>> To achieve DMARC compliance we have stopped changing the Subject: > >>>>>>> field and no longer add the mailing list footer to the messages. > >>>>>>> > >>>>>>> This is because From: Subject: and mail body among other fields are > >>>>>>> often DKIM signed, so if we should change them we would not pass DKIM > >>>>>>> signature check and thereby not be DMARC compliant. > >>>>>>> > >>>>>>> Sorry for the inconvenience, we do not make the rules... > >>>>>>> / Raimo Niskanen > >>>>>>> > >>>>>>> On Fri, Oct 25, 2019 at 3:23 PM Raimo Niskanen >>>>>>> > wrote: > >>>>>>>> > >>>>>>>> The reason we changed mailing list servers was to get better DMARC and > >>>>>>>> DKIM compliance. This is a test post for us to inspect its headers... > >>>>>>>> -- > >>>>>>>> Raimo Niskanen > >>>>>>> > >>>>>> > >>>> > >>> > >> > >> -- > >> Lo?c Hoguin > >> https://ninenines.eu > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ben.browitt@REDACTED Wed Nov 6 14:54:39 2019 From: ben.browitt@REDACTED (Ben Browitt) Date: Wed, 6 Nov 2019 15:54:39 +0200 Subject: socket module rcvbuf and sndbuf Message-ID: Hi, I'm writing a simple wrapper around the socket nif to test the performance compared to gen_udp. My questions are specific to UDP sockets. How do the socket rcvbuf and the otp rcvbuf relate to each other? The 'rcvbuf' with level = socket is the size of the OS level receive buffer. The 'rcvbuf' with level = otp is the size of the buffer "we" read into when we read data (with recv/recvfrom/recvmsg). If UDP MTU is 1500 bytes, what happens if the socket rcvbuf=MTU or rcvbuf=MTU*N? Should the otp rcvbuf be equal to the socket rcvbuf or larger? What's the difference? What happen if the wrapper module doesn't read messages fast enough from the socket nif? Does sndbuf have any effect on UDP sockets? Should the wrapper module queue packets if the sndbuf is full and resend when possible or should it just drop packets? What's the difference between recv, recvfrom, recvmsg and similarly send, sendmsg, sendto? Is it relatively safe to use the socket module in production? What should I be careful about when implementing the udp wrapper module? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Wed Nov 6 16:39:01 2019 From: marc@REDACTED (Marc Worrell) Date: Wed, 6 Nov 2019 16:39:01 +0100 Subject: [ANN] Zotonic 0.52.0 released Message-ID: <448AD1C4-5977-47B0-AFDF-5C4BAC0E2BD1@worrell.nl> Hi, Zotonic[1] is the Erlang web framework and content management system. We have released Zotonic version 0.52.0. This is a maintenance release and marks our 101th release. Some features of Zotonic are: * Rich content editing environment * Powerful templating system * Flexible domain model based on semantic web * Rich media support, automatic resizing and preview generation * Multilingual content * Timezone support * Email send and receive * File storage on S3 or file system * Virtual hosting, run multiple independent sites on same host * Many more modules for SEO, social integration, etc. etc. Work on the major 1.x release is progressing, we will be able to share more about that release in the coming months. Fixes and changes in release 0.52.0: ? Add controller option set_session_cookie, set to false to not set a session cookie ? Add a delete button in the tinymce media edit dialog ? API for fetching the active language list /api/translation/language_list ? Add tinyMCE release 4.9.6 ? Fix site install using the cli addsite command; replace .dev with .test ? Fix a problem where ui logging in mod_logging could not be disabled ? Fix a problem with resource merge where edges notifications were not sent ? Fix a problem where path names in template debug were truncated Full release notes and download links can be found here here: https://github.com/zotonic/zotonic/releases/tag/0.52.0 Kind regards, The Zotonic core team [1] http://zotonic.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From marthin@REDACTED Thu Nov 7 09:04:51 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Thu, 07 Nov 2019 10:04:51 +0200 Subject: [Original] Re: Solving the right problem In-Reply-To: References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> <9f1a5343-2759-ddee-ae0d-aa2ace3d887b@mansionfamily.plus.com> <0B4AD10F-C097-4B40-9AA3-1D8A4D544C24@lobeshare.co.za> Message-ID: Thanks. I?m sorry for not making it clear that in my case the communication question is integral to my business logic. I certainly don?t want to abstract it away behind a general purpose tool which involves sequential fundamentals such as single threaded message queues. XMPP would inevitably come into play higher up in the application stack but I?d be hard-pressed to latch my design paradigm to the general messaging mindset. From my project?s perspective messaging would be of its many derivatives. I?m interested in learning as much as possible from those who designed other more general purpose implementations such a you mentioned. I particularly found the RabbitMQ best practices article very useful. The general gist of those boils down to using your best understanding of your clients, data and resources to design your messages and message queues so bottlenecks are avoided as a result. That?s fine, when you know those things in advance which I simply don?t. Everything about my clients, data and resources would be constantly changing. I?d be a fool to try and predict any of it at design time and an idiot to then try impose my predictions on reality. But I am confident that I can abstract the nature of changes my clients, data and resources would changes would be subjected to well enough to design and build the code to measure and respond to those changes as and when they happen. In terms of the RabbitMQ best practices I mentioned, that would translate into adapting the design of my queues, nodes and messages back and forth based on transient, slow-changing and fast-acting changes ? not what I?d call simple nor practical. I?ve managed to simplify the solution down to a single core datatype in which all data is stored, exchanged and referenced, ranging from a few bytes to as big as needed but with a large composite key. The obvious storage problem this introduces is overcome by applying the optimization techniques which Edgar Codd taught and many more dynamically at run-time based on measured meta-data as opposed to design time based on forecast-based modelling. While that may seem impossibly complex to pull off, it actually becomes brilliantly simple and easy once you?ve set up the environment at the appropriate level of abstraction. From: T Ty Date: Wednesday, 6 November, 2019 at 11:04 To: "marthin@REDACTED" Cc: Erlang Questions Subject: Re: [Original] Re: Solving the right problem I would suggest you ignore the Erlang/Elixir mode of thought as a starting point. Your architecture is closely related to either XMPP or AMQP protocol and suggest you base it off these instead of heading straight down do a coding language. Of course I would be bias towards MongooseIM or RabbitMQ. This abstracts the entire communication question away and you can focus on the business logic. In XMPP, each client device and server would be XMPP clients and both would communicate using simple text based coded messages. You automatically get federation and presence notification. Store and forward is automatically built in. XMPP is also moving towards addressing IoT devices which might further simplify your architecture. In RabbitMQ each client/server would have a two queue (in/out) and all communications goes through these. Multiple clients can share a single queue pair if they are grouped together e.g. all clients within the same geo-location. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Nov 7 15:46:05 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 7 Nov 2019 15:46:05 +0100 Subject: Solving the right problem In-Reply-To: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: On Mon, Nov 4, 2019 at 1:10 PM Marthin Laubscher wrote: > Hi everyone, > > > > Please pardon my long absences from this awesome (mature) community and my > limited knowledge which is likely outdated as well. I?ve known since 1996 > when I was first told (in confidence by an Ericsson Radio Systems liaison) > about Erlang that it would have to play a role when I eventually get to > implementing the system I?ve been working on designing since 1991. That big > day is drawing near, so now I?d like to reaffirm my high level > understanding of what the language is suited for. > > If you have been working on this design since 1991, I would recommend you start some experiments into parts of the design. You can often simulate a large amount of devices, and you can also simulate the server side. If there are multiple servers involved, solving the problem in the small scale is often useful: you get to learn about some of the small-scale problems of the design right away. As you scale such a system up, you will run into trouble that has to be solved. But often such solutions stem from the smaller scale ideas you tackled first. At scale, I claim any system breaks down. The real work starts when you reach that level. Almost any web server technology has a breaking point, and you will hit it sooner or later. Erlang has a breaking point and you will hit it sooner or later. It also depends on the nature of the problem you are trying to solve. If you are looking at number crunching, I'd definitely design a hybrid where that is outsourced to some other language suited for fast execution. On the other hand, if you are looking at a problem where you have massive concurrency, you might get away with Erlang or Go. If you add the requirement of robustness on top, at a fine granular level, then I know no other thing than the Erlang stack which can do it. Other languages achieve robustness by containerization of computation and then restarting faulty containers. Yet, the disadvantage of such a method are that errors in such systems can "bleed" to sibling requests. This might not be suitable, depending on the problem at hand. But you really need to start experiments. It is almost impossible to design a software system without the duality of design and implementation helping each other along. That is, design leads to partial implementation. Partial implementation leads to insights. And insights leads to a better design. Over time, it is often the case the design shrinks to a fundamental core. You want that as small as possible as it makes it understandable. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 64024 bytes Desc: not available URL: From ingela.andin@REDACTED Thu Nov 7 15:55:35 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 7 Nov 2019 15:55:35 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi! I tried this out and it is not out of order, it sends the peer cert followed by the intermediate cert repeated, that is the chain looks like [Peer, CA1, CA1]. Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I suppose we need to add that. Regards Ingela Erlang/OTP team - Ericsson AB Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : > Hey, > > I confirm that out of order certs does not seems to be fixed, and it fails > with 'Unknown CA' error: > > > iex(2)> :hackney.get("https://social.fluffel.io") > {:error, > {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} > > > the only issue with this server TLS certificates is the chain order (CA is > Letsencrypt): > https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io > > > On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: > > Hi! > > Just curious if there is an update on out of order certs. > > The example has id0, id1, id2, id3 certs with id1 being the natural > root of id2 who is the root of id3, who is the root of id0. > > We can correct the out of order problem by including id1,id2,id3 certs > in our chain. > > It would be nice to hear from the erlang maintainers around what kind of > "out of order" erlang can handle nicely and if there is planned support for > our case! > > Thank you again, > > Curtis. > > > Sent through ProtonMail Encrypted Email Channel. > > > ??????? Original Message ??????? > On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield > wrote: > > Hi! Thank you. > > > I included the root cert in the example. The root cert is id1 in cert > chain - this is evident in the other file. > > It seems because the root cert is out of order - the cert chain is invalid > - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 > > > Thank you for your consideration! > > > Sent from ProtonMail Mobile > > > On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin > wrote: > > > Hi! > > "Unknown CA" means that you did not have the ROOT certificate of the > chian in your "trusted store" (cacerts option). > If you do not own the ROOT certificate you can not trust the chain. > > Regards Ingela Erlang/OTP Team - Ericsson AB > > Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : > > Dear Erlang Questions: > > > SSL 9.0.2 mentions a patch to fix out of order cert chains > > In SSL 9.2 we have a root CA and an out of order cert chain > for host hooks.glip.com. > > When we try to verify peer with the out of order cert > chain we get 'Unknown CA'. > > Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? > > The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes > mention that other care may need to be taken to ensure compatibility. > > Reproduce error: > > https://github.com/robotarmy/out-of-order-ssl > > Thank you, > Curtis and Team DevEco > > > > > Sent through ProtonMail Encrypted Email Channel. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andras.Bekes@REDACTED Thu Nov 7 17:59:12 2019 From: Andras.Bekes@REDACTED (Bekes, Andras G) Date: Thu, 7 Nov 2019 16:59:12 +0000 Subject: [erlang-questions] Segfault with Erlang R22 In-Reply-To: <20191102100047.296cb65d@willy.fritz.box> References: <649d87fb8e0346f7ad0a039e483a46af@morganstanley.com> <50f54b6f7f9b4e88ae3d1e540c881ba2@morganstanley.com> <20191102100047.296cb65d@willy.fritz.box> Message-ID: <2b2ee1bea56f4ec387b0955403a856dc@morganstanley.com> ?I am not entirely sure of what we're doing, but here is the output: (gdb) frame 0 #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 5252 c_p->seq_trace_lastcnt = unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); (gdb) print x_reg_array $4 = (Eterm *) 0x20002 (gdb) print *x_reg_array Cannot access memory at address 0x20002 (gdb) print f_reg_array $5 = (FloatDef *) 0x2 (gdb) print *f_reg_array Cannot access memory at address 0x2 (gdb) print c_p $7 = (gdb) print *c_p value has been optimized out (gdb) frame 1 #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at beam/erl_process.c:8465 8465 process_main(esdp->x_reg_array, esdp->f_reg_array); (gdb) print esdp $8 = (ErtsSchedulerData *) 0x2b8244840200 (gdb) print *esdp $9 = {x_reg_array = 0x2b823e940200, f_reg_array = 0x2b823e942240, timer_wheel = 0x2b82450f5c80, next_tmo_ref = 0x2b8245136120, timer_service = 0x2b8245176680, tid = 47838514915072, erl_bits_state = { byte_buf_ = 0x2b823ad81058 "", byte_buf_len_ = 1, erts_current_bin_ = 0x2b832e60c688 "\n\274\362T", erts_bin_offset_ = 32, erts_writable_bin_ = 0}, match_pseudo_process = 0x2b823bec7c78, free_process = 0x0, thr_progress_data = {id = 1, is_managed = 1, is_blocking = 0, is_temporary = 0, wakeup_request = {5707836, 5707869, 5707862, 5707859}, leader = 0, active = 1, confirmed = 5707879, leader_state = {next = 5707875, current = 18446744073709551615, chk_next_ix = 2, umrefc_ix = {current = 0, waiting = -1}}}, ssi = 0x2b823be7e680, current_process = 0x2b82431401d8, type = ERTS_SCHED_NORMAL, no = 1, dirty_no = 0, flxctr_slot_no = 1, current_nif = 0x0, dirty_shadow_process = 0x0, current_port = 0x0, run_queue = 0x2b823be77ec0, virtual_reds = 0, cpu_id = -1, aux_work_data = {sched_id = 1, esdp = 0x2b8244840200, ssi = 0x2b823be7e680, current_thr_prgr = 5707878, latest_wakeup = 5707869, misc = {ix = 0, thr_prgr = 18446744073709551615}, dd = {thr_prgr = 5707869}, cncld_tmrs = { thr_prgr = 5707146}, later_op = {thr_prgr = 5707880, size = 65384, first = 0x2b832e8d76c8, last = 0x2b832e8d76c8}, async_ready = {need_thr_prgr = 0, thr_prgr = 18446744073709551615, queue = 0x2b8245059880}, delayed_wakeup = {next = 18446744073709551615, sched2jix = 0x2b82443650c8, jix = -1, job = 0x2b8244364f00}, yield = {alcu_blockscan = {current = 0x0, last = 0x0}, ets_all = {ongoing = 0x0, hfrag = 0x0, tab = 0x0, queue = 0x0}}, debug = {wait_completed = {flags = 0, callback = 0, arg = 0x0}}}, atom_cache_map = {hdr_sz = -1, sz = 0, long_atoms = 0, cix = {0 }, cache = {{atom = 0, iix = -1} }}, last_monotonic_time = 54631431230926, check_time_reds = 3137, thr_id = 1, unique = 251, ref = 1016430404454740281, alloc_data = {deallctr = {0x0, 0x0, 0x0, 0x2b81f77a9200, 0x2b823ad37200, 0x2b823bdaa200, 0x2b823e8d5200, 0x2b8240f13200, 0x2b8242ff9200, 0x0, 0x0, 0x2b823fea0200, 0x2b8241f86200, 0x0}, pref_ix = {0, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1}, flist_ix = {0 }, pre_alc_ix = 0}, io = { out = 21255996115, in = 21476723837}, pending_signal = {sig = 0x0, to = 0}, reductions = 1006796378, sched_wall_time = {u = {mod = {counter = 0}, need = 0}, enabled = 0, start = 0, working = {total = 0, start = 0}}, gc_info = {reclaimed = 775481964, garbage_cols = 680476}, nosuspend_port_task_handle = {counter = 0}, ets_tables = { count = {counter = 0}, clist = 0x0}} (gdb) print esdp->x_reg_array $10 = (Eterm *) 0x2b823e940200 (gdb) print *esdp->x_reg_array $11 = 2522015978211937347 (gdb) print esdp->f_reg_array $12 = (FloatDef *) 0x2b823e942240 (gdb) print *esdp->f_reg_array $13 = {fd = 0.002545, fb = "\323\023\226x@\331d?", fs = {5075, 30870, 55616, 16228}, fw = {2023101395, 1063573824}, fdw = 4568014792984761299} frame 2 is in already in pthread/ethread.c -----Original Message----- From: erlang-questions [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Eckard Brauer Sent: Saturday, November 02, 2019 10:01 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Segfault with Erlang R22 It's a few years ago, but IIRC either "print *c_p" or "print *((Process*) c_p)". Problem would probably be that the processor already left the stack frame where c_p is valid. You can do "info stack" at this point and select the frame with "frame <#>" to try it again. If you're a little familiar with assembly language, you can even have a look at "disassemble
" or "disassemble function" to get an idea of where values are at what point in the instruction/processing flow - sometimes this helps too. I'd investigate starting with frame 2 here, as all frames below are already in libpthread. Hope that helps a bit... Eckard Am Fri, 1 Nov 2019 18:22:18 +0000 schrieb "Bekes, Andras G" : > Program terminated with signal 11, Segmentation fault. > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > > 5252 c_p->seq_trace_lastcnt = > unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); Missing separate > debuginfos, use: debuginfo-install glibc-2.12-1.212.el6_10.3.x86_64 > (gdb) bt > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at > beam/erl_process.c:8465 > #2 0x000000000069262a in thr_wrapper (vtwd=) > at pthread/ethread.c:118 > #3 0x00002b81f80f7dd5 in _L_unlock_48 () from /lib64/libpthread.so.0 > #4 0x00002b81f80f5eb3 in __find_thread_by_id () from > /lib64/libpthread.so.0 > #5 0x0000000000000000 in ?? () > (gdb) > > I am not sure how to " print the c_p parameter via its raw value > (print *(Process*)0x.....)". Where should I take the value 0x..... > from? > > -----Original Message----- > From: Mikael Pettersson [mailto:mikpelinux@REDACTED] > Sent: Thursday, October 24, 2019 7:10 PM > To: Bekes, Andras G (IST) > Cc: Erlang Questions > Subject: Re: [erlang-questions] Segfault with Erlang R22 > > On Thu, Oct 24, 2019 at 4:57 PM Bekes, Andras G > wrote: > [...] > > I'd try to get a backtrace (bt command in gdb) from the crashed > thread, then maybe print > the c_p parameter via its raw value (print *(Process*)0x.....) if gdb > insists that the value > is optimized out. > > /Mikael > > [...] > [...] > [...] > -- Wir haften nicht f?r die korrekte Funktion der in dieser eMail enthaltenen Viren. We are not liable for correct function of the viruses in this email! :) From curtis@REDACTED Thu Nov 7 19:23:33 2019 From: curtis@REDACTED (Curtis J Schofield) Date: Thu, 07 Nov 2019 18:23:33 +0000 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi Ingela Thank you for your attention- perhaps Micheal can explain this better.. Sent from ProtonMail Mobile On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin wrote: > Hi! > > I tried this out and it is not out of order, it sends the peer cert followed by the intermediate cert repeated, that is the chain looks like [Peer, CA1, CA1]. > Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I suppose we need to add that. > > Regards Ingela Erlang/OTP team - Ericsson AB > > Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : > >> Hey, >> >> I confirm that out of order certs does not seems to be fixed, and it fails with 'Unknown CA' error: >> >> iex(2)> :hackney.get("https://social.fluffel.io") >> {:error, >> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} >> >> the only issue with this server TLS certificates is the chain order (CA is Letsencrypt): https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >> >> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >> >>> Hi! >>> >>> Just curious if there is an update on out of order certs. >>> >>> The example has id0, id1, id2, id3 certs with id1 being the natural >>> root of id2 who is the root of id3, who is the root of id0. >>> >>> We can correct the out of order problem by including id1,id2,id3 certs >>> in our chain. >>> >>> It would be nice to hear from the erlang maintainers around what kind of >>> "out of order" erlang can handle nicely and if there is planned support for >>> our case! >>> >>> Thank you again, >>> >>> Curtis. >>> >>> Sent through [ProtonMail](https://protonmail.com) Encrypted Email Channel. >>> >>> ??????? Original Message ??????? >>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield wrote: >>> >>>> Hi! Thank you. >>>> >>>> I included the root cert in the example. The root cert is id1 in cert chain - this is evident in the other file. >>>> >>>> It seems because the root cert is out of order - the cert chain is invalid - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 >>>> >>>> Thank you for your consideration! >>>> >>>> Sent from ProtonMail Mobile >>>> >>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin wrote: >>>> >>>>> Hi! >>>>> >>>>> "Unknown CA" means that you did not have the ROOT certificate of the chian in your "trusted store" (cacerts option). >>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>> >>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>> >>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >>>>> >>>>>> Dear Erlang Questions: >>>>>> >>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>> >>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>> for host hooks.glip.com. >>>>>> >>>>>> When we try to verify peer with the out of order cert >>>>>> chain we get 'Unknown CA'. >>>>>> >>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>> >>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>> mention that other care may need to be taken to ensure compatibility. >>>>>> >>>>>> Reproduce error: >>>>>> >>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>> >>>>>> Thank you, >>>>>> Curtis and Team DevEco >>>>>> >>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From beastie@REDACTED Thu Nov 7 19:45:45 2019 From: beastie@REDACTED (Mark Reynolds) Date: Thu, 07 Nov 2019 19:45:45 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: <1605b523-8d8a-4f40-b7f5-48785b057717@www.fastmail.com> If that helps, SSLLabs reports "Incorrect order, contains anchor" for this one. Maybe it's related to the anchor? On Thu, Nov 7, 2019, at 19:35, Michael Viveros wrote: > Hi Ingela, > > Curtis' example server from his first message, hooks.glip.com, presents its certificates out-of-order. The correct order is Peer -> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown CA" error. You can confirm this by running `openssl s_client -connect hooks.glip.com:443`. > > On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield wrote: >> Hi Ingela >> >> Thank you for your attention- perhaps Micheal can explain this better.. ____ >> >> Sent from ProtonMail Mobile >> >> >> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin wrote: >>> Hi! >>> >>> I tried this out and it is not out of order, it sends the peer cert followed by the intermediate cert repeated, that is the chain looks like [Peer, CA1, CA1]. >>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I suppose we need to add that. >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>> __ >>>> Hey, >>>> >>>> I confirm that out of order certs does not seems to be fixed, and it fails with 'Unknown CA' error: >>>> >>>> >>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>> {:error, >>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} >>>> >>>> >>>> the only issue with this server TLS certificates is the chain order (CA is Letsencrypt): https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>> >>>> >>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>> Hi! >>>>> >>>>> Just curious if there is an update on out of order certs. >>>>> >>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>> root of id2 who is the root of id3, who is the root of id0. >>>>> >>>>> We can correct the out of order problem by including id1,id2,id3 certs >>>>> in our chain. >>>>> >>>>> It would be nice to hear from the erlang maintainers around what kind of >>>>> "out of order" erlang can handle nicely and if there is planned support for >>>>> our case! >>>>> >>>>> Thank you again, >>>>> >>>>> Curtis. >>>>> >>>>> >>>>> Sent through ProtonMail Encrypted Email Channel. >>>>> >>>>> >>>>> ??????? Original Message ??????? >>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield wrote: >>>>> >>>>>> Hi! Thank you. >>>>>> >>>>>> >>>>>> I included the root cert in the example. The root cert is id1 in cert chain - this is evident in the other file. >>>>>> >>>>>> It seems because the root cert is out of order - the cert chain is invalid - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 >>>>>> >>>>>> >>>>>> Thank you for your consideration! >>>>>> >>>>>> >>>>>> Sent from ProtonMail Mobile >>>>>> >>>>>> >>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin wrote: >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> "Unknown CA" means that you did not have the ROOT certificate of the chian in your "trusted store" (cacerts option). >>>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>>> >>>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>>> >>>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >>>>>>>> Dear Erlang Questions: >>>>>>>> >>>>>>>> >>>>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>>>> >>>>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>>>> for host hooks.glip.com. >>>>>>>> >>>>>>>> When we try to verify peer with the out of order cert >>>>>>>> chain we get 'Unknown CA'. >>>>>>>> >>>>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>>>> >>>>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>>>> mention that other care may need to be taken to ensure compatibility. >>>>>>>> >>>>>>>> Reproduce error: >>>>>>>> >>>>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>>>> >>>>>>>> Thank you, >>>>>>>> Curtis and Team DevEco >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Nov 7 20:53:37 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 7 Nov 2019 20:53:37 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi! Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < michaelviveros@REDACTED>: > Hi Ingela, > > Curtis' example server from his first message, hooks.glip.com, presents > its certificates out-of-order. The correct order is Peer -> Intermediate CA > 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root > CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown > CA" error. You can confirm this by running `openssl s_client -connect > hooks.glip.com:443`. > > Yes I agree that this is an out of order chain, in contrast to the social.fluffel.io. I will look into it at work tomorrow. Regards Ingela Erlang/OTP Team - Ericsson AB > On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield wrote: > >> Hi Ingela >> >> Thank you for your attention- perhaps Micheal can explain this better.. >> >> Sent from ProtonMail Mobile >> >> >> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >> wrote: >> >> Hi! >> >> I tried this out and it is not out of order, it sends the peer cert >> followed by the intermediate cert repeated, that is the chain looks like >> [Peer, CA1, CA1]. >> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I >> suppose we need to add that. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >> >>> Hey, >>> >>> I confirm that out of order certs does not seems to be fixed, and it >>> fails with 'Unknown CA' error: >>> >>> >>> iex(2)> :hackney.get("https://social.fluffel.io") >>> {:error, >>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} >>> >>> >>> the only issue with this server TLS certificates is the chain order (CA >>> is Letsencrypt): >>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>> >>> >>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>> >>> Hi! >>> >>> Just curious if there is an update on out of order certs. >>> >>> The example has id0, id1, id2, id3 certs with id1 being the natural >>> root of id2 who is the root of id3, who is the root of id0. >>> >>> We can correct the out of order problem by including id1,id2,id3 certs >>> in our chain. >>> >>> It would be nice to hear from the erlang maintainers around what kind of >>> "out of order" erlang can handle nicely and if there is planned support >>> for >>> our case! >>> >>> Thank you again, >>> >>> Curtis. >>> >>> >>> Sent through ProtonMail Encrypted Email >>> Channel. >>> >>> >>> ??????? Original Message ??????? >>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>> wrote: >>> >>> Hi! Thank you. >>> >>> >>> I included the root cert in the example. The root cert is id1 in cert >>> chain - this is evident in the other file. >>> >>> It seems because the root cert is out of order - the cert chain is >>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>> TLS1.2 >>> >>> >>> Thank you for your consideration! >>> >>> >>> Sent from ProtonMail Mobile >>> >>> >>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin >>> wrote: >>> >>> >>> Hi! >>> >>> "Unknown CA" means that you did not have the ROOT certificate of the >>> chian in your "trusted store" (cacerts option). >>> If you do not own the ROOT certificate you can not trust the chain. >>> >>> Regards Ingela Erlang/OTP Team - Ericsson AB >>> >>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >>> >>> Dear Erlang Questions: >>> >>> >>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>> >>> In SSL 9.2 we have a root CA and an out of order cert chain >>> for host hooks.glip.com. >>> >>> When we try to verify peer with the out of order cert >>> chain we get 'Unknown CA'. >>> >>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>> >>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>> mention that other care may need to be taken to ensure compatibility. >>> >>> Reproduce error: >>> >>> https://github.com/robotarmy/out-of-order-ssl >>> >>> Thank you, >>> Curtis and Team DevEco >>> >>> >>> >>> >>> Sent through ProtonMail Encrypted Email Channel. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> >>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Fri Nov 8 01:57:39 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Fri, 8 Nov 2019 01:57:39 +0100 Subject: mnesia initialization Message-ID: Hi, I have problem with mnesia initialization, I have a db_init function that initializes db so that if tables do not exist they are created (ram with disc_copies). For example: db_init() -> mnesia:create_table(some_record,[{attributes, record_info(fields, some_record)}, {type, bag}, {disc_copies, [node()]}]). I'm running this function on the app start to make sure the table exists and is ready. However half of the time it destroys old table returning {atomic, ok} and another half it returns with already_exists. Also, this happens in the regular fashion, if I have n starts of application, even starts would destroy data and even wold keeps old data. I know that I am missing something probably obvious here but if anyone has any insights on why is this happening and how to fix it I would greatly appreciate it. Thank You, Karlo. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michaelviveros@REDACTED Thu Nov 7 19:35:23 2019 From: michaelviveros@REDACTED (Michael Viveros) Date: Thu, 7 Nov 2019 13:35:23 -0500 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi Ingela, Curtis' example server from his first message, hooks.glip.com, presents its certificates out-of-order. The correct order is Peer -> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown CA" error. You can confirm this by running `openssl s_client -connect hooks.glip.com:443`. On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield wrote: > Hi Ingela > > Thank you for your attention- perhaps Micheal can explain this better.. > > Sent from ProtonMail Mobile > > > On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin > wrote: > > Hi! > > I tried this out and it is not out of order, it sends the peer cert > followed by the intermediate cert repeated, that is the chain looks like > [Peer, CA1, CA1]. > Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I > suppose we need to add that. > > Regards Ingela Erlang/OTP team - Ericsson AB > > Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : > >> Hey, >> >> I confirm that out of order certs does not seems to be fixed, and it >> fails with 'Unknown CA' error: >> >> >> iex(2)> :hackney.get("https://social.fluffel.io") >> {:error, >> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} >> >> >> the only issue with this server TLS certificates is the chain order (CA >> is Letsencrypt): >> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >> >> >> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >> >> Hi! >> >> Just curious if there is an update on out of order certs. >> >> The example has id0, id1, id2, id3 certs with id1 being the natural >> root of id2 who is the root of id3, who is the root of id0. >> >> We can correct the out of order problem by including id1,id2,id3 certs >> in our chain. >> >> It would be nice to hear from the erlang maintainers around what kind of >> "out of order" erlang can handle nicely and if there is planned support >> for >> our case! >> >> Thank you again, >> >> Curtis. >> >> >> Sent through ProtonMail Encrypted Email Channel. >> >> >> ??????? Original Message ??????? >> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >> wrote: >> >> Hi! Thank you. >> >> >> I included the root cert in the example. The root cert is id1 in cert >> chain - this is evident in the other file. >> >> It seems because the root cert is out of order - the cert chain is >> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >> TLS1.2 >> >> >> Thank you for your consideration! >> >> >> Sent from ProtonMail Mobile >> >> >> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin >> wrote: >> >> >> Hi! >> >> "Unknown CA" means that you did not have the ROOT certificate of the >> chian in your "trusted store" (cacerts option). >> If you do not own the ROOT certificate you can not trust the chain. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >> >> Dear Erlang Questions: >> >> >> SSL 9.0.2 mentions a patch to fix out of order cert chains >> >> In SSL 9.2 we have a root CA and an out of order cert chain >> for host hooks.glip.com. >> >> When we try to verify peer with the out of order cert >> chain we get 'Unknown CA'. >> >> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >> >> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >> mention that other care may need to be taken to ensure compatibility. >> >> Reproduce error: >> >> https://github.com/robotarmy/out-of-order-ssl >> >> Thank you, >> Curtis and Team DevEco >> >> >> >> >> Sent through ProtonMail Encrypted Email Channel. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From spylik@REDACTED Fri Nov 8 03:44:42 2019 From: spylik@REDACTED (Oleksii Semilietov) Date: Fri, 8 Nov 2019 09:44:42 +0700 Subject: mnesia initialization In-Reply-To: References: Message-ID: Actually you need first to check does table exists on local node or not. Instead of just create_table you should have some routine like init_table(TableName, TableDef) -> AllTables = mnesia:system_info(tables), case lists:member(TableName, AllTables) of false -> mnesia:create_table(TableName, TableDef); true -> wait_table(TableName) end . In wait_table you should implement mnesia loading table routine. The great starting point how to wrap up mnesia routine is in https://github.com/rabbitmq/rabbitmq-server/blob/6cbd36bd79dd72817ec1ebe4a01c0d8a4bc7d69e/src/rabbit_mnesia.erl and in https://github.com/rabbitmq/rabbitmq-server/blob/6cbd36bd79dd72817ec1ebe4a01c0d8a4bc7d69e/src/rabbit_table.erl On Fri, 8 Nov 2019 at 07:58, Karlo Kuna wrote: > Hi, > > I have problem with mnesia initialization, > > I have a db_init function that initializes db so that if tables do not > exist they are created (ram with disc_copies). For example: > > db_init() -> > mnesia:create_table(some_record,[{attributes, record_info(fields, > some_record)}, > {type, > bag}, {disc_copies, [node()]}]). > > I'm running this function on the app start to make sure the table exists > and is ready. > However half of the time it destroys old table returning {atomic, ok} and > another half it returns with already_exists. Also, this happens in the > regular fashion, if I have n starts of application, even starts would > destroy data and even wold keeps old data. I know that I am missing > something probably obvious here but if anyone has any insights on why is > this happening and how to fix it I would greatly appreciate it. > > Thank You, > Karlo. > -- Best regards, Alex [Oleksii Semilietov] -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Fri Nov 8 06:45:59 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Fri, 8 Nov 2019 12:45:59 +0700 Subject: mnesia initialization In-Reply-To: References: Message-ID: Hi, Make sure to: 1. Start your erlang node with -mnesia dir "/path/to/where/you/want/to/put/mnesia" 2. Create the schema first 3. start mnesia by using command mnesia:start() 4. The create your tables >> I have a db_init function that initializes db so that if tables do not exist they are created (ram with disc_copies) If you want to do a table existence detection, you can do like the following: case catch mnesia:table(your_table_name, version) of {{_, _}, []} -> table_exists; Fail when Fail =:= {'EXIT',{aborted,{no_exists,your_table_name,version}}}; Fail =:= {aborted,{no_exists,your_table_name,version}} -> %% create your your_table_name table end Also, as i stated above, you have to create your schema first then start mnesia application. Example: case mnesia:create_schema([node()]) of {error,{_,{already_exists,_}}} -> io:format("Schema Already exists~n"); ok -> io:format("Schema Created~n") end, application:ensure_started(mnesia), %% then init your tables here %% %% . Please read http://erlang.org/doc/apps/mnesia/Mnesia_chap2.html >> However half of the time it destroys old table returning {atomic, ok} and another half it returns with already_exists I'm not sure with this. But maybe i think your schema location was not consistent I suggest you to use rebar3 to manage your project. Pada tanggal Jum, 8 Nov 2019 pukul 07.57 Karlo Kuna menulis: > Hi, > > I have problem with mnesia initialization, > > I have a db_init function that initializes db so that if tables do not > exist they are created (ram with disc_copies). For example: > > db_init() -> > mnesia:create_table(some_record,[{attributes, record_info(fields, > some_record)}, > {type, > bag}, {disc_copies, [node()]}]). > > I'm running this function on the app start to make sure the table exists > and is ready. > However half of the time it destroys old table returning {atomic, ok} and > another half it returns with already_exists. Also, this happens in the > regular fashion, if I have n starts of application, even starts would > destroy data and even wold keeps old data. I know that I am missing > something probably obvious here but if anyone has any insights on why is > this happening and how to fix it I would greatly appreciate it. > > Thank You, > Karlo. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marthin@REDACTED Fri Nov 8 09:31:42 2019 From: marthin@REDACTED (Marthin Laubscher) Date: Fri, 08 Nov 2019 10:31:42 +0200 Subject: Solving the right problem In-Reply-To: References: <215784F6-365A-44B9-B365-FCF461652A03@lobeshare.co.za> Message-ID: <3E170530-E19B-4752-9513-F5A9904097EF@lobeshare.co.za> Spot on. I really should allow the rubber to hit the road now, which I am doing. For the record, what happened in 1991 wasn?t to that I started designing the server I talk about, but that is when I accepted it as my mission to solve a problem I could no longer ignore like others still do. Erlang/OTP was one of many things I filled my toolbox with for use as and when the time comes. Some of the hurdles and mental blocks I am looking to break through include:- the apparent manual labour involved in setting up and managing distributed Erlang nodes including setting up physical machines, configuring virtual machines, installing OS, installing Erlang, installing the app, controlling automatic startup and shutdown, plumbing the network and hooking up release management so changes in the experimental code gets rolled out orderly and automatically, the incredible performance of modern hardware in a single processor node, the cost and complexity of monitoring virtual machines? activity on the underlying hardware, that generating suitable traffic to stress test candidate code might be more troublesome to simulate than it?s worth and programming isn?t what gives meaning to my life but I love it and tends to get too deeply involved and thus distracted by it from other responsibilities. So by all means, come and join me and my project. Help me overcome and/or sidestep my issues so we can collaborate at the appropriate level on that setting up this adaptive hyper-scaling platform for matching requests and resources to deliver near-real-time responses. We might even call it NoQ, NoMQ or OPR (see, no Q ??, or Optimal Processing Resources). From: Jesper Louis Andersen Date: Thursday, 7 November, 2019 at 16:48 To: "marthin@REDACTED" Cc: "Erlang (E-mail)" Subject: Re: Solving the right problem On Mon, Nov 4, 2019 at 1:10 PM Marthin Laubscher wrote: Hi everyone, Please pardon my long absences from this awesome (mature) community and my limited knowledge which is likely outdated as well. I?ve known since 1996 when I was first told (in confidence by an Ericsson Radio Systems liaison) about Erlang that it would have to play a role when I eventually get to implementing the system I?ve been working on designing since 1991. That big day is drawing near, so now I?d like to reaffirm my high level understanding of what the language is suited for. If you have been working on this design since 1991, I would recommend you start some experiments into parts of the design. You can often simulate a large amount of devices, and you can also simulate the server side. If there are multiple servers involved, solving the problem in the small scale is often useful: you get to learn about some of the small-scale problems of the design right away. As you scale such a system up, you will run into trouble that has to be solved. But often such solutions stem from the smaller scale ideas you tackled first. At scale, I claim any system breaks down. The real work starts when you reach that level. Almost any web server technology has a breaking point, and you will hit it sooner or later. Erlang has a breaking point and you will hit it sooner or later. It also depends on the nature of the problem you are trying to solve. If you are looking at number crunching, I'd definitely design a hybrid where that is outsourced to some other language suited for fast execution. On the other hand, if you are looking at a problem where you have massive concurrency, you might get away with Erlang or Go. If you add the requirement of robustness on top, at a fine granular level, then I know no other thing than the Erlang stack which can do it. Other languages achieve robustness by containerization of computation and then restarting faulty containers. Yet, the disadvantage of such a method are that errors in such systems can "bleed" to sibling requests. This might not be suitable, depending on the problem at hand. But you really need to start experiments. It is almost impossible to design a software system without the duality of design and implementation helping each other along. That is, design leads to partial implementation. Partial implementation leads to insights. And insights leads to a better design. Over time, it is often the case the design shrinks to a fundamental core. You want that as small as possible as it makes it understandable. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonas.falkevik@REDACTED Fri Nov 8 10:37:09 2019 From: jonas.falkevik@REDACTED (Jonas Falkevik) Date: Fri, 8 Nov 2019 10:37:09 +0100 Subject: [erlang-questions] Segfault with Erlang R22 In-Reply-To: <2b2ee1bea56f4ec387b0955403a856dc@morganstanley.com> References: <649d87fb8e0346f7ad0a039e483a46af@morganstanley.com> <50f54b6f7f9b4e88ae3d1e540c881ba2@morganstanley.com> <20191102100047.296cb65d@willy.fritz.box> <2b2ee1bea56f4ec387b0955403a856dc@morganstanley.com> Message-ID: Have you tried loading the erts gdb scripts? should be found under "erts/etc/unix/etp-commands" Then you can get stacktrace from process for example.. (gdb) source (gdb) set $p = (Process *)0x2b82431401d8 (gdb) etp-stacktrace $p /Jonas On Thu, Nov 7, 2019 at 5:59 PM Bekes, Andras G wrote: > I am not entirely sure of what we're doing, but here is the output: > > (gdb) frame 0 > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > 5252 c_p->seq_trace_lastcnt = > unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); > (gdb) print x_reg_array > $4 = (Eterm *) 0x20002 > (gdb) print *x_reg_array > Cannot access memory at address 0x20002 > (gdb) print f_reg_array > $5 = (FloatDef *) 0x2 > (gdb) print *f_reg_array > Cannot access memory at address 0x2 > (gdb) print c_p > $7 = > (gdb) print *c_p > value has been optimized out > > (gdb) frame 1 > #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at > beam/erl_process.c:8465 > 8465 process_main(esdp->x_reg_array, esdp->f_reg_array); > (gdb) print esdp > $8 = (ErtsSchedulerData *) 0x2b8244840200 > (gdb) print *esdp > $9 = {x_reg_array = 0x2b823e940200, f_reg_array = 0x2b823e942240, > timer_wheel = 0x2b82450f5c80, > next_tmo_ref = 0x2b8245136120, timer_service = 0x2b8245176680, tid = > 47838514915072, erl_bits_state = { > byte_buf_ = 0x2b823ad81058 "", byte_buf_len_ = 1, erts_current_bin_ = > 0x2b832e60c688 "\n\274\362T", > erts_bin_offset_ = 32, erts_writable_bin_ = 0}, match_pseudo_process = > 0x2b823bec7c78, free_process = 0x0, > thr_progress_data = {id = 1, is_managed = 1, is_blocking = 0, > is_temporary = 0, wakeup_request = {5707836, 5707869, > 5707862, 5707859}, leader = 0, active = 1, confirmed = 5707879, > leader_state = {next = 5707875, > current = 18446744073709551615, chk_next_ix = 2, umrefc_ix = > {current = 0, waiting = -1}}}, ssi = 0x2b823be7e680, > current_process = 0x2b82431401d8, type = ERTS_SCHED_NORMAL, no = 1, > dirty_no = 0, flxctr_slot_no = 1, > current_nif = 0x0, dirty_shadow_process = 0x0, current_port = 0x0, > run_queue = 0x2b823be77ec0, virtual_reds = 0, > cpu_id = -1, aux_work_data = {sched_id = 1, esdp = 0x2b8244840200, ssi = > 0x2b823be7e680, current_thr_prgr = 5707878, > latest_wakeup = 5707869, misc = {ix = 0, thr_prgr = > 18446744073709551615}, dd = {thr_prgr = 5707869}, cncld_tmrs = { > thr_prgr = 5707146}, later_op = {thr_prgr = 5707880, size = 65384, > first = 0x2b832e8d76c8, > last = 0x2b832e8d76c8}, async_ready = {need_thr_prgr = 0, thr_prgr = > 18446744073709551615, > queue = 0x2b8245059880}, delayed_wakeup = {next = > 18446744073709551615, sched2jix = 0x2b82443650c8, jix = -1, > job = 0x2b8244364f00}, yield = {alcu_blockscan = {current = 0x0, > last = 0x0}, ets_all = {ongoing = 0x0, > hfrag = 0x0, tab = 0x0, queue = 0x0}}, debug = {wait_completed = > {flags = 0, callback = 0, arg = 0x0}}}, > atom_cache_map = {hdr_sz = -1, sz = 0, long_atoms = 0, cix = {0 2048 times>}, cache = {{atom = 0, > iix = -1} }}, last_monotonic_time = > 54631431230926, check_time_reds = 3137, thr_id = 1, > unique = 251, ref = 1016430404454740281, alloc_data = {deallctr = {0x0, > 0x0, 0x0, 0x2b81f77a9200, 0x2b823ad37200, > 0x2b823bdaa200, 0x2b823e8d5200, 0x2b8240f13200, 0x2b8242ff9200, 0x0, > 0x0, 0x2b823fea0200, 0x2b8241f86200, 0x0}, > pref_ix = {0, -1, 1, 1, 1, 1, 1, 1, 1, -1, -1, 1, 1, -1}, flist_ix = > {0 }, pre_alc_ix = 0}, io = { > out = 21255996115, in = 21476723837}, pending_signal = {sig = 0x0, to > = 0}, reductions = 1006796378, > sched_wall_time = {u = {mod = {counter = 0}, need = 0}, enabled = 0, > start = 0, working = {total = 0, start = 0}}, > gc_info = {reclaimed = 775481964, garbage_cols = 680476}, > nosuspend_port_task_handle = {counter = 0}, ets_tables = { > count = {counter = 0}, clist = 0x0}} > (gdb) print esdp->x_reg_array > $10 = (Eterm *) 0x2b823e940200 > (gdb) print *esdp->x_reg_array > $11 = 2522015978211937347 > (gdb) print esdp->f_reg_array > $12 = (FloatDef *) 0x2b823e942240 > (gdb) print *esdp->f_reg_array > $13 = {fd = 0.002545, fb = "\323\023\226x@\331d?", fs = {5075, 30870, > 55616, 16228}, fw = {2023101395, 1063573824}, > fdw = 4568014792984761299} > > frame 2 is in already in pthread/ethread.c > > > -----Original Message----- > From: erlang-questions [mailto:erlang-questions-bounces@REDACTED] > On Behalf Of Eckard Brauer > Sent: Saturday, November 02, 2019 10:01 AM > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Segfault with Erlang R22 > > It's a few years ago, but IIRC either "print *c_p" or "print > *((Process*) c_p)". Problem would probably be that the processor > already left the stack frame where c_p is valid. > > You can do "info stack" at this point and select the frame with "frame > <#>" to try it again. If you're a little familiar with assembly > language, you can even have a look at "disassemble
" or > "disassemble function" to get an idea of where values are at what point > in the instruction/processing flow - sometimes this helps too. > > I'd investigate starting with frame 2 here, as all frames below are > already in libpthread. > > Hope that helps a bit... > > Eckard > > > Am Fri, 1 Nov 2019 18:22:18 +0000 > schrieb "Bekes, Andras G" : > > > Program terminated with signal 11, Segmentation fault. > > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > > > > 5252 c_p->seq_trace_lastcnt = > > unsigned_val(SEQ_TRACE_TOKEN_SERIAL(c_p)); Missing separate > > debuginfos, use: debuginfo-install glibc-2.12-1.212.el6_10.3.x86_64 > > (gdb) bt > > #0 process_main (x_reg_array=0x20002, f_reg_array=0x2) at > > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:5252 > > #1 0x00000000004641a4 in sched_thread_func (vesdp=0x2b8244840200) at > > beam/erl_process.c:8465 > > #2 0x000000000069262a in thr_wrapper (vtwd=) > > at pthread/ethread.c:118 > > #3 0x00002b81f80f7dd5 in _L_unlock_48 () from /lib64/libpthread.so.0 > > #4 0x00002b81f80f5eb3 in __find_thread_by_id () from > > /lib64/libpthread.so.0 > > #5 0x0000000000000000 in ?? () > > (gdb) > > > > I am not sure how to " print the c_p parameter via its raw value > > (print *(Process*)0x.....)". Where should I take the value 0x..... > > from? > > > > -----Original Message----- > > From: Mikael Pettersson [mailto:mikpelinux@REDACTED] > > Sent: Thursday, October 24, 2019 7:10 PM > > To: Bekes, Andras G (IST) > > Cc: Erlang Questions > > Subject: Re: [erlang-questions] Segfault with Erlang R22 > > > > On Thu, Oct 24, 2019 at 4:57 PM Bekes, Andras G > > wrote: > > [...] > > > > I'd try to get a backtrace (bt command in gdb) from the crashed > > thread, then maybe print > > the c_p parameter via its raw value (print *(Process*)0x.....) if gdb > > insists that the value > > is optimized out. > > > > /Mikael > > > > [...] > > [...] > > [...] > > > > > > -- > Wir haften nicht f?r die korrekte Funktion der in dieser eMail > enthaltenen Viren. We are not liable for correct function of the > viruses in this email! :) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Fri Nov 8 16:29:44 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Fri, 8 Nov 2019 16:29:44 +0100 Subject: mnesia initialization In-Reply-To: References: Message-ID: thank you all, but it seems that my problem is connected to "Cannot get connection id for node" when mnesia starts I'm having a node at name@REDACTED could that be the issue?? On Fri, Nov 8, 2019 at 6:46 AM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > Make sure to: > > 1. Start your erlang node with -mnesia dir > "/path/to/where/you/want/to/put/mnesia" > 2. Create the schema first > 3. start mnesia by using command mnesia:start() > 4. The create your tables > > >> I have a db_init function that initializes db so that if tables do not > exist they are created (ram with disc_copies) > If you want to do a table existence detection, you can do like the > following: > > case catch mnesia:table(your_table_name, version) of > {{_, _}, []} > -> table_exists; > > Fail when Fail =:= > {'EXIT',{aborted,{no_exists,your_table_name,version}}}; Fail =:= > {aborted,{no_exists,your_table_name,version}} > -> %% create your your_table_name table > end > > Also, as i stated above, you have to create your schema first then start > mnesia application. Example: > > case mnesia:create_schema([node()]) of > {error,{_,{already_exists,_}}} -> > io:format("Schema Already exists~n"); > > ok -> > io:format("Schema Created~n") > end, > > application:ensure_started(mnesia), > > %% then init your tables here > %% > %% > . > > Please read http://erlang.org/doc/apps/mnesia/Mnesia_chap2.html > > >> However half of the time it destroys old table returning {atomic, ok} > and another half it returns with already_exists > I'm not sure with this. But maybe i think your schema location was not > consistent > > I suggest you to use rebar3 to manage your project. > > > > > > > > > > > > > > > Pada tanggal Jum, 8 Nov 2019 pukul 07.57 Karlo Kuna > menulis: > >> Hi, >> >> I have problem with mnesia initialization, >> >> I have a db_init function that initializes db so that if tables do not >> exist they are created (ram with disc_copies). For example: >> >> db_init() -> >> mnesia:create_table(some_record,[{attributes, record_info(fields, >> some_record)}, >> {type, >> bag}, {disc_copies, [node()]}]). >> >> I'm running this function on the app start to make sure the table exists >> and is ready. >> However half of the time it destroys old table returning {atomic, ok} and >> another half it returns with already_exists. Also, this happens in the >> regular fashion, if I have n starts of application, even starts would >> destroy data and even wold keeps old data. I know that I am missing >> something probably obvious here but if anyone has any insights on why is >> this happening and how to fix it I would greatly appreciate it. >> >> Thank You, >> Karlo. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Fri Nov 8 18:59:16 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sat, 9 Nov 2019 00:59:16 +0700 Subject: mnesia initialization In-Reply-To: References: Message-ID: Hi, >> but it seems that my problem is connected to "Cannot get connection id for node" when mnesia starts How you start mnesia? It's so rare to happen. it looks like auto-connect things. >> I'm having a node at name@REDACTED could that be the issue?? That's strange, if you run node() in your erlang node shell you should get nonode@REDACTED Except, if you run your erlang node with command: erl -sname name@REDACTED Try to start your erlang node with better node name such: erl -sname node1@REDACTED where myhost mapped in `/etc/hosts` as 127.0.0.1 So then combined with your mnesia needs, you should run erl -sname node1@REDACTED -mnesia dir "/home/yourusername/MY_MNESIA_PUT_HERE". >> However half of the time it destroys old table returning {atomic, ok} and another half it returns with already_exists. I think it's because you set your mnesia's directory at `/tmp/*` then your `/tmp/*` got auto-cleaned by the OS. Pada tanggal Jum, 8 Nov 2019 pukul 22.29 Karlo Kuna menulis: > thank you all, > > but it seems that my problem is connected to "Cannot get connection id for > node" when mnesia starts > I'm having a node at name@REDACTED could that be the issue?? > > On Fri, Nov 8, 2019 at 6:46 AM I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > >> Hi, >> >> Make sure to: >> >> 1. Start your erlang node with -mnesia dir >> "/path/to/where/you/want/to/put/mnesia" >> 2. Create the schema first >> 3. start mnesia by using command mnesia:start() >> 4. The create your tables >> >> >> I have a db_init function that initializes db so that if tables do not >> exist they are created (ram with disc_copies) >> If you want to do a table existence detection, you can do like the >> following: >> >> case catch mnesia:table(your_table_name, version) of >> {{_, _}, []} >> -> table_exists; >> >> Fail when Fail =:= >> {'EXIT',{aborted,{no_exists,your_table_name,version}}}; Fail =:= >> {aborted,{no_exists,your_table_name,version}} >> -> %% create your your_table_name table >> end >> >> Also, as i stated above, you have to create your schema first then start >> mnesia application. Example: >> >> case mnesia:create_schema([node()]) of >> {error,{_,{already_exists,_}}} -> >> io:format("Schema Already exists~n"); >> >> ok -> >> io:format("Schema Created~n") >> end, >> >> application:ensure_started(mnesia), >> >> %% then init your tables here >> %% >> %% >> . >> >> Please read http://erlang.org/doc/apps/mnesia/Mnesia_chap2.html >> >> >> However half of the time it destroys old table returning {atomic, ok} >> and another half it returns with already_exists >> I'm not sure with this. But maybe i think your schema location was not >> consistent >> >> I suggest you to use rebar3 to manage your project. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Pada tanggal Jum, 8 Nov 2019 pukul 07.57 Karlo Kuna >> menulis: >> >>> Hi, >>> >>> I have problem with mnesia initialization, >>> >>> I have a db_init function that initializes db so that if tables do not >>> exist they are created (ram with disc_copies). For example: >>> >>> db_init() -> >>> mnesia:create_table(some_record,[{attributes, record_info(fields, >>> some_record)}, >>> {type, >>> bag}, {disc_copies, [node()]}]). >>> >>> I'm running this function on the app start to make sure the table exists >>> and is ready. >>> However half of the time it destroys old table returning {atomic, ok} >>> and another half it returns with already_exists. Also, this happens in the >>> regular fashion, if I have n starts of application, even starts would >>> destroy data and even wold keeps old data. I know that I am missing >>> something probably obvious here but if anyone has any insights on why is >>> this happening and how to fix it I would greatly appreciate it. >>> >>> Thank You, >>> Karlo. >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Fri Nov 8 19:29:10 2019 From: kuna.prime@REDACTED (Karlo Kuna) Date: Fri, 8 Nov 2019 19:29:10 +0100 Subject: mnesia initialization In-Reply-To: References: Message-ID: Thank you once again, I really don't understand why it is happening but I have found a solution. i am initializing node simply by -name name and then everything works as expected and cannot get connection error goes away On Fri, Nov 8, 2019 at 6:59 PM I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > >> but it seems that my problem is connected to "Cannot get connection id > for node" when mnesia starts > How you start mnesia? It's so rare to happen. it looks like auto-connect > things. > > >> I'm having a node at name@REDACTED could that be the issue?? > That's strange, if you run node() in your erlang node shell you should > get nonode@REDACTED Except, if you run your erlang node with command: erl > -sname name@REDACTED > > Try to start your erlang node with better node name such: erl -sname > node1@REDACTED where myhost mapped in `/etc/hosts` as 127.0.0.1 > > So then combined with your mnesia needs, you should run erl -sname > node1@REDACTED -mnesia dir "/home/yourusername/MY_MNESIA_PUT_HERE". > > >> However half of the time it destroys old table returning {atomic, ok} > and another half it returns with already_exists. > I think it's because you set your mnesia's directory at `/tmp/*` then your > `/tmp/*` got auto-cleaned by the OS. > > > > Pada tanggal Jum, 8 Nov 2019 pukul 22.29 Karlo Kuna > menulis: > >> thank you all, >> >> but it seems that my problem is connected to "Cannot get connection id >> for node" when mnesia starts >> I'm having a node at name@REDACTED could that be the issue?? >> >> On Fri, Nov 8, 2019 at 6:46 AM I Gusti Ngurah Oka Prinarjaya < >> okaprinarjaya@REDACTED> wrote: >> >>> Hi, >>> >>> Make sure to: >>> >>> 1. Start your erlang node with -mnesia dir >>> "/path/to/where/you/want/to/put/mnesia" >>> 2. Create the schema first >>> 3. start mnesia by using command mnesia:start() >>> 4. The create your tables >>> >>> >> I have a db_init function that initializes db so that if tables do >>> not exist they are created (ram with disc_copies) >>> If you want to do a table existence detection, you can do like the >>> following: >>> >>> case catch mnesia:table(your_table_name, version) of >>> {{_, _}, []} >>> -> table_exists; >>> >>> Fail when Fail =:= >>> {'EXIT',{aborted,{no_exists,your_table_name,version}}}; Fail =:= >>> {aborted,{no_exists,your_table_name,version}} >>> -> %% create your your_table_name table >>> end >>> >>> Also, as i stated above, you have to create your schema first then start >>> mnesia application. Example: >>> >>> case mnesia:create_schema([node()]) of >>> {error,{_,{already_exists,_}}} -> >>> io:format("Schema Already exists~n"); >>> >>> ok -> >>> io:format("Schema Created~n") >>> end, >>> >>> application:ensure_started(mnesia), >>> >>> %% then init your tables here >>> %% >>> %% >>> . >>> >>> Please read http://erlang.org/doc/apps/mnesia/Mnesia_chap2.html >>> >>> >> However half of the time it destroys old table returning {atomic, ok} >>> and another half it returns with already_exists >>> I'm not sure with this. But maybe i think your schema location was not >>> consistent >>> >>> I suggest you to use rebar3 to manage your project. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Pada tanggal Jum, 8 Nov 2019 pukul 07.57 Karlo Kuna < >>> kuna.prime@REDACTED> menulis: >>> >>>> Hi, >>>> >>>> I have problem with mnesia initialization, >>>> >>>> I have a db_init function that initializes db so that if tables do not >>>> exist they are created (ram with disc_copies). For example: >>>> >>>> db_init() -> >>>> mnesia:create_table(some_record,[{attributes, record_info(fields, >>>> some_record)}, >>>> {type, >>>> bag}, {disc_copies, [node()]}]). >>>> >>>> I'm running this function on the app start to make sure the table >>>> exists and is ready. >>>> However half of the time it destroys old table returning {atomic, ok} >>>> and another half it returns with already_exists. Also, this happens in the >>>> regular fashion, if I have n starts of application, even starts would >>>> destroy data and even wold keeps old data. I know that I am missing >>>> something probably obvious here but if anyone has any insights on why is >>>> this happening and how to fix it I would greatly appreciate it. >>>> >>>> Thank You, >>>> Karlo. >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Nov 8 21:15:40 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 8 Nov 2019 21:15:40 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi! I think I am on to the problem, we have only whitebox tested the unorded chain functionality (we intend to create a blackbox but that is a bigger job as we need to find some introp software that can create such chains or create a simulation modle), so I am positive I found that we do not reach the code for sorting the chain. I will try to fix it next week. It is easier now that I at least I got a server that I can manually blackbox test with :) Regards Ingela Erlang/OTP Team - Ericsson AB Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin : > Hi! > > Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < > michaelviveros@REDACTED>: > >> Hi Ingela, >> >> Curtis' example server from his first message, hooks.glip.com, presents >> its certificates out-of-order. The correct order is Peer -> Intermediate CA >> 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root >> CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown >> CA" error. You can confirm this by running `openssl s_client -connect >> hooks.glip.com:443`. >> >> > Yes I agree that this is an out of order chain, in contrast to the > social.fluffel.io. I will look into it at work tomorrow. > > Regards Ingela Erlang/OTP Team - Ericsson AB > > > >> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield wrote: >> >>> Hi Ingela >>> >>> Thank you for your attention- perhaps Micheal can explain this better.. >>> >>> Sent from ProtonMail Mobile >>> >>> >>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >>> wrote: >>> >>> Hi! >>> >>> I tried this out and it is not out of order, it sends the peer cert >>> followed by the intermediate cert repeated, that is the chain looks like >>> [Peer, CA1, CA1]. >>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so >>> I suppose we need to add that. >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>> >>>> Hey, >>>> >>>> I confirm that out of order certs does not seems to be fixed, and it >>>> fails with 'Unknown CA' error: >>>> >>>> >>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>> {:error, >>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>> CA'}}} >>>> >>>> >>>> the only issue with this server TLS certificates is the chain order (CA >>>> is Letsencrypt): >>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>> >>>> >>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>> >>>> Hi! >>>> >>>> Just curious if there is an update on out of order certs. >>>> >>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>> root of id2 who is the root of id3, who is the root of id0. >>>> >>>> We can correct the out of order problem by including id1,id2,id3 certs >>>> in our chain. >>>> >>>> It would be nice to hear from the erlang maintainers around what kind of >>>> "out of order" erlang can handle nicely and if there is planned support >>>> for >>>> our case! >>>> >>>> Thank you again, >>>> >>>> Curtis. >>>> >>>> >>>> Sent through ProtonMail Encrypted Email >>>> Channel. >>>> >>>> >>>> ??????? Original Message ??????? >>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>> wrote: >>>> >>>> Hi! Thank you. >>>> >>>> >>>> I included the root cert in the example. The root cert is id1 in cert >>>> chain - this is evident in the other file. >>>> >>>> It seems because the root cert is out of order - the cert chain is >>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>> TLS1.2 >>>> >>>> >>>> Thank you for your consideration! >>>> >>>> >>>> Sent from ProtonMail Mobile >>>> >>>> >>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin >>>> wrote: >>>> >>>> >>>> Hi! >>>> >>>> "Unknown CA" means that you did not have the ROOT certificate of the >>>> chian in your "trusted store" (cacerts option). >>>> If you do not own the ROOT certificate you can not trust the chain. >>>> >>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>> >>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>> >: >>>> >>>> Dear Erlang Questions: >>>> >>>> >>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>> >>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>> for host hooks.glip.com. >>>> >>>> When we try to verify peer with the out of order cert >>>> chain we get 'Unknown CA'. >>>> >>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>> >>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>> mention that other care may need to be taken to ensure compatibility. >>>> >>>> Reproduce error: >>>> >>>> https://github.com/robotarmy/out-of-order-ssl >>>> >>>> Thank you, >>>> Curtis and Team DevEco >>>> >>>> >>>> >>>> >>>> Sent through ProtonMail Encrypted Email Channel. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From curtis@REDACTED Fri Nov 8 21:45:52 2019 From: curtis@REDACTED (Curtis J Schofield) Date: Fri, 08 Nov 2019 20:45:52 +0000 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Oh this is wonderful news!! Glad you were able to identify the code not reached !! Deeply appreciate your support and expertise! Best, Curtis Sent from ProtonMail Mobile On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin wrote: > Hi! > > I think I am on to the problem, we have only whitebox tested the unorded chain functionality (we intend to create a blackbox but that is a bigger job as we need to find some introp software that can create such chains or create a simulation modle), > so I am positive I found that we do not reach the code for sorting the chain. I will try to fix it next week. It is easier now that I at least I got a server that I can manually blackbox test with :) > > Regards Ingela Erlang/OTP Team - Ericsson AB > > Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin : > >> Hi! >> >> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros : >> >>> Hi Ingela, >>> >>> Curtis' example server from his first message, hooks.glip.com, presents its certificates out-of-order. The correct order is Peer -> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown CA" error. You can confirm this by running `openssl s_client -connect hooks.glip.com:443`. >> >> Yes I agree that this is an out of order chain, in contrast to the social.fluffel.io. I will look into it at work tomorrow. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield wrote: >>> >>>> Hi Ingela >>>> >>>> Thank you for your attention- perhaps Micheal can explain this better.. >>>> >>>> Sent from ProtonMail Mobile >>>> >>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin wrote: >>>> >>>>> Hi! >>>>> >>>>> I tried this out and it is not out of order, it sends the peer cert followed by the intermediate cert repeated, that is the chain looks like [Peer, CA1, CA1]. >>>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so I suppose we need to add that. >>>>> >>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>> >>>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>>> >>>>>> Hey, >>>>>> >>>>>> I confirm that out of order certs does not seems to be fixed, and it fails with 'Unknown CA' error: >>>>>> >>>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>>> {:error, >>>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown CA'}}} >>>>>> >>>>>> the only issue with this server TLS certificates is the chain order (CA is Letsencrypt): https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>>> >>>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> Just curious if there is an update on out of order certs. >>>>>>> >>>>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>>>> root of id2 who is the root of id3, who is the root of id0. >>>>>>> >>>>>>> We can correct the out of order problem by including id1,id2,id3 certs >>>>>>> in our chain. >>>>>>> >>>>>>> It would be nice to hear from the erlang maintainers around what kind of >>>>>>> "out of order" erlang can handle nicely and if there is planned support for >>>>>>> our case! >>>>>>> >>>>>>> Thank you again, >>>>>>> >>>>>>> Curtis. >>>>>>> >>>>>>> Sent through [ProtonMail](https://protonmail.com) Encrypted Email Channel. >>>>>>> >>>>>>> ??????? Original Message ??????? >>>>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield wrote: >>>>>>> >>>>>>>> Hi! Thank you. >>>>>>>> >>>>>>>> I included the root cert in the example. The root cert is id1 in cert chain - this is evident in the other file. >>>>>>>> >>>>>>>> It seems because the root cert is out of order - the cert chain is invalid - IIRC this may be true for tls1.2 - however the negotiation is at TLS1.2 >>>>>>>> >>>>>>>> Thank you for your consideration! >>>>>>>> >>>>>>>> Sent from ProtonMail Mobile >>>>>>>> >>>>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin wrote: >>>>>>>> >>>>>>>>> Hi! >>>>>>>>> >>>>>>>>> "Unknown CA" means that you did not have the ROOT certificate of the chian in your "trusted store" (cacerts option). >>>>>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>>>>> >>>>>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>>>>> >>>>>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield : >>>>>>>>> >>>>>>>>>> Dear Erlang Questions: >>>>>>>>>> >>>>>>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>>>>>> >>>>>>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>>>>>> for host hooks.glip.com. >>>>>>>>>> >>>>>>>>>> When we try to verify peer with the out of order cert >>>>>>>>>> chain we get 'Unknown CA'. >>>>>>>>>> >>>>>>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>>>>>> >>>>>>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>>>>>> mention that other care may need to be taken to ensure compatibility. >>>>>>>>>> >>>>>>>>>> Reproduce error: >>>>>>>>>> >>>>>>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>>>>>> >>>>>>>>>> Thank you, >>>>>>>>>> Curtis and Team DevEco >>>>>>>>>> >>>>>>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> erlang-questions mailing list >>>>>>>>>> erlang-questions@REDACTED >>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Nov 8 23:25:22 2019 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 9 Nov 2019 01:25:22 +0300 Subject: multiarch erlang installation Message-ID: We in Flussonic have our own hardware video transcoder right now and it is using our erlang-based flussonic on intel server and on arm nodes. ARM node mounts nfs root from master server. Right now I'm changing erlang installation so that we can use same folder from intel and from arm. It happened that almost all erlang nifs are ready for this: they are trying to load so from system_arch subfolder. Looks like I'm not the first for this idea! However it was a bit tricky to move binary files. Right now I've ended up with multiarch.sh loader: #!/bin/sh ARCH=`uname -i` APP=`basename $0` exec /usr/lib/erlang/bin/${ARCH}-linux-gnu/${APP} $* Also I have to patch crypto compilation Makefile in runtime: RUN sed -i'' 's|SSL_LIBDIR = /usr/lib|SSL_LIBDIR = /usr/lib/aarch64-linux-gnu|' lib/crypto/c_src/aarch64-unknown-linux-gnu/Makefile Has anybody else done such things? -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Nov 9 00:14:24 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 9 Nov 2019 00:14:24 +0100 Subject: snit (SNI Termination Library) to replace Nginx Message-ID: Hi guys Anyone familiar with snit? https://github.com/heroku/snit We?re facing a performance issue with Nginx used as TLS Termination. Nginx is in front of our two Erlang webapps. Both running on the same machine, and both based on Cowboy 2.7.0. The problem: [1] directly accessing the two webapps (plain HTTP) is fast enough for us, and Cowboy is doing just great. [2] accessing any of the two apps with Nginx (HTTPS) is 3x-5x slower than in [1] We selected Nginx for its ability to hide our apps, and be able to access them both on port 443 (default HTTPS). Our Nginx config is pretty simple, tuned for SSL/TLS. ______________________________________________ server { listen 443 ssl; server_name app1.acme.com; # the 2nd webapp is running on: app2.acme.com ssl on; ssl_certificate /etc/nginx/certs/app1/crt.pem; ssl_certificate_key /etc/nginx/certs/app1/key.pem; ssl_dhparam /etc/nginx/certs/app1/dh.pem; ssl_protocols TLSv1.2; ssl_prefer_server_ciphers on; ssl_ecdh_curve secp384r1 ssl_session_cache shared:SSL:50m; ssl_session_timeout 1d; ssl_session_tickets off; ssl_stapling on; ssl_stapling_verify on; resolver 8.8.8.8 8.8.4.4 valid=300s; resolver_timeout 5s; ssl_buffer_size 8k; keepalive_timeout 0; client_max_body_size 0; client_body_buffer_size 4m; client_header_timeout 300; client_body_timeout 300; client_header_buffer_size 1k; large_client_header_buffers 4 4k; location = /favicon.ico { access_log off; return 204; } location / { send_timeout 5; proxy_http_version 1.1; proxy_buffering off; proxy_request_buffering off; proxy_ignore_headers "Cache-Control" "Expires"; proxy_max_temp_file_size 30m; proxy_connect_timeout 300; proxy_read_timeout 300; proxy_send_timeout 300; proxy_intercept_errors off; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://127.0.0.1:2222; # the 2nd webapp has: proxy_pass http://127.0.0.1:3333; } } ______________________________________________ Can snit be used to replace Nginx? Help and suggestions appreciated. Best /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From karlsson.rm@REDACTED Sat Nov 9 03:58:48 2019 From: karlsson.rm@REDACTED (Mikael Karlsson) Date: Sat, 9 Nov 2019 03:58:48 +0100 Subject: snit (SNI Termination Library) to replace Nginx In-Reply-To: References: Message-ID: Hi, Did you try with proxy_buffering set to on, and/or changing the proxy_buffer_size? Regards Mikael Den l?r 9 nov. 2019 00:14Frank Muller skrev: > Hi guys > > Anyone familiar with snit? > https://github.com/heroku/snit > > We?re facing a performance issue with Nginx used as TLS Termination. > Nginx is in front of our two Erlang webapps. Both running on the same > machine, and both based on Cowboy 2.7.0. > > The problem: > [1] directly accessing the two webapps (plain HTTP) is fast enough for us, > and Cowboy is doing just great. > [2] accessing any of the two apps with Nginx (HTTPS) is 3x-5x slower than > in [1] > > We selected Nginx for its ability to hide our apps, and be able to access > them both on port 443 (default HTTPS). > > Our Nginx config is pretty simple, tuned for SSL/TLS. > ______________________________________________ > server { > listen 443 ssl; > > server_name app1.acme.com; # the 2nd webapp is running on: > app2.acme.com > > ssl on; > ssl_certificate /etc/nginx/certs/app1/crt.pem; > ssl_certificate_key /etc/nginx/certs/app1/key.pem; > ssl_dhparam /etc/nginx/certs/app1/dh.pem; > > ssl_protocols TLSv1.2; > > ssl_prefer_server_ciphers on; > > ssl_ecdh_curve > secp384r1 > > ssl_session_cache shared:SSL:50m; > ssl_session_timeout 1d; > ssl_session_tickets off; > > ssl_stapling on; > ssl_stapling_verify on; > > resolver 8.8.8.8 8.8.4.4 valid=300s; > resolver_timeout 5s; > > ssl_buffer_size 8k; > > keepalive_timeout 0; > > > client_max_body_size 0; > client_body_buffer_size 4m; > client_header_timeout 300; > client_body_timeout 300; > client_header_buffer_size 1k; > large_client_header_buffers 4 4k; > > location = /favicon.ico { > access_log off; > return 204; > } > > location / { > send_timeout 5; > > proxy_http_version 1.1; > proxy_buffering off; > proxy_request_buffering off; > proxy_ignore_headers "Cache-Control" "Expires"; > proxy_max_temp_file_size 30m; > proxy_connect_timeout 300; > proxy_read_timeout 300; > proxy_send_timeout 300; > proxy_intercept_errors off; > > proxy_set_header X-Real-IP $remote_addr; > proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; > > proxy_pass http://127.0.0.1:2222; # the 2nd webapp has: > proxy_pass http://127.0.0.1:3333; > } > } > ______________________________________________ > > Can snit be used to replace Nginx? > Help and suggestions appreciated. > > Best > /Frank > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Nov 9 08:22:04 2019 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 9 Nov 2019 08:22:04 +0100 Subject: snit (SNI Termination Library) to replace Nginx In-Reply-To: References: Message-ID: Hi Mikael We mainly upload large files (20mB to 100mB) to our two webapps behind Nginx. And yes, we tried these 2 options but they dont help in this situation. In our case, we are completely disabling buffering on Nginx (a feature introduced in version Nginx 1.7.3 taken from its fork Tengine at Taobao: https://tengine.taobao.org). Disabling buffer was a big win, but still much slower than direct HTTP access via cowboy. How did we find out that Nginx was the culprit? Simply by testing with another TLS termination proxy called Hitch (from Varnish): https://hitch-tls.org Hitch is 1.5x - 2x slower than Cowboy. Unfortunately it only supports one upstream backend server at a time. Thus, we can?t serve our two webapps on port 443. Another constraint is that our two webapps has to run on the same host (a customer?s requirement). Finally the system is not even under load. Maximum of 10 files upload per hour. Forgot to mention our config: 1. Erlang 22.1.6 2. Linux kernel 4.15.0.66 / Ubuntu LTS 18.10 x86_64 3. Physical machine: 32gB of RAM, 8-Cores Intel Xeon CPU E3-1270 v6@REDACTED 4. Nginx V1.14.0 5. Sysctl tuned by our engineers for handle fast TCP connections with enough open files limits (ulimit -n: 200000) /Frank Le sam. 9 nov. 2019 ? 03:58, Mikael Karlsson a ?crit : > Hi, > Did you try with proxy_buffering set to on, and/or changing the > proxy_buffer_size? > Regards Mikael > > > Den l?r 9 nov. 2019 00:14Frank Muller skrev: > >> Hi guys >> >> Anyone familiar with snit? >> https://github.com/heroku/snit >> >> We?re facing a performance issue with Nginx used as TLS Termination. >> Nginx is in front of our two Erlang webapps. Both running on the same >> machine, and both based on Cowboy 2.7.0. >> >> The problem: >> [1] directly accessing the two webapps (plain HTTP) is fast enough for >> us, and Cowboy is doing just great. >> [2] accessing any of the two apps with Nginx (HTTPS) is 3x-5x slower >> than in [1] >> >> We selected Nginx for its ability to hide our apps, and be able to access >> them both on port 443 (default HTTPS). >> >> Our Nginx config is pretty simple, tuned for SSL/TLS. >> ______________________________________________ >> server { >> listen 443 ssl; >> >> server_name app1.acme.com; # the 2nd webapp is running on: >> app2.acme.com >> >> ssl on; >> ssl_certificate /etc/nginx/certs/app1/crt.pem; >> ssl_certificate_key /etc/nginx/certs/app1/key.pem; >> ssl_dhparam /etc/nginx/certs/app1/dh.pem; >> >> ssl_protocols TLSv1.2; >> >> ssl_prefer_server_ciphers on; >> >> ssl_ecdh_curve >> secp384r1 >> >> ssl_session_cache shared:SSL:50m; >> ssl_session_timeout 1d; >> ssl_session_tickets off; >> >> ssl_stapling on; >> ssl_stapling_verify on; >> >> resolver 8.8.8.8 8.8.4.4 valid=300s; >> resolver_timeout 5s; >> >> ssl_buffer_size 8k; >> >> keepalive_timeout 0; >> >> >> client_max_body_size 0; >> client_body_buffer_size 4m; >> client_header_timeout 300; >> client_body_timeout 300; >> client_header_buffer_size 1k; >> large_client_header_buffers 4 4k; >> >> location = /favicon.ico { >> access_log off; >> return 204; >> } >> >> location / { >> send_timeout 5; >> >> proxy_http_version 1.1; >> proxy_buffering off; >> proxy_request_buffering off; >> proxy_ignore_headers "Cache-Control" "Expires"; >> proxy_max_temp_file_size 30m; >> proxy_connect_timeout 300; >> proxy_read_timeout 300; >> proxy_send_timeout 300; >> proxy_intercept_errors off; >> >> proxy_set_header X-Real-IP $remote_addr; >> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; >> >> proxy_pass http://127.0.0.1:2222; # the 2nd webapp has: >> proxy_pass http://127.0.0.1:3333; >> } >> } >> ______________________________________________ >> >> Can snit be used to replace Nginx? >> Help and suggestions appreciated. >> >> Best >> /Frank >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sat Nov 9 13:43:52 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sat, 9 Nov 2019 19:43:52 +0700 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) Message-ID: Hi, I need to know the best practice for Erlang's process design to become a website downloader. I don't need heavy parsing the website like what a scrapper do. Maybe i only need to parse url's . What had just come to my mind was create N number of Erlang's process under a supervisor. N is number of url found in a website's pages. But i'm not sure that's a good design. So i need recommendations from you who have experience on it. Thank you, I appreciate all of your time and attention -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Sat Nov 9 17:30:01 2019 From: dch@REDACTED (Dave Cottlehuber) Date: Sat, 09 Nov 2019 16:30:01 +0000 Subject: snit (SNI Termination Library) to replace Nginx In-Reply-To: References: Message-ID: <1948933d-6c44-4779-80f9-70a88883bb4b@www.fastmail.com> On Sat, 9 Nov 2019, at 07:22, Frank Muller wrote: > We mainly upload large files (20mB to 100mB) to our two webapps behind Nginx. > ssl_prefer_server_ciphers on; > ssl_ecdh_curve > secp384r1 TLDR: - use TLS1.3 if you can - most of the decisions have been made for you - ensure your cipher choice is hardware accelerated in your openssl - look at actual network traffic to see if there's any issues there - no easy answers, benchmark your setup I hope this helps point you in the right direction. "SSL/TLS accounts for less than 1% of the CPU load, less than 10 KB of memory per connection and less than 2% of network overhead." -- https://www.imperialviolet.org/2010/06/25/overclocking-ssl.html It should be possible to transfer traffic over TLS at rates significantly faster than what you're reporting, obviously. However, I would be surprised if nginx itself is the problem, given netflix can saturate their pipes with nginx, admittedly with a lot of tweaking [1], [2] and a custom FreeBSD build. I would first look to see if you can restrict your ciphers to provide better performance for your hardware, and highly recommend capturing data with tcpdump & wireshark to do some network level analysis. This will vary a lot depending on what control you have over client TLS capabilities [3], and if you have OpenSSL 1.1.x available, and perhaps http2 on clients also. Intel's notes from 2016 [4] show a noticeable difference between algorithms so you need to benchmark your load on your hardware. Personally, for TLS termination I prefer haproxy[5] but all of hitch, nginx, snit, haproxy should be able to achieve similar results.[6] is interesting but 2.x haproxy handles multiple processes itself. You can use https://www.ssllabs.com/ssltest/analyze.html or https://github.com/drwetter/testssl.sh to help validate protocol choices. Useful references: - https://istlsfastyet.com/ - https://hpbn.co/transport-layer-security-tls/ - https://www.haproxy.com/knowledge-base/ssl/ - https://www.feistyduck.com/books/bulletproof-ssl-and-tls/ get the ebook direct as amazon seems to have an out of date version [0]: https://www.imperialviolet.org/2010/06/25/overclocking-ssl.html [1]: https://openconnect.netflix.com/publications/asiabsd_2015_tls.pdf [2]: https://openconnect.netflix.com/publications/asiabsd_tls_improved.pdf [3]: https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_Ciphersuite [4]: https://software.intel.com/en-us/articles/accelerating-ssl-load-balancers-with-intel-xeon-e5-v4-processors [5]: https://www.haproxy.com/blog/haproxy-ssl-termination/ [6]: https://www.freecodecamp.org/news/how-we-fine-tuned-haproxy-to-achieve-2-000-000-concurrent-ssl-connections-d017e61a4d27/ [7]: https://www.ssllabs.com/ssltest/analyze.html [8]: https://github.com/drwetter/testssl.sh From mononcqc@REDACTED Sun Nov 10 03:47:48 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 9 Nov 2019 21:47:48 -0500 Subject: snit (SNI Termination Library) to replace Nginx In-Reply-To: References: Message-ID: <20191110024748.GA4698@ferdmbp.local> On 11/09, Frank Muller wrote: >Anyone familiar with snit? >https://github.com/heroku/snit > I'm one of the people who wrote it. >We?re facing a performance issue with Nginx used as TLS Termination. >Nginx is in front of our two Erlang webapps. Both running on the same >machine, and both based on Cowboy 2.7.0. > >The problem: >[1] directly accessing the two webapps (plain HTTP) is fast enough for us, >and Cowboy is doing just great. >[2] accessing any of the two apps with Nginx (HTTPS) is 3x-5x slower than >in [1] > Chances are you might have some tuning issues regarding TLS, If you nevertheless decide to benchmark snit and have it replace nginx, be aware that snit only handles TLS termination with SNI, and is not a general proxy; it was in fact a component that was used along with a router that was built on top of vegur (https://github.com/heroku/vegur) at Heroku. As such, it wouldn't replace what nginx does for you. If you decide to use snit, I would recommend using it to front the nginx instances you would have anyway, to see if it can terminate TLS faster. But nginx does other stuff, such as request buffering and offering forms of overload protection your app would no longer have without it (or another actual proxy server) offering protection. Another thing you can do if you find that snit gives you good performance is look with tcpdump or wireshark and see what TLS options, extensions, ciphersuites, and elliptic curves are being chosen. Most of the heavy cryptographic lifting is done by underlying C libraries, and until you get similar priorities chosen by both servers, the comparison will not be equitable. If the settings are the same, then you are starting to compare apples with apples and the higher-level code may be making a difference. Regards, Fred. From okaprinarjaya@REDACTED Sun Nov 10 09:33:42 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 10 Nov 2019 15:33:42 +0700 Subject: [erlang-questions] Matching IP address in socket module In-Reply-To: <82bf8e1d-9eec-ea4d-c2f3-58b322724b92@ericsson.com> References: <20191024100444.GA28388@tpccl.tty1.net> <20191024153351.GB28388@tpccl.tty1.net> <64c9307c-f8ec-3599-3d21-f86fb09c9770@ericsson.com> <497d87b2-f66c-6e45-5dfe-304c9eb27171@ninenines.eu> <82bf8e1d-9eec-ea4d-c2f3-58b322724b92@ericsson.com> Message-ID: Awesome! Can't wait gen_tcp rewritten with new socket module available in public and stable. And at which version will be available? at 22.X or 23 ? @Thomas Pircher facing another new problem with socket module. It looks like we hijacked his thread question ^^ :D Pada tanggal Sel, 29 Okt 2019 pukul 18.09 Micael Karlberg < micael.karlberg@REDACTED> menulis: > No, its on the todo list, there is actually a ticket for this. > I would be good for completeness, if nothing else. > > /BMK > > On 2019-10-29 11:33, pablo platt wrote: > > Micael, are there similar tests for udp? > > > > On Tue, Oct 29, 2019 at 12:12 PM Micael Karlberg < > micael.karlberg@REDACTED > > > wrote: > > > > Hi, > > > > Its not a secret :) You will find it in the emulator test catalog. > > The only problem is that it was never intended for public > "consumption" > > and is therefor not documented: > > > > socket_test_ttest_tcp_socket.erl (uses socket.erl) > > socket_test_ttest_tcp_gen.erl (uses gen_tcp.erl) > > > > The idea here is that they should have an "identical" interface (so > > the users should work with both transports). > > Obviously the "socket" is different (a gen_tcp socket is a port and > > a socket socket is a term()). > > > > The other socket_test_ttest_tcp* modules "can" be used to see how to > > use these... > > > > Also, there is work on making gen_tcp work with the new socket > module. > > This is ongoing. There is a lot of special options that gen_tcp/inet > > manages (packaging for instance). > > > > Regards, > > /BMK > > > > On 2019-10-29 10:54, Lo?c Hoguin wrote: > > > Hello, > > > > > > I would like to do a similar experiment with RabbitMQ. Would you > mind sharing the code that > > makes > > > 'socket' work similar to 'gen_tcp'? That may give me a head > start, even if the code is not > > complete. > > > > > > Thanks, > > > > > > On 25/10/2019 12:39, Micael Karlberg wrote: > > >> Hi, > > >> > > >> Its early days still, but the goal is definitely that it should > > >> be faster. > > >> > > >> Here is some figures from a (time) test tool which is part of the > > >> test suite (its basically a ping-pong case), both server and > client > > >> running on the same host (but in different VMs): > > >> > > >> Two transports: > > >> gen: gen_tcp > > >> sock: socket (tcp) > > >> > > >> The socket active mode is a simulation of gen_tcp active mode > > >> (no active = N). > > >> > > >> The tables are the exchange from the client side. > > >> > > >> With server side using gen (active = false): > > >> > > >> Transport Active Data > > >> gen false 10192 byte/ms, 154 > msgs/ms > > >> gen true 10383 byte/ms, 157 > msgs/ms > > >> gen once 6003 byte/ms, 90 > msgs/ms > > >> sock false 14050 byte/ms, 212 > msgs/ms > > >> sock true 14772 byte/ms, 223 > msgs/ms > > >> sock once 14050 byte/ms, 210 > msgs/ms > > >> > > >> > > >> With server side using gen (active = true): > > >> > > >> Transport Active Data > > >> gen false 9447 byte/ms, 143 > msgs/ms > > >> gen true 22345 byte/ms, 338 > msgs/ms > > >> gen once 5532 byte/ms, 83 > msgs/ms > > >> sock false 15316 byte/ms, 232 > msgs/ms > > >> sock true 23693 byte/ms, 358 > msgs/ms > > >> sock once 22068 byte/ms, 334 > msgs/ms > > >> > > >> > > >> With server side using sock (active = false, async = true): > > >> > > >> Transport Active Data > > >> gen false 11260 byte/ms, 170 > msgs/ms > > >> gen true 22273 byte/ms, 337 > msgs/ms > > >> gen once 7703 byte/ms, 116 > msgs/ms > > >> sock false 15211 byte/ms, 230 > msgs/ms > > >> sock true 24778 byte/ms, 375 > msgs/ms > > >> sock once 23086 byte/ms, 349 > msgs/ms > > >> > > >> > > >> With server side using sock (active = true, async = true): > > >> > > >> Transport Active Data > > >> gen false 11351 byte/ms, 171 > msgs/ms > > >> gen true 22469 byte/ms, 340 > msgs/ms > > >> gen once 7407 byte/ms, 112 > msgs/ms > > >> sock false 15484 byte/ms, 234 > msgs/ms > > >> sock true 24885 byte/ms, 377 > msgs/ms > > >> sock once 23570 byte/ms, 357 > msgs/ms > > >> > > >> > > >> There is of course a bit of overhead since the socket transport > > >> is trying to emulate (part of) gen_tcp. > > >> > > >> This is run on a Dell Precision T1700 running SLES 12 SP2 (not > that > > >> that effects the relative performance). > > >> Run with a snapshot from the maint branch. > > >> > > >> This is obviously not a real use case, but it can be a guideline. > > >> Also no UDP test (at the moment). > > >> > > >> /BMK > > >> > > >> On 2019-10-24 18:48, Frank Muller wrote: > > >>> Hi Thomas > > >>> > > >>> Is the socket module faster than gen_tcp/gen_udp in your case? > If yes, can you please share > > some > > >>> numbers. > > >>> > > >>> /Frank > > >>> > > >>> > case socket:recvfrom(Sock, 0, nowait) of > > >>> > {ok, {#{family := Domain, > > >>> > port := Port, > > >>> > addr := Addr}, Data}} -> > > >>> > . > > >>> > > >>> Hi Micael, Mark, > > >>> > > >>> thanks for your replies. The snippet above helped me > getting the record > > >>> matched. > > >>> > > >>> I'm quite impressed with the socket module, it seems to be > pretty > > >>> complete, at least for my application. > > >>> > > >>> Thanks, > > >>> Thomas > > >>> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sun Nov 10 11:07:04 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 10 Nov 2019 17:07:04 +0700 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) In-Reply-To: References: Message-ID: Hi, Anyone? Pada tanggal Sab, 9 Nov 2019 pukul 19.43 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > Hi, > > I need to know the best practice for Erlang's process design to become a > website downloader. I don't need heavy parsing the website like what a > scrapper do. Maybe i only need to parse url's . > > What had just come to my mind was create N number of Erlang's process > under a supervisor. N is number of url found in a > website's pages. But i'm not sure that's a good design. So i need > recommendations from you who have experience on it. > > Thank you, I appreciate all of your time and attention > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Sun Nov 10 18:12:25 2019 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 10 Nov 2019 17:12:25 +0000 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) In-Reply-To: References: Message-ID: Hi Gusti, I would suggest to create a pool of N processes and a queue of URLs to process. Every time a new URL is encountered it's added to the queue. Then a scheduler would pick up those URLs and distribute them across the pool of processes. I would not suggest to create a new process for each URL unless you can be sure it doesn't leant to an explosion of processes, i.e. that the number of URLs is limited. Greg On 10/11/2019 10:07, I Gusti Ngurah Oka Prinarjaya wrote: > Hi, > > Anyone? > > > > Pada tanggal Sab, 9 Nov 2019 pukul 19.43 I Gusti Ngurah Oka Prinarjaya > > menulis: > > Hi, > > I need to know the best practice for Erlang's process design to > become a website downloader. I don't need heavy parsing the > website like what a scrapper do. Maybe i only need to parse url's > . > > What had just come to my mind was create N number of Erlang's > process under a supervisor. N is number of url > found in a website's pages. But i'm not sure that's a good design. > So i need recommendations from you who have experience on it. > > Thank you, I appreciate all of? your time and attention > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Mon Nov 11 02:26:15 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Mon, 11 Nov 2019 08:26:15 +0700 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) In-Reply-To: References: Message-ID: Hi Grzegorz, Thank you for your suggestion. >> I would not suggest to create a new process for each URL unless you can be sure it doesn't leant to an explosion of processes Thanks for reminding me Pada tanggal Sen, 11 Nov 2019 pukul 00.12 Grzegorz Junka menulis: > Hi Gusti, > > I would suggest to create a pool of N processes and a queue of URLs to > process. Every time a new URL is encountered it's added to the queue. Then > a scheduler would pick up those URLs and distribute them across the pool of > processes. I would not suggest to create a new process for each URL unless > you can be sure it doesn't leant to an explosion of processes, i.e. that > the number of URLs is limited. > > Greg > > > On 10/11/2019 10:07, I Gusti Ngurah Oka Prinarjaya wrote: > > Hi, > > Anyone? > > > > Pada tanggal Sab, 9 Nov 2019 pukul 19.43 I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> menulis: > >> Hi, >> >> I need to know the best practice for Erlang's process design to become a >> website downloader. I don't need heavy parsing the website like what a >> scrapper do. Maybe i only need to parse url's . >> >> What had just come to my mind was create N number of Erlang's process >> under a supervisor. N is number of url found in a >> website's pages. But i'm not sure that's a good design. So i need >> recommendations from you who have experience on it. >> >> Thank you, I appreciate all of your time and attention >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Mon Nov 11 06:10:26 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 11 Nov 2019 06:10:26 +0100 (CET) Subject: Patch Package OTP 22.1.7 Released Message-ID: <20191111051026.2CC19254772@hel.cslab.ericsson.net> Patch Package: OTP 22.1.7 Git Tag: OTP-22.1.7 Date: 2019-11-11 Trouble Report Id: OTP-16193, OTP-16224, OTP-16259, OTP-16265 Seq num: ERIERL-436, ERL-1044, ERL-1064 System: OTP Release: 22 Application: compiler-7.4.9, erts-10.5.5 Predecessor: OTP 22.1.6 Check out the git tag OTP-22.1.7, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.4.9 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.4.9 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16259 Application(s): compiler Related Id(s): ERIERL-436 Fixed a performance bug that caused repeated matches of large records to take a very long time to compile. Full runtime dependencies of compiler-7.4.9: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-10.5.5 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.5.5 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16193 Application(s): erts A literal area could prematurely be released before all uses of it had been removed. This occurred either when a terminating process had a complex exit reason referring to a literal that concurrently was removed, or when a terminating process continued executing a dirty NIF accessing a literal (via the heap) that concurrently was removed. OTP-16224 Application(s): erts Related Id(s): ERL-1044 Fix bug causing VM crash due to memory corruption of distribution entry. Probability of crash increases if Erlang distribution is frequently disconnected and reestablished towards same node names. Bug exists since OTP-21.0. OTP-16265 Application(s): erts Related Id(s): ERL-1064 Fixed bug causing crash of VM built with configuration --enable--sharing-preserving. Provoked when a sent message contains both a bit string and the heap binary (< 65 bytes) which the bit string was matched from. Bug exists since OTP-19.0 but has seen to be easier to provoke since OTP-22.1. Full runtime dependencies of erts-10.5.5: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From ostinelli@REDACTED Mon Nov 11 09:56:04 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 11 Nov 2019 09:56:04 +0100 Subject: Know if process is monitored Message-ID: All, Is there a way to know if a process is monitored after a call to erlang:monitor/2, end even better get the references() after that? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Mon Nov 11 10:23:46 2019 From: freeakk@REDACTED (Michael Uvarov) Date: Mon, 11 Nov 2019 10:23:46 +0100 Subject: Know if process is monitored In-Reply-To: References: Message-ID: erlang:process_info(PID, monitored_by) Returns a list of pids who monitor process with PID Pid. On Mon, 11 Nov 2019, 09:56 Roberto Ostinelli, wrote: > All, > Is there a way to know if a process is monitored after a call to > erlang:monitor/2, end even better get the references() after that? > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon Nov 11 11:16:59 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 11 Nov 2019 11:16:59 +0100 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Hi! I have a patch that solves the problem. I have pushed it to my gitrepo https://github.com/IngelaAndin/otp/tree/ingela/ssl/unorded-chain I had to set the options {depth, 2} and {customize_hostname_check, [{match_fun, CustomFun}] where CustomFun = public_key:pkix_verify_hostname_match_fun(https) as the peer certificate is a wildcard cert. Regard Ingela Erlang/OTP team Den fre 8 nov. 2019 kl 21:46 skrev Curtis J Schofield : > Oh this is wonderful news!! Glad you were able to identify the code not > reached !! > > Deeply appreciate your support and expertise! > > Best, > Curtis > > Sent from ProtonMail Mobile > > > On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin > wrote: > > Hi! > > I think I am on to the problem, we have only whitebox tested the unorded > chain functionality (we intend to create a blackbox but that is a bigger > job as we need to find some introp software that can create such chains or > create a simulation modle), > so I am positive I found that we do not reach the code for sorting the > chain. I will try to fix it next week. It is easier now that I at least I > got a server that I can manually blackbox test with :) > > Regards Ingela Erlang/OTP Team - Ericsson AB > > Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin : > >> Hi! >> >> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < >> michaelviveros@REDACTED>: >> >>> Hi Ingela, >>> >>> Curtis' example server from his first message, hooks.glip.com, presents >>> its certificates out-of-order. The correct order is Peer -> Intermediate CA >>> 1 - > Intermediate CA 2 -> Root CA but they get presented as Peer -> Root >>> CA -> Intermediate CA 2 -> Intermediate CA 1 and this returns the "Unknown >>> CA" error. You can confirm this by running `openssl s_client -connect >>> hooks.glip.com:443`. >>> >>> >> Yes I agree that this is an out of order chain, in contrast to the >> social.fluffel.io. I will look into it at work tomorrow. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> >> >>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield >>> wrote: >>> >>>> Hi Ingela >>>> >>>> Thank you for your attention- perhaps Micheal can explain this better.. >>>> >>>> Sent from ProtonMail Mobile >>>> >>>> >>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >>>> wrote: >>>> >>>> Hi! >>>> >>>> I tried this out and it is not out of order, it sends the peer cert >>>> followed by the intermediate cert repeated, that is the chain looks like >>>> [Peer, CA1, CA1]. >>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, so >>>> I suppose we need to add that. >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>> >>>>> Hey, >>>>> >>>>> I confirm that out of order certs does not seems to be fixed, and it >>>>> fails with 'Unknown CA' error: >>>>> >>>>> >>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>> {:error, >>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>>> CA'}}} >>>>> >>>>> >>>>> the only issue with this server TLS certificates is the chain order >>>>> (CA is Letsencrypt): >>>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>> >>>>> >>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>> >>>>> Hi! >>>>> >>>>> Just curious if there is an update on out of order certs. >>>>> >>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>> root of id2 who is the root of id3, who is the root of id0. >>>>> >>>>> We can correct the out of order problem by including id1,id2,id3 certs >>>>> in our chain. >>>>> >>>>> It would be nice to hear from the erlang maintainers around what kind >>>>> of >>>>> "out of order" erlang can handle nicely and if there is planned >>>>> support for >>>>> our case! >>>>> >>>>> Thank you again, >>>>> >>>>> Curtis. >>>>> >>>>> >>>>> Sent through ProtonMail Encrypted Email >>>>> Channel. >>>>> >>>>> >>>>> ??????? Original Message ??????? >>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>>> wrote: >>>>> >>>>> Hi! Thank you. >>>>> >>>>> >>>>> I included the root cert in the example. The root cert is id1 in cert >>>>> chain - this is evident in the other file. >>>>> >>>>> It seems because the root cert is out of order - the cert chain is >>>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>>> TLS1.2 >>>>> >>>>> >>>>> Thank you for your consideration! >>>>> >>>>> >>>>> Sent from ProtonMail Mobile >>>>> >>>>> >>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin >>>>> wrote: >>>>> >>>>> >>>>> Hi! >>>>> >>>>> "Unknown CA" means that you did not have the ROOT certificate of the >>>>> chian in your "trusted store" (cacerts option). >>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>> >>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>> >>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>>> >: >>>>> >>>>> Dear Erlang Questions: >>>>> >>>>> >>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>> >>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>> for host hooks.glip.com. >>>>> >>>>> When we try to verify peer with the out of order cert >>>>> chain we get 'Unknown CA'. >>>>> >>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>> >>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>> mention that other care may need to be taken to ensure compatibility. >>>>> >>>>> Reproduce error: >>>>> >>>>> https://github.com/robotarmy/out-of-order-ssl >>>>> >>>>> Thank you, >>>>> Curtis and Team DevEco >>>>> >>>>> >>>>> >>>>> >>>>> Sent through ProtonMail Encrypted Email Channel. >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seriy.pr@REDACTED Mon Nov 11 12:05:50 2019 From: seriy.pr@REDACTED (=?UTF-8?B?0KHQtdGA0LPQtdC5INCf0YDQvtGF0L7RgNC+0LI=?=) Date: Mon, 11 Nov 2019 12:05:50 +0100 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) Message-ID: Hi, I have quite some years of experience writing web scrapers in Erlang. The design that I came to over time is following: I have a top-level supervisor with following 3 processes: - workers_supervisor (queue_name) - queue (queue_name) - scheduler (queue_name) All of them are registered based on queue name to make it possible to have multiple sets of spiders at the same time, but it's not essential if you only need one. - workers supervisor is just `simple_one_for_one` supervisor that supervises worker processes. - queue - gen_server. How it operates does depend on the type of the spider; if it is recursive spider (downloads all the pages of a website) - this process holds: - a queue of urls to download (regular `queue:queue()`), - `ets` table that holds a set of URLs that were ever added to the queue (to avoid downloading the same link more than once: queue process only adds new URLs to the queue if it is NOT in this ETS), - a dictionary / map of tasks currently in progress (taken by worker but not yet finished) as a map `#{ => task()}` - if worker crashes, this task can be re-schedulled. - list of worker pid's subscribed to this queue (maybe monitored). - It may also contain some set of rules to exclude some pages (eg, based on robots.txt). - You should also have URL normalisation function (eg, to threat absolute and relative URLs as the same URL; should decide if `?filter=wasd&page=2` is the same as `?page=2&filter=wasd`, strip URL hashes `#footer` etc). It has 2 APIs: `subscribe` queue has quite simple API: `push` / `push_many`, `subscribe` and `ack`. Workers gen-servers call `subscribe` and wait for a task message (it contains URL and unique reference). When task is done - they call `ack(Reference)` and are ready to get next task. - scheduler: it's basically an entry point and the only "brain" of your spider. It takes in the tasks from whatever you want to take them (PUB/SUB queues / CRON / HTTP api, put the "seed" URLs to the queue and spawns workers (usually at start time by calling "workers supervisor" API and I only used to have fixed number of workers to avoid overloading the website or crawler); it can also monitor queue size progress; workers may report to scheduler when task is taken/done; highly depends on your needs actually. - and of course workers: gen_servers, they are supervised by "workers supervisor", their start is initiated by scheduler (or might be just fixed at app start time actually). At start time they call `queue:subscribe` and just wait for messages from the queue. When message is received, it downloads the page, parses it, pushes all found URLs to queue (queue decides which URLs to accept and which to ignore) and saves the results to database; calls `queue:ack` in the end and waits for next task. There is a choice - let the worker crash on errors or have a top-level "try - catch". I prefer to catch to not spam erlang's crash logs, but it depends on your requirements and expected error rates. This structure proved to be very flexible and allows not only recursive crawlers but some other kinds of crawlers, eg non-recursive that do take their entire URL set from external source and just downloading what they were asked for and saving to DB (in this case scheduler fetches URLs from the task source and puts them to the queue; queue doesn't have duplicate filter). Queue can have namespaces in case you want to parse some website morethan once and sometimes in parallel: for each task you use taks_id as a namespace, so duplicate filter discards URLs based on {taks_id, URL} pair. Hope this will help a bit. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Mon Nov 11 15:54:34 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 11 Nov 2019 15:54:34 +0100 Subject: Know if process is monitored In-Reply-To: References: Message-ID: Thank you Michael, And obviously it was there in some place I didn't find. :D > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michaelviveros@REDACTED Tue Nov 12 00:16:30 2019 From: michaelviveros@REDACTED (Michael Viveros) Date: Mon, 11 Nov 2019 18:16:30 -0500 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Great news, thanks Ingela! I tried confirming the fix on my end but I'm an Erlang noob so it's taking me awhile, will try again tomorrow. On Mon, Nov 11, 2019 at 5:17 AM Ingela Andin wrote: > Hi! > > I have a patch that solves the problem. I have pushed it to my gitrepo > https://github.com/IngelaAndin/otp/tree/ingela/ssl/unorded-chain > > I had to set the options {depth, 2} and {customize_hostname_check, > [{match_fun, CustomFun}] > where CustomFun = public_key:pkix_verify_hostname_match_fun(https) as the > peer certificate is a wildcard cert. > > > Regard Ingela Erlang/OTP team > > > Den fre 8 nov. 2019 kl 21:46 skrev Curtis J Schofield : > >> Oh this is wonderful news!! Glad you were able to identify the code not >> reached !! >> >> Deeply appreciate your support and expertise! >> >> Best, >> Curtis >> >> Sent from ProtonMail Mobile >> >> >> On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin >> wrote: >> >> Hi! >> >> I think I am on to the problem, we have only whitebox tested the unorded >> chain functionality (we intend to create a blackbox but that is a bigger >> job as we need to find some introp software that can create such chains or >> create a simulation modle), >> so I am positive I found that we do not reach the code for sorting the >> chain. I will try to fix it next week. It is easier now that I at least I >> got a server that I can manually blackbox test with :) >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin > >: >> >>> Hi! >>> >>> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < >>> michaelviveros@REDACTED>: >>> >>>> Hi Ingela, >>>> >>>> Curtis' example server from his first message, hooks.glip.com, >>>> presents its certificates out-of-order. The correct order is Peer -> >>>> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented >>>> as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this >>>> returns the "Unknown CA" error. You can confirm this by running `openssl >>>> s_client -connect hooks.glip.com:443`. >>>> >>>> >>> Yes I agree that this is an out of order chain, in contrast to the >>> social.fluffel.io. I will look into it at work tomorrow. >>> >>> Regards Ingela Erlang/OTP Team - Ericsson AB >>> >>> >>> >>>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield >>>> wrote: >>>> >>>>> Hi Ingela >>>>> >>>>> Thank you for your attention- perhaps Micheal can explain this >>>>> better.. >>>>> >>>>> Sent from ProtonMail Mobile >>>>> >>>>> >>>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >>>>> wrote: >>>>> >>>>> Hi! >>>>> >>>>> I tried this out and it is not out of order, it sends the peer cert >>>>> followed by the intermediate cert repeated, that is the chain looks like >>>>> [Peer, CA1, CA1]. >>>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, >>>>> so I suppose we need to add that. >>>>> >>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>> >>>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>>> >>>>>> Hey, >>>>>> >>>>>> I confirm that out of order certs does not seems to be fixed, and it >>>>>> fails with 'Unknown CA' error: >>>>>> >>>>>> >>>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>>> {:error, >>>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>>>> CA'}}} >>>>>> >>>>>> >>>>>> the only issue with this server TLS certificates is the chain order >>>>>> (CA is Letsencrypt): >>>>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>>> >>>>>> >>>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>>> >>>>>> Hi! >>>>>> >>>>>> Just curious if there is an update on out of order certs. >>>>>> >>>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>>> root of id2 who is the root of id3, who is the root of id0. >>>>>> >>>>>> We can correct the out of order problem by including id1,id2,id3 certs >>>>>> in our chain. >>>>>> >>>>>> It would be nice to hear from the erlang maintainers around what kind >>>>>> of >>>>>> "out of order" erlang can handle nicely and if there is planned >>>>>> support for >>>>>> our case! >>>>>> >>>>>> Thank you again, >>>>>> >>>>>> Curtis. >>>>>> >>>>>> >>>>>> Sent through ProtonMail Encrypted Email >>>>>> Channel. >>>>>> >>>>>> >>>>>> ??????? Original Message ??????? >>>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>>>> wrote: >>>>>> >>>>>> Hi! Thank you. >>>>>> >>>>>> >>>>>> I included the root cert in the example. The root cert is id1 in cert >>>>>> chain - this is evident in the other file. >>>>>> >>>>>> It seems because the root cert is out of order - the cert chain is >>>>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>>>> TLS1.2 >>>>>> >>>>>> >>>>>> Thank you for your consideration! >>>>>> >>>>>> >>>>>> Sent from ProtonMail Mobile >>>>>> >>>>>> >>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin < >>>>>> ingela.andin@REDACTED> wrote: >>>>>> >>>>>> >>>>>> Hi! >>>>>> >>>>>> "Unknown CA" means that you did not have the ROOT certificate of the >>>>>> chian in your "trusted store" (cacerts option). >>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>> >>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>> >>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>>>> >: >>>>>> >>>>>> Dear Erlang Questions: >>>>>> >>>>>> >>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>> >>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>> for host hooks.glip.com. >>>>>> >>>>>> When we try to verify peer with the out of order cert >>>>>> chain we get 'Unknown CA'. >>>>>> >>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>> >>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>> mention that other care may need to be taken to ensure compatibility. >>>>>> >>>>>> Reproduce error: >>>>>> >>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>> >>>>>> Thank you, >>>>>> Curtis and Team DevEco >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Tue Nov 12 11:06:56 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 12 Nov 2019 17:06:56 +0700 Subject: Best practice for Erlang's process design to build a website-downloader (super simple crawler) In-Reply-To: References: Message-ID: Hi, Wowww.. thank you very very much for sharing your experience and strategy with me. I do really appreciate it. Ok, l'll start now to write my own website-crawler. Thank you Pada tanggal Sen, 11 Nov 2019 pukul 18.06 ?????? ???????? < seriy.pr@REDACTED> menulis: > Hi, > > I have quite some years of experience writing web scrapers in Erlang. The > design that I came to over time is following: > > I have a top-level supervisor with following 3 processes: > > - workers_supervisor (queue_name) > - queue (queue_name) > - scheduler (queue_name) > > All of them are registered based on queue name to make it possible to have > multiple sets of spiders at the same time, but it's not essential if you > only need one. > > - workers supervisor is just `simple_one_for_one` supervisor that > supervises worker processes. > - queue - gen_server. How it operates does depend on the type of the > spider; if it is recursive spider (downloads all the pages of a website) - > this process holds: > - a queue of urls to download (regular `queue:queue()`), > - `ets` table that holds a set of URLs that were ever added to the > queue (to avoid downloading the same link more than once: queue process > only adds new URLs to the queue if it is NOT in this ETS), > - a dictionary / map of tasks currently in progress (taken by > worker but not yet finished) as a map `#{ > => task()}` - if worker crashes, this task can be re-schedulled. > - list of worker pid's subscribed to this queue (maybe monitored). > - It may also contain some set of rules to exclude some pages (eg, > based on robots.txt). > - You should also have URL normalisation function (eg, to threat > absolute and relative URLs as the same URL; should decide if > `?filter=wasd&page=2` is the same as `?page=2&filter=wasd`, strip URL > hashes `#footer` etc). It has 2 APIs: `subscribe` > > queue has quite simple API: `push` / `push_many`, `subscribe` and > `ack`. Workers gen-servers call `subscribe` and wait for a task message (it > contains URL and unique reference). When task is done - they call > `ack(Reference)` and are ready to get next task. > - scheduler: it's basically an entry point and the only "brain" of > your spider. It takes in the tasks from whatever you want to take them > (PUB/SUB queues / CRON / HTTP api, put the "seed" URLs to the queue and > spawns workers (usually at start time by calling "workers supervisor" API > and I only used to have fixed number of workers to avoid overloading the > website or crawler); it can also monitor queue size progress; workers may > report to scheduler when task is taken/done; highly depends on your needs > actually. > - and of course workers: gen_servers, they are supervised by "workers > supervisor", their start is initiated by scheduler (or might be just fixed > at app start time actually). At start time they call `queue:subscribe` and > just wait for messages from the queue. When message is received, it > downloads the page, parses it, pushes all found URLs to queue (queue > decides which URLs to accept and which to ignore) and saves the results to > database; calls `queue:ack` in the end and waits for next task. > There is a choice - let the worker crash on errors or have a top-level > "try - catch". I prefer to catch to not spam erlang's crash logs, but it > depends on your requirements and expected error rates. > > This structure proved to be very flexible and allows not only recursive > crawlers but some other kinds of crawlers, eg non-recursive that do take > their entire URL set from external source and just downloading what they > were asked for and saving to DB (in this case scheduler fetches URLs from > the task source and puts them to the queue; queue doesn't have duplicate > filter). > Queue can have namespaces in case you want to parse some website morethan > once and sometimes in parallel: for each task you use taks_id as a > namespace, so duplicate filter discards URLs based on {taks_id, URL} pair. > > Hope this will help a bit. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From micael.karlberg@REDACTED Tue Nov 12 15:52:12 2019 From: micael.karlberg@REDACTED (Micael Karlberg) Date: Tue, 12 Nov 2019 15:52:12 +0100 Subject: socket module rcvbuf and sndbuf In-Reply-To: References: Message-ID: On 2019-11-06 14:54, Ben Browitt wrote: > Hi, > > I'm writing a simple wrapper around the socket nif to test the performance compared to gen_udp. > My questions are specific to UDP sockets. > > How do the socket rcvbuf and the otp rcvbuf relate to each other? > The 'rcvbuf' with level = socket is the size of the OS level receive buffer. > The 'rcvbuf' with level = otp is the size of the buffer "we" read into when we read data (with > recv/recvfrom/recvmsg). > If UDP MTU is 1500 bytes, what happens if the socket rcvbuf=MTU or rcvbuf=MTU*N? The buffer should be large enough to hold the largest message you expect to receive. > Should the otp rcvbuf be equal to the socket rcvbuf or larger? What's the difference? The size of the socket:rcvbuf varies a bit with the platforms. On my Linux box its 212992. Enough to hold several UDP messages. There is no need to set the otp rcvbuf equal to or larger then the socket:rcvbuf, since you only read one message at a time. (Unless you connect and use recv, which I have not tested). > What happen if the wrapper module doesn't read messages fast enough from the socket nif? The OS buffers will (eventually) become full (and incoming messages will be lost, I assume). > > Does sndbuf have any effect on UDP sockets? Sure, but on linux its quite large (212992 on my Linux box). If you write faster then the kernel can write, it will fill up. > Should the wrapper module queue packets if the sndbuf is full and resend when possible or should it > just drop packets? Queue'ing will only lead to other problems. Much simpler to just return the error code and leave it to the sender. > > What's the difference between recv, recvfrom, recvmsg and similarly send, sendmsg, sendto? * recv is used to receive bytes. * recvfrom and recvmsg is used to receive messages * recvmsg is used when you need more control or need the control-related messages or miscellaneous ancillary data it can provide. * recv is used by TCP (and connected UDP) * recvfrom is used by UDP * recvmsg is used by both TCP and UDP > > Is it relatively safe to use the socket module in production? I think we have said that we intend for 'socket' to be "officially official" with OTP 23. Until then we consider it "experimental". The point of that is that we should be able to change things if we need to. But: *) we still do not support Windows. *) SCTP is untested. Both of these may force us to alter the API. > What should I be careful about when implementing the udp wrapper module? > > Thanks > /BMK From michaelviveros@REDACTED Tue Nov 12 23:47:56 2019 From: michaelviveros@REDACTED (Michael Viveros) Date: Tue, 12 Nov 2019 17:47:56 -0500 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Just confirmed your fix worked! On Mon, Nov 11, 2019 at 6:16 PM Michael Viveros wrote: > Great news, thanks Ingela! I tried confirming the fix on my end but I'm an > Erlang noob so it's taking me awhile, will try again tomorrow. > > On Mon, Nov 11, 2019 at 5:17 AM Ingela Andin > wrote: > >> Hi! >> >> I have a patch that solves the problem. I have pushed it to my gitrepo >> https://github.com/IngelaAndin/otp/tree/ingela/ssl/unorded-chain >> >> I had to set the options {depth, 2} and {customize_hostname_check, >> [{match_fun, CustomFun}] >> where CustomFun = public_key:pkix_verify_hostname_match_fun(https) as >> the peer certificate is a wildcard cert. >> >> >> Regard Ingela Erlang/OTP team >> >> >> Den fre 8 nov. 2019 kl 21:46 skrev Curtis J Schofield : >> >>> Oh this is wonderful news!! Glad you were able to identify the code not >>> reached !! >>> >>> Deeply appreciate your support and expertise! >>> >>> Best, >>> Curtis >>> >>> Sent from ProtonMail Mobile >>> >>> >>> On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin >>> wrote: >>> >>> Hi! >>> >>> I think I am on to the problem, we have only whitebox tested the >>> unorded chain functionality (we intend to create a blackbox but that is a >>> bigger job as we need to find some introp software that can create such >>> chains or create a simulation modle), >>> so I am positive I found that we do not reach the code for sorting the >>> chain. I will try to fix it next week. It is easier now that I at least I >>> got a server that I can manually blackbox test with :) >>> >>> Regards Ingela Erlang/OTP Team - Ericsson AB >>> >>> Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin >> >: >>> >>>> Hi! >>>> >>>> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < >>>> michaelviveros@REDACTED>: >>>> >>>>> Hi Ingela, >>>>> >>>>> Curtis' example server from his first message, hooks.glip.com, >>>>> presents its certificates out-of-order. The correct order is Peer -> >>>>> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented >>>>> as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this >>>>> returns the "Unknown CA" error. You can confirm this by running `openssl >>>>> s_client -connect hooks.glip.com:443`. >>>>> >>>>> >>>> Yes I agree that this is an out of order chain, in contrast to the >>>> social.fluffel.io. I will look into it at work tomorrow. >>>> >>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>> >>>> >>>> >>>>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield >>>>> wrote: >>>>> >>>>>> Hi Ingela >>>>>> >>>>>> Thank you for your attention- perhaps Micheal can explain this >>>>>> better.. >>>>>> >>>>>> Sent from ProtonMail Mobile >>>>>> >>>>>> >>>>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >>>>>> wrote: >>>>>> >>>>>> Hi! >>>>>> >>>>>> I tried this out and it is not out of order, it sends the peer cert >>>>>> followed by the intermediate cert repeated, that is the chain looks like >>>>>> [Peer, CA1, CA1]. >>>>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, >>>>>> so I suppose we need to add that. >>>>>> >>>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>>> >>>>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>>>> >>>>>>> Hey, >>>>>>> >>>>>>> I confirm that out of order certs does not seems to be fixed, and it >>>>>>> fails with 'Unknown CA' error: >>>>>>> >>>>>>> >>>>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>>>> {:error, >>>>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>>>>> CA'}}} >>>>>>> >>>>>>> >>>>>>> the only issue with this server TLS certificates is the chain order >>>>>>> (CA is Letsencrypt): >>>>>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>>>> >>>>>>> >>>>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> Just curious if there is an update on out of order certs. >>>>>>> >>>>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>>>> root of id2 who is the root of id3, who is the root of id0. >>>>>>> >>>>>>> We can correct the out of order problem by including id1,id2,id3 >>>>>>> certs >>>>>>> in our chain. >>>>>>> >>>>>>> It would be nice to hear from the erlang maintainers around what >>>>>>> kind of >>>>>>> "out of order" erlang can handle nicely and if there is planned >>>>>>> support for >>>>>>> our case! >>>>>>> >>>>>>> Thank you again, >>>>>>> >>>>>>> Curtis. >>>>>>> >>>>>>> >>>>>>> Sent through ProtonMail Encrypted Email >>>>>>> Channel. >>>>>>> >>>>>>> >>>>>>> ??????? Original Message ??????? >>>>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>>>>> wrote: >>>>>>> >>>>>>> Hi! Thank you. >>>>>>> >>>>>>> >>>>>>> I included the root cert in the example. The root cert is id1 in >>>>>>> cert chain - this is evident in the other file. >>>>>>> >>>>>>> It seems because the root cert is out of order - the cert chain is >>>>>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>>>>> TLS1.2 >>>>>>> >>>>>>> >>>>>>> Thank you for your consideration! >>>>>>> >>>>>>> >>>>>>> Sent from ProtonMail Mobile >>>>>>> >>>>>>> >>>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin < >>>>>>> ingela.andin@REDACTED> wrote: >>>>>>> >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> "Unknown CA" means that you did not have the ROOT certificate of >>>>>>> the chian in your "trusted store" (cacerts option). >>>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>>> >>>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>>> >>>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>>>>>> : >>>>>>> >>>>>>> Dear Erlang Questions: >>>>>>> >>>>>>> >>>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>>> >>>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>>> for host hooks.glip.com. >>>>>>> >>>>>>> When we try to verify peer with the out of order cert >>>>>>> chain we get 'Unknown CA'. >>>>>>> >>>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>>> >>>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>>> mention that other care may need to be taken to ensure compatibility. >>>>>>> >>>>>>> Reproduce error: >>>>>>> >>>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>>> >>>>>>> Thank you, >>>>>>> Curtis and Team DevEco >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From peterdmv@REDACTED Wed Nov 13 12:34:39 2019 From: peterdmv@REDACTED (Peter Dimitrov) Date: Wed, 13 Nov 2019 12:34:39 +0100 Subject: Handshake -> psk_key_exchange_modes Message-ID: Hi, This is a bug in the current implementation when configuring a TLS server to support TLS 1.2 or lower and using option {handshake, hello} to pause the handshake. (The {handshake, hello} option is not yet implemented for TLS 1.3.) What happens here is that the client apparently sends a psk_key_exchange_modes extension and possibly even a pre_shared_key extension in its ClientHello. These extensions are used in TLS 1.3 when trying to resume a session with a ticket received in a previous session. Please report this issue on https://bugs.erlang.org/! BR/Peter -----Original Message----- From: erlang-questions On Behalf Of Oliver Bollmann Sent: den 6 november 2019 10:58 To: erlang-questions Subject: Handshake -> psk_key_exchange_modes Hi, 0) Erlang/OTP 22 [erts-10.5.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [hipe] 1) client_server:start(). -> Port 2) Open browser: https://localhost:Port using Safari,Chrome 3) exception error: no function clause matching ssl_handshake:extension_value({psk_key_exchange_modes,[psk_dhe_ke]}) (ssl_handshake.erl, line 1492) in function maps:map_1/2 (maps.erl, line 252) in call from maps:map_1/2 (maps.erl, line 252) in call from maps:map/2 (maps.erl, line 243) in call from ssl_connection:handshake/2 (ssl_connection.erl, line 127) in call from client_server:start/0 (client_server.erl, line 42) Any Hints? -- Gr??e Oliver Bollmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From publicityifl@REDACTED Thu Nov 14 09:15:29 2019 From: publicityifl@REDACTED (Jurriaan Hage) Date: Thu, 14 Nov 2019 00:15:29 -0800 Subject: Second call for draft papers for TFPIE 2020 (Trends in Functional Programming in Education) Message-ID: Hello, Please, find below the second call for draft papers for TFPIE 2020. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Chair of TFPIE 2020 ======================================================================== TFPIE 2020 Call for papers http://www.staff.science.uu.nl/~hage0101/tfpie2020/index.html February 12th 2020, Krakow, Poland (co-located with TFP 2020 and Lambda Days) TFPIE 2020 welcomes submissions describing techniques used in the classroom, tools used in and/or developed for the classroom and any creative use of functional programming (FP) to aid education in or outside Computer Science. Topics of interest include, but are not limited to: FP and beginning CS students FP and Computational Thinking FP and Artificial Intelligence FP in Robotics FP and Music Advanced FP for undergraduates FP in graduate education Engaging students in research using FP FP in Programming Languages FP in the high school curriculum FP as a stepping stone to other CS topics FP and Philosophy The pedagogy of teaching FP FP and e-learning: MOOCs, automated assessment etc. Best Lectures - more details below In addition to papers, we are requesting best lecture presentations. What's your best lecture topic in an FP related course? Do you have a fun way to present FP concepts to novices or perhaps an especially interesting presentation of a difficult topic? In either case, please consider sharing it. Best lecture topics will be selected for presentation based on a short abstract describing the lecture and its interest to TFPIE attendees. The length of the presentation should be comparable to that of a paper. On top of the lecture itself, the presentation can also provide commentary on the lecture. Submissions Potential presenters are invited to submit an extended abstract (4-6 pages) or a draft paper (up to 20 pages) in EPTCS style. The authors of accepted presentations will have their preprints and their slides made available on the workshop's website. Papers and abstracts can be submitted via easychair at the following link: https://easychair.org/conferences/?conf=tfpie2020 . After the workshop, presenters will be invited to submit (a revised version of) their article for review. The PC will select the best articles that will be published in the Electronic Proceedings in Theoretical Computer Science (EPTCS). Articles rejected for presentation and extended abstracts will not be formally reviewed by the PC. Dates Submission deadline: January 14th 2020, Anywhere on Earth. Notification: January 17th 2020 TFPIE Registration Deadline: January 20th 2020 Workshop: February 12th 2020 Submission for formal review: April 19th 2020, Anywhere on Earth. Notification of full article: June 6th 2020 Camera ready: July 1st 2020 Program Committee Olaf Chitil - University of Kent Youyou Cong - Tokyo Institute of Technology Marko van Eekelen - Open University of the Netherlands and Radboud University Nijmegen Jurriaan Hage (Chair) - Utrecht University Marco T. Morazan - Seton Hall University, USA Sharon Tuttle - Humboldt State University, USA Janis Voigtlaender - University of Duisburg-Essen Viktoria Zsok - Eotvos Lorand University Note: information on TFP is available at http://www.cse.chalmers.se/~rjmh/tfp/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Thu Nov 14 13:40:42 2019 From: andre@REDACTED (Andre Nathan) Date: Thu, 14 Nov 2019 09:40:42 -0300 Subject: Line editing not working in to_erl Message-ID: <664f8d0b-4140-e55a-ab27-1a46b508049d@digirati.com.br> Hi I've recently repackaged an old application to run on an upgraded server, and after installing it I noticed that when using the release init script's "attach" command (in other words, "to_erl"), line editing doesn't work, while it works fine when running the release's "erl" command. The strange thing is that on a test VirtualBox VM, with the same operating system version, and installing the app using the same pre-compiled package, I can't reproduce the problem. I've found some reports of similar issues that were related to SSH and terminal handling, but the problem happens in the server also when connecting locally via a console, while on the VM it works both ways. Has anyone ever seen this behavior? Thanks, Andre From ostinelli@REDACTED Thu Nov 14 20:29:05 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Thu, 14 Nov 2019 20:29:05 +0100 Subject: function_exported/3 and CT Message-ID: All, I'm experiencing something a little strange. I have a module test_event_handler.erl which exports the method resolve_registry_conflict/3. This file is in the /test directory from where I run all of my Common Tests. For some reason, the call to erlang:function_exported(test_event_handler, resolve_registry_conflict, 3) returns true if I run rebar3 clean before running tests, otherwise false. This is true both when this call is run into the test itself, or in the code being tested. Has someone seen this behavior? What am I missing? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Nov 14 20:34:01 2019 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Thu, 14 Nov 2019 20:34:01 +0100 Subject: function_exported/3 and CT In-Reply-To: References: Message-ID: It is probably related to the module being loaded? function_exported/3 does not load modules and will return false if the module has not been loaded yet. Something in rebar3 lifecycle may be loading it under certain circumstances. Try calling code:ensure_loaded/1 before function_exported and see if the behavior becomes consistent.-- *Jos? Valim* www.plataformatec.com.br Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Fri Nov 15 05:46:37 2019 From: petergi@REDACTED (Peter J Etheridge) Date: Fri, 15 Nov 2019 15:46:37 +1100 Subject: canberra-gtk-module Message-ID: <49adc6d385ff39c13ef645e17d2639a2a27fca8a@webmail.iinet.net.au> dear friends,running OTP21 on ubuntuas my wx module starts the shell displays; failed to load module "canberra-gtk-module" where should i look to rectify this failure? thank you in advance,peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Fri Nov 15 10:22:41 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 15 Nov 2019 10:22:41 +0100 Subject: function_exported/3 and CT In-Reply-To: References: Message-ID: You were definitely right. Only change I've done is that instead of using code:ensure_loaded/1 (which doesn't work in embedded mode, for the 3 people that use it in the world ^^_) I am using a simple catch test_event_handler:module_info(exports) which seems to do the trick. Thank you Jose, r. On Fri, Nov 15, 2019 at 10:20 AM Roberto Ostinelli wrote: > You were definitely right. Only change I've done is that instead of using > code:ensure_loaded/1 (which doesn't work in embedded mode, for the 3 > people that use it in the world ^^_) I am using a simple catch > test_event_handler:module_info(exports) which seems to do the trick. > > Thank you Jose, > r. > > On Thu, Nov 14, 2019 at 8:34 PM Jos? Valim < > jose.valim@REDACTED> wrote: > >> It is probably related to the module being loaded? function_exported/3 >> does not load modules and will return false if the module has not been >> loaded yet. Something in rebar3 lifecycle may be loading it under certain >> circumstances. Try calling code:ensure_loaded/1 before function_exported >> and see if the behavior becomes consistent.-- >> >> >> *Jos? Valim* >> www.plataformatec.com.br >> Founder and Director of R&D >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Nov 15 21:24:42 2019 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Nov 2019 20:24:42 +0000 Subject: Whence eunit:test/0? Message-ID: 1> eunit:test(). All 6 tests passed. ok But there's no `test/0` function *in* eunit.erl. What's going on? From elbrujohalcon@REDACTED Fri Nov 15 21:29:39 2019 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Fri, 15 Nov 2019 17:29:39 -0300 Subject: Whence eunit:test/0? In-Reply-To: References: Message-ID: It?s the magic of parse transforms! eunit.erl includes eunit.hrl: https://github.com/erlang/otp/blob/8e4abc44d58e61753716e63b23c6b44d221feb15/lib/eunit/src/eunit.erl#L30 When you include that header file, it will create a test/0 function for you. Check it out: http://erlang.org/doc/apps/eunit/chapter.html#running-eunit Brujo Benavides > On 15 Nov 2019, at 17:24, Roger Lipscombe wrote: > > 1> eunit:test(). > All 6 tests passed. > ok > > But there's no `test/0` function *in* eunit.erl. What's going on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Nov 15 21:47:05 2019 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 15 Nov 2019 20:47:05 +0000 Subject: Whence eunit:test/0? In-Reply-To: References: Message-ID: Ah. Specifically this: https://github.com/erlang/otp/blob/master/lib/eunit/src/eunit_autoexport.erl#L89 Module:test() -> eunit:test(Module). ...which invokes the test cases in Module *and* Module_tests. In the case of eunit_tests, there are 6 tests. It's been a long day. On Fri, 15 Nov 2019 at 20:29, Brujo Benavides wrote: > > It?s the magic of parse transforms! > > eunit.erl includes eunit.hrl: https://github.com/erlang/otp/blob/8e4abc44d58e61753716e63b23c6b44d221feb15/lib/eunit/src/eunit.erl#L30 > > When you include that header file, it will create a test/0 function for you. > Check it out: http://erlang.org/doc/apps/eunit/chapter.html#running-eunit > > ________________________________ > Brujo Benavides > > > > On 15 Nov 2019, at 17:24, Roger Lipscombe wrote: > > 1> eunit:test(). > All 6 tests passed. > ok > > But there's no `test/0` function *in* eunit.erl. What's going on? > > From gergo.nyiro@REDACTED Sat Nov 16 18:38:15 2019 From: gergo.nyiro@REDACTED (=?UTF-8?B?TnlpcsWRIEdlcmfFkQ==?=) Date: Sat, 16 Nov 2019 18:38:15 +0100 Subject: Manage application env from rebar3 profile Message-ID: Dear list, I would like to change some constant during testing e.g.: cycle time of a periodic job. It would be ideal to manage it from rebar3 config, but I don't see how should I edit the application env variables from rebar3 profiles. Is that even possible? Should I use persistent_term instead of application:get_env? Thanks for your help in advance, Gerg? From okaprinarjaya@REDACTED Sat Nov 16 19:41:08 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 17 Nov 2019 01:41:08 +0700 Subject: Manage application env from rebar3 profile In-Reply-To: References: Message-ID: Hi, Yes it's possible. You can utilise sys.config with configurations content like below: [ {YOUR_APP_NAME, [ {your_env_var1_defined_in_os_in_lowercase, "${YOUR_ENV_VAR1_DEFINED_IN_OS}"}, {your_env_var2_defined_in_os_in_lowercase, "${YOUR_ENV_VAR2_DEFINED_IN_OS}"}, {your_env_var3_defined_in_os_in_lowercase, "${YOUR_ENV_VAR3_DEFINED_IN_OS}"}, .... you can define more here as much as you want ]} ]. Then, in order to be able shipping / bring sys.config into your release, you need to treat sys.config as a dynamic configuration source file. Follow steps below: 1. create a directory named config at root APP / project directory 2. create file sys.config inside config/ directory 3. the edit your rebar.config as below: {shell, [ {sys_config_src, "config/sys.config.src"}, {apps, [APP_NAME]} ]}. {relx, [ {sys_config_src, "config/sys.config.src"}, { release, {APP_NAME, "0.1.0"}, }, {dev_mode, true}, {extended_start_script, true} ]}. For further information you can read here: http://www.rebar3.org/docs/releases#section-dynamic-configuration and here: http://www.rebar3.org/docs/dynamic-configuration (more advance) Hope it helps. Pada tanggal Min, 17 Nov 2019 pukul 00.38 Nyir? Gerg? menulis: > Dear list, > > I would like to change some constant during testing e.g.: cycle time > of a periodic job. > It would be ideal to manage it from rebar3 config, but I don't see how > should I edit > the application env variables from rebar3 profiles. Is that even > possible? Should I use > persistent_term instead of application:get_env? > > Thanks for your help in advance, > Gerg? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Sat Nov 16 19:50:11 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Sun, 17 Nov 2019 01:50:11 +0700 Subject: Manage application env from rebar3 profile In-Reply-To: References: Message-ID: Hi, >> Should I use persistent_term instead of application:get_env? Use application:get_env(YOUR_APP_NAME, your_env_var1_defined_in_os_in_lowercase) No need to use persistent_term , that's a different thing and not related to consuming-os-env-vars task. Because I assume your needs is consuming OS ENV vars. Pada tanggal Min, 17 Nov 2019 pukul 01.41 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > Hi, > > Yes it's possible. > > You can utilise sys.config with configurations content like below: > > [ > {YOUR_APP_NAME, [ > {your_env_var1_defined_in_os_in_lowercase, > "${YOUR_ENV_VAR1_DEFINED_IN_OS}"}, > {your_env_var2_defined_in_os_in_lowercase, > "${YOUR_ENV_VAR2_DEFINED_IN_OS}"}, > {your_env_var3_defined_in_os_in_lowercase, > "${YOUR_ENV_VAR3_DEFINED_IN_OS}"}, > .... you can define more here as much as you want > ]} > ]. > > Then, in order to be able shipping / bring sys.config into your release, > you need to treat sys.config as a dynamic configuration source file. > > Follow steps below: > 1. create a directory named config at root APP / project directory > 2. create file sys.config inside config/ directory > 3. the edit your rebar.config as below: > > {shell, [ > {sys_config_src, "config/sys.config.src"}, > {apps, [APP_NAME]} > ]}. > > {relx, [ > {sys_config_src, "config/sys.config.src"}, > { > release, > {APP_NAME, "0.1.0"}, > }, > {dev_mode, true}, > {extended_start_script, true} > ]}. > > For further information you can read here: > http://www.rebar3.org/docs/releases#section-dynamic-configuration > and here: http://www.rebar3.org/docs/dynamic-configuration (more advance) > > Hope it helps. > > > > > Pada tanggal Min, 17 Nov 2019 pukul 00.38 Nyir? Gerg? < > gergo.nyiro@REDACTED> menulis: > >> Dear list, >> >> I would like to change some constant during testing e.g.: cycle time >> of a periodic job. >> It would be ideal to manage it from rebar3 config, but I don't see how >> should I edit >> the application env variables from rebar3 profiles. Is that even >> possible? Should I use >> persistent_term instead of application:get_env? >> >> Thanks for your help in advance, >> Gerg? >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sat Nov 16 21:07:39 2019 From: t@REDACTED (Tristan Sloughter) Date: Sat, 16 Nov 2019 13:07:39 -0700 Subject: Manage application env from rebar3 profile In-Reply-To: References: Message-ID: <06723a2c-a573-4975-b78c-7c243f45aaac@www.fastmail.com> If you are using Common Test you can use `ct_opts` to point to different sys.config files for test suites. The sys.config files are loaded before test suites are run: {ct_opts, [{sys_config, "test/sys.config"}]}. Then you can always have different profiles with different `ct_opts`pointing to different `sys.config` files. Tristan On Sat, Nov 16, 2019, at 10:38, Nyir? Gerg? wrote: > Dear list, > > I would like to change some constant during testing e.g.: cycle time > of a periodic job. > It would be ideal to manage it from rebar3 config, but I don't see how > should I edit > the application env variables from rebar3 profiles. Is that even > possible? Should I use > persistent_term instead of application:get_env? > > Thanks for your help in advance, > Gerg? > From ostinelli@REDACTED Sat Nov 16 22:36:39 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Sat, 16 Nov 2019 22:36:39 +0100 Subject: Common Tests and logs Message-ID: All, I'm having a hard time to get the logs of slave nodes in Common Tests, started with ct_slave:start/1,2,3. More specifically, when I use ct:pal I only see the messages on the main ct@ node. How can I capture logs running on slave nodes? Given that io:format also does not work in the CT environment, I'm not really sure how to proceed. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andras.boroska@REDACTED Sat Nov 16 23:07:54 2019 From: andras.boroska@REDACTED (=?UTF-8?Q?Boroska_Andr=C3=A1s?=) Date: Sat, 16 Nov 2019 22:07:54 +0000 Subject: Common Tests and logs In-Reply-To: References: Message-ID: Hi Roberto, Did you try io:format/3 where the first argument is the 'user' atom? Br, Andras On Sat, Nov 16, 2019 at 9:37 PM Roberto Ostinelli wrote: > All, > I'm having a hard time to get the logs of slave nodes in Common Tests, > started with ct_slave:start/1,2,3. More specifically, when I use ct:pal I > only see the messages on the main ct@ node. > > How can I capture logs running on slave nodes? Given that io:format also > does not work in the CT environment, I'm not really sure how to proceed. > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Sun Nov 17 14:02:51 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 17 Nov 2019 14:02:51 +0100 Subject: Common Tests and logs In-Reply-To: References: Message-ID: Hey Andras, This still doesn't print logs for other nodes AFAIK. I've ended up redirecting all logs to disk during tests, with something like error_logger:logfile({open, atom_to_list(node())}). This makes individual log files be created in the test result's directory... If there is a better solution, I'd love to hear it. On Sat, Nov 16, 2019 at 11:08 PM Boroska Andr?s wrote: > Hi Roberto, > > Did you try io:format/3 where the first argument is the 'user' atom? > > Br, > Andras > > On Sat, Nov 16, 2019 at 9:37 PM Roberto Ostinelli > wrote: > >> All, >> I'm having a hard time to get the logs of slave nodes in Common Tests, >> started with ct_slave:start/1,2,3. More specifically, when I use ct:pal >> I only see the messages on the main ct@ node. >> >> How can I capture logs running on slave nodes? Given that io:format also >> does not work in the CT environment, I'm not really sure how to proceed. >> >> Thank you, >> r. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Sun Nov 17 23:44:05 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 17 Nov 2019 23:44:05 +0100 Subject: mnesia:ets/2 and secondary indices Message-ID: All, I'm using mnesia tables in ram only, not replicated, not fragmented. The only reason why I'm using mnesia over ETS is because of the secondary index feature in mnesia. I'm at about right in terms of requirements for writing performance as per now, but would like to know if I can get an extra boost. For this, I was wandering if mnesia:ets/2 does create these secondary indices or not. I do not seem to find documentation that answers this question. Thank you for any insights you may provide, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From neustradamus@REDACTED Sun Nov 17 22:48:19 2019 From: neustradamus@REDACTED (- Neustradamus -) Date: Sun, 17 Nov 2019 21:48:19 +0000 Subject: SCRAM-SHA-256(-PLUS) for ejabberd Message-ID: Hello all, Currently there is only SCRAM-SHA-1 in ejabberd. I search people to look the code for have more easy possibilities: - SCRAM-SHA-1-PLUS - SCRAM-SHA-224 - SCRAM-SHA-224-PLUS - SCRAM-SHA-256 - SCRAM-SHA-256-PLUS - SCRAM-SHA-384 - SCRAM-SHA-384-PLUS - SCRAM-SHA-512 - SCRAM-SHA-512-PLUS -PLUS variants -> "tls-unique" Links: - https://github.com/processone/fast_tls/blob/master/src/p1_sha.erl - https://github.com/processone/xmpp/ - https://github.com/processone/ejabberd/ - https://github.com/processone/ejabberd-contrib/ RFCs: - RFC5802: Salted Challenge Response Authentication Mechanism (SCRAM) SASL and GSS-API Mechanisms: https://tools.ietf.org/html/rfc5802 - RFC7677: SCRAM-SHA-256 and SCRAM-SHA-256-PLUS Simple Authentication and Security Layer (SASL) Mechanisms: https://tools.ietf.org/html/rfc7677 - since 2015-11-02 - RFC5056: On the Use of Channel Bindings to Secure Channels: https://tools.ietf.org/html/rfc5056 - RFC5929: Channel Bindings for TLS: https://tools.ietf.org/html/rfc5929 - RFC5803: Lightweight Directory Access Protocol (LDAP) Schema for Storing Salted: Challenge Response Authentication Mechanism (SCRAM) Secrets: https://tools.ietf.org/html/rfc5803 - RFC7804: Salted Challenge Response HTTP Authentication Mechanism: https://tools.ietf.org/html/rfc7804 IANA: - Simple Authentication and Security Layer (SASL) Mechanisms: https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml - Channel-Binding Types: https://www.iana.org/assignments/channel-binding-types/channel-binding-types.xhtml Cyrus SASL supports: - SCRAM-SHA-1 - SCRAM-SHA-1-PLUS - SCRAM-SHA-224 - SCRAM-SHA-224-PLUS - SCRAM-SHA-256 - SCRAM-SHA-256-PLUS - SCRAM-SHA-384 - SCRAM-SHA-384-PLUS - SCRAM-SHA-512 - SCRAM-SHA-512-PLUS -> https://cyrusimap.org/sasl/sasl/authentication_mechanisms.html -> https://github.com/cyrusimap/cyrus-sasl/commits/master Dovecot SASL supports: - SCRAM-SHA-1 -> https://doc.dovecot.org/configuration_manual/authentication/password_schemes/ GNU SASL supports: - SCRAM-SHA-1 - SCRAM-SHA-1-PLUS -> http://www.gnu.org/software/gsasl/ CRAM-MD5 to Historic: - https://tools.ietf.org/html/draft-ietf-sasl-crammd5-to-historic-00 // 20 November 2008 RFC6331: Moving DIGEST-MD5 to Historic - https://tools.ietf.org/html/rfc6331 since July 2011 More informations: - https://github.com/scram-xmpp/info/issues/1 Thanks in advance for your help. Regards, Neustradamus From otp@REDACTED Mon Nov 18 12:44:06 2019 From: otp@REDACTED (Erlang/OTP) Date: Mon, 18 Nov 2019 12:44:06 +0100 (CET) Subject: Patch Package OTP 20.3.8.24 Released Message-ID: <20191118114406.AEC98254730@hel.cslab.ericsson.net> Patch Package: OTP 20.3.8.24 Git Tag: OTP-20.3.8.24 Date: 2019-11-18 Trouble Report Id: OTP-14849, OTP-16193, OTP-16287 Seq num: ERL-545 System: OTP Release: 20 Application: common_test-1.15.4.4, erts-9.3.3.13, ssh-4.6.9.5 Predecessor: OTP 20.3.8.23 Check out the git tag OTP-20.3.8.24, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.15.4.4 -------------------------------------------- --------------------------------------------------------------------- The common_test-1.15.4.4 application can be applied independently of other applications on a full OTP 20 installation. --- Improvements and New Features --- OTP-16287 Application(s): common_test The ct_property_test logging is improved. Full runtime dependencies of common_test-1.15.4.4: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-3.4, syntax_tools-1.7, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --- erts-9.3.3.13 --------------------------------------------------- --------------------------------------------------------------------- The erts-9.3.3.13 application can be applied independently of other applications on a full OTP 20 installation. --- Fixed Bugs and Malfunctions --- OTP-16193 Application(s): erts A literal area could prematurely be released before all uses of it had been removed. This occurred either when a terminating process had a complex exit reason referring to a literal that concurrently was removed, or when a terminating process continued executing a dirty NIF accessing a literal (via the heap) that concurrently was removed. Full runtime dependencies of erts-9.3.3.13: kernel-5.0, sasl-3.0.1, stdlib-3.0 --------------------------------------------------------------------- --- ssh-4.6.9.5 ----------------------------------------------------- --------------------------------------------------------------------- Note! The ssh-4.6.9.5 application *cannot* be applied independently of other applications on an arbitrary OTP 20 installation. On a full OTP 20 installation, also the following runtime dependencies have to be satisfied: -- crypto-4.2 (first satisfied in OTP 20.2) -- public_key-1.5.2 (first satisfied in OTP 20.2) --- Fixed Bugs and Malfunctions --- OTP-14849 Application(s): ssh Related Id(s): ERL-545 The ssh cli (e.g shell) server behaved strangely when characters were inserted in a string so that the last characters tried to wrap the line. Full runtime dependencies of ssh-4.6.9.5: crypto-4.2, erts-6.0, kernel-3.0, public_key-1.5.2, stdlib-3.3 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From joshzinny@REDACTED Mon Nov 18 11:40:06 2019 From: joshzinny@REDACTED (josh zinny) Date: Mon, 18 Nov 2019 10:40:06 +0000 (UTC) Subject: Inets http2 support References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> Message-ID: <1451515160.1108713.1574073606814@mail.yahoo.com> Hi guys, I was wondering if http2/3 is one the roadmap for Inets? I know there's is Cowboy and Gun. However, I'm interested in the Inets httpd(http2). I have read the Erlang 22.1 documentation, but it only references http/1.1. Also, any suggestions on how to achieve this or a workaround will be greatly appreciated. Many thanks,Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Mon Nov 18 14:54:06 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 18 Nov 2019 14:54:06 +0100 Subject: [ANN] Syn 2.0.0-rc.1 Message-ID: All, Syn 2.0.0-rc.1 has just been released. You can grab it from the usual places: https://github.com/ostinelli/syn https://hex.pm/packages/syn For those of you who don't know it, Syn is a global Process Registry and Process Group manager for Erlang & Elixir, which supports PubSub. Syn v2 is a complete rewrite and has been driven by a series of considerations, mainly to substitute the usage of automated mnesia replication features in favour of a custom solution. If you are curious, I've written about it in a blog post [1]. As always, feedback welcome. Thank you, r. [1] http://www.ostinelli.net/a-journey-to-syn-v2/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Nov 18 14:56:15 2019 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 18 Nov 2019 08:56:15 -0500 Subject: Inets http2 support In-Reply-To: <1451515160.1108713.1574073606814@mail.yahoo.com> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> Message-ID: On Mon, Nov 18, 2019 at 7:55 AM josh zinny wrote: > Hi guys, > > I was wondering if http2/3 is one the roadmap for Inets? I know there's is > Cowboy and Gun. > However, I'm interested in the Inets httpd(http2). I have read the Erlang > 22.1 documentation, but it only references http/1.1. > > Also, any suggestions on how to achieve this or a workaround will be > greatly appreciated. > > I do not know about the OTP team's plans with regards to supporting HTTP/2, but I can say this: HTTP/2 is sufficiently complex and different from HTTP/1.1 that it's not just a workaround to add support to it; it essentially asks people to reimplement something reminiscent of SCTP over TCP in order to provide multiple channels, and includes things like reimplementing the sliding window mechanisms that exist within individual TCP connections. The workaround to "wanting to use HTTP/2" is "use an HTTP/2 library"; it's too complex to just add rapidly. If not Cowboy and Gun, then something like Chatterbox can be an acceptable library, and if these do not work, then using some sort of HTTP2-to-HTTP1 proxy, which some service providers in the cloud can do for you, generally. Regards, Fred. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Mon Nov 18 14:59:01 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 18 Nov 2019 14:59:01 +0100 Subject: mnesia:ets/2 and secondary indices In-Reply-To: References: Message-ID: For anyone reading this in the future ^^_ and having the same question: atm no, indices get not created when writing with mnesia:ets/2. Cheers, r. On Sun, Nov 17, 2019 at 11:44 PM Roberto Ostinelli wrote: > All, > I'm using mnesia tables in ram only, not replicated, not fragmented. The > only reason why I'm using mnesia over ETS is because of the secondary index > feature in mnesia. > > I'm at about right in terms of requirements for writing performance as per > now, but would like to know if I can get an extra boost. For this, I was > wandering if mnesia:ets/2 does create these secondary indices or not. I > do not seem to find documentation that answers this question. > > Thank you for any insights you may provide, > r. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Nov 18 19:46:45 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 18 Nov 2019 19:46:45 +0100 Subject: bugs.erlang.org login with github is gone? Message-ID: Hello, It seems that "Login with GitHub" is no longer available on https://bugs.erlang.org/login.jsp?os_destination=%2Fsecure%2FDashboard.jspa Is this an oversight following a JIRA upgrade maybe? Thanks, -- Lo?c Hoguin https://ninenines.eu From mjtruog@REDACTED Mon Nov 18 20:48:35 2019 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 18 Nov 2019 11:48:35 -0800 Subject: bugs.erlang.org login with github is gone? In-Reply-To: References: Message-ID: <4d198b23-630b-ee36-5b82-12975ac3ab52@gmail.com> https://bugs.erlang.org/browse/ERL-523 was an open bug regarding it (an issue during the past 2+ years), so it may be best that it was removed. Best Regards, Michael On 11/18/19 10:46 AM, Lo?c Hoguin wrote: > Hello, > > It seems that "Login with GitHub" is no longer available on > https://bugs.erlang.org/login.jsp?os_destination=%2Fsecure%2FDashboard.jspa > > Is this an oversight following a JIRA upgrade maybe? > > Thanks, > From otp@REDACTED Tue Nov 19 10:00:20 2019 From: otp@REDACTED (Erlang/OTP) Date: Tue, 19 Nov 2019 10:00:20 +0100 (CET) Subject: Patch Package OTP 21.3.8.11 Released Message-ID: <20191119090020.700352547AB@hel.cslab.ericsson.net> Patch Package: OTP 21.3.8.11 Git Tag: OTP-21.3.8.11 Date: 2019-11-19 Trouble Report Id: OTP-16193, OTP-16224, OTP-16241, OTP-16243, OTP-16265 Seq num: ERL-1044, ERL-1064, ERL-1076 System: OTP Release: 21 Application: erts-10.3.5.7, ftp-1.0.2.2 Predecessor: OTP 21.3.8.10 Check out the git tag OTP-21.3.8.11, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.3.5.7 --------------------------------------------------- --------------------------------------------------------------------- Note! The erts-10.3.5.7 application *cannot* be applied independently of other applications on an arbitrary OTP 21 installation. On a full OTP 21 installation, also the following runtime dependencies have to be satisfied: -- kernel-6.1 (first satisfied in OTP 21.1) -- sasl-3.3 (first satisfied in OTP 21.2) --- Fixed Bugs and Malfunctions --- OTP-16193 Application(s): erts A literal area could prematurely be released before all uses of it had been removed. This occurred either when a terminating process had a complex exit reason referring to a literal that concurrently was removed, or when a terminating process continued executing a dirty NIF accessing a literal (via the heap) that concurrently was removed. OTP-16224 Application(s): erts Related Id(s): ERL-1044 Fix bug causing VM crash due to memory corruption of distribution entry. Probability of crash increases if Erlang distribution is frequently disconnected and reestablished towards same node names. Bug exists since OTP-21.0. OTP-16241 Application(s): erts Related Id(s): ERL-1076, OTP-16219 Fix bug where the receive marker used by the runtime to do the receive queue optimization could be incorrectly set. The symptom of this would be that a message that should match in a receive never matches. The bug requires the OTP-22 compiler and multiple selective receives to trigger. See OTP-16219 for details about the bug fix in the compiler. OTP-16265 Application(s): erts Related Id(s): ERL-1064 Fixed bug causing crash of VM built with configuration --enable--sharing-preserving. Provoked when a sent message contains both a bit string and the heap binary (< 65 bytes) which the bit string was matched from. Bug exists since OTP-19.0 but has seen to be easier to provoke since OTP-22.1. Full runtime dependencies of erts-10.3.5.7: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --- ftp-1.0.2.2 ----------------------------------------------------- --------------------------------------------------------------------- The ftp-1.0.2.2 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-16243 Application(s): ftp Related Id(s): OTP-16056, PR-2436 A possibly infinite loop is removed. Full runtime dependencies of ftp-1.0.2.2: erts-7.0, kernel-6.0, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From jesper.louis.andersen@REDACTED Tue Nov 19 17:16:57 2019 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 19 Nov 2019 17:16:57 +0100 Subject: Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> Message-ID: On Mon, Nov 18, 2019 at 2:56 PM Fred Hebert wrote: > I do not know about the OTP team's plans with regards to supporting > HTTP/2, but I can say this: HTTP/2 is sufficiently complex and different > from HTTP/1.1 that it's not just a workaround to add support to it; it > essentially asks people to reimplement something reminiscent of SCTP over > TCP in order to provide multiple channels, and includes things like > reimplementing the sliding window mechanisms that exist within individual > TCP connections. The workaround to "wanting to use HTTP/2" is "use an > HTTP/2 library"; it's too complex to just add rapidly. If not Cowboy and > Gun, then something like Chatterbox can be an acceptable library, and if > these do not work, then using some sort of HTTP2-to-HTTP1 proxy, which some > service providers in the cloud can do for you, generally. > > And HTTP/3 requires you to implement a dialect of QUIC on top of UDP. That is, you "just" have to write a complete TCP-like protocol and get it to run fast. The complexity of this is through the roof. In principle, this effort should have been modular, so you have a chance to catch up, but alas, no. I'm not sure I like the direction this is going in, because the resource investment is so heavy. Mind you, last time such a thing happened we had XML and then Douglas Crockford comes by with a hack. The hack then won, despite all its flaws. Another important point is that when you start deploying new protocols next to TCP, or alter TCPs congestion control system, you risk seriously damaging the internet. There is a recent paper (from CMU I think) which argues that TCP-BBR is bad for the TCP flows in many situations. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michaelviveros@REDACTED Thu Nov 21 03:22:20 2019 From: michaelviveros@REDACTED (Michael Viveros) Date: Wed, 20 Nov 2019 21:22:20 -0500 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Any update about next steps, Ingela? On Tue, Nov 12, 2019 at 5:47 PM Michael Viveros wrote: > Just confirmed your fix worked! > > On Mon, Nov 11, 2019 at 6:16 PM Michael Viveros > wrote: > >> Great news, thanks Ingela! I tried confirming the fix on my end but I'm >> an Erlang noob so it's taking me awhile, will try again tomorrow. >> >> On Mon, Nov 11, 2019 at 5:17 AM Ingela Andin >> wrote: >> >>> Hi! >>> >>> I have a patch that solves the problem. I have pushed it to my gitrepo >>> https://github.com/IngelaAndin/otp/tree/ingela/ssl/unorded-chain >>> >>> I had to set the options {depth, 2} and {customize_hostname_check, >>> [{match_fun, CustomFun}] >>> where CustomFun = public_key:pkix_verify_hostname_match_fun(https) as >>> the peer certificate is a wildcard cert. >>> >>> >>> Regard Ingela Erlang/OTP team >>> >>> >>> Den fre 8 nov. 2019 kl 21:46 skrev Curtis J Schofield : >>> >>>> Oh this is wonderful news!! Glad you were able to identify the code not >>>> reached !! >>>> >>>> Deeply appreciate your support and expertise! >>>> >>>> Best, >>>> Curtis >>>> >>>> Sent from ProtonMail Mobile >>>> >>>> >>>> On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin >>>> wrote: >>>> >>>> Hi! >>>> >>>> I think I am on to the problem, we have only whitebox tested the >>>> unorded chain functionality (we intend to create a blackbox but that is a >>>> bigger job as we need to find some introp software that can create such >>>> chains or create a simulation modle), >>>> so I am positive I found that we do not reach the code for sorting the >>>> chain. I will try to fix it next week. It is easier now that I at least I >>>> got a server that I can manually blackbox test with :) >>>> >>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>> >>>> Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin < >>>> ingela.andin@REDACTED>: >>>> >>>>> Hi! >>>>> >>>>> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < >>>>> michaelviveros@REDACTED>: >>>>> >>>>>> Hi Ingela, >>>>>> >>>>>> Curtis' example server from his first message, hooks.glip.com, >>>>>> presents its certificates out-of-order. The correct order is Peer -> >>>>>> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented >>>>>> as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this >>>>>> returns the "Unknown CA" error. You can confirm this by running `openssl >>>>>> s_client -connect hooks.glip.com:443`. >>>>>> >>>>>> >>>>> Yes I agree that this is an out of order chain, in contrast to the >>>>> social.fluffel.io. I will look into it at work tomorrow. >>>>> >>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>> >>>>> >>>>> >>>>>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield >>>>>> wrote: >>>>>> >>>>>>> Hi Ingela >>>>>>> >>>>>>> Thank you for your attention- perhaps Micheal can explain this >>>>>>> better.. >>>>>>> >>>>>>> Sent from ProtonMail Mobile >>>>>>> >>>>>>> >>>>>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin >>>>>>> wrote: >>>>>>> >>>>>>> Hi! >>>>>>> >>>>>>> I tried this out and it is not out of order, it sends the peer cert >>>>>>> followed by the intermediate cert repeated, that is the chain looks like >>>>>>> [Peer, CA1, CA1]. >>>>>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored too, >>>>>>> so I suppose we need to add that. >>>>>>> >>>>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>>>> >>>>>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>>>>> >>>>>>>> Hey, >>>>>>>> >>>>>>>> I confirm that out of order certs does not seems to be fixed, and >>>>>>>> it fails with 'Unknown CA' error: >>>>>>>> >>>>>>>> >>>>>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>>>>> {:error, >>>>>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>>>>>> CA'}}} >>>>>>>> >>>>>>>> >>>>>>>> the only issue with this server TLS certificates is the chain order >>>>>>>> (CA is Letsencrypt): >>>>>>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> Just curious if there is an update on out of order certs. >>>>>>>> >>>>>>>> The example has id0, id1, id2, id3 certs with id1 being the natural >>>>>>>> root of id2 who is the root of id3, who is the root of id0. >>>>>>>> >>>>>>>> We can correct the out of order problem by including id1,id2,id3 >>>>>>>> certs >>>>>>>> in our chain. >>>>>>>> >>>>>>>> It would be nice to hear from the erlang maintainers around what >>>>>>>> kind of >>>>>>>> "out of order" erlang can handle nicely and if there is planned >>>>>>>> support for >>>>>>>> our case! >>>>>>>> >>>>>>>> Thank you again, >>>>>>>> >>>>>>>> Curtis. >>>>>>>> >>>>>>>> >>>>>>>> Sent through ProtonMail Encrypted Email >>>>>>>> Channel. >>>>>>>> >>>>>>>> >>>>>>>> ??????? Original Message ??????? >>>>>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi! Thank you. >>>>>>>> >>>>>>>> >>>>>>>> I included the root cert in the example. The root cert is id1 in >>>>>>>> cert chain - this is evident in the other file. >>>>>>>> >>>>>>>> It seems because the root cert is out of order - the cert chain is >>>>>>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>>>>>> TLS1.2 >>>>>>>> >>>>>>>> >>>>>>>> Thank you for your consideration! >>>>>>>> >>>>>>>> >>>>>>>> Sent from ProtonMail Mobile >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin < >>>>>>>> ingela.andin@REDACTED> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> "Unknown CA" means that you did not have the ROOT certificate of >>>>>>>> the chian in your "trusted store" (cacerts option). >>>>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>>>> >>>>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>>>> >>>>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>>>>>>> : >>>>>>>> >>>>>>>> Dear Erlang Questions: >>>>>>>> >>>>>>>> >>>>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>>>> >>>>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>>>> for host hooks.glip.com. >>>>>>>> >>>>>>>> When we try to verify peer with the out of order cert >>>>>>>> chain we get 'Unknown CA'. >>>>>>>> >>>>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>>>> >>>>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>>>> mention that other care may need to be taken to ensure >>>>>>>> compatibility. >>>>>>>> >>>>>>>> Reproduce error: >>>>>>>> >>>>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>>>> >>>>>>>> Thank you, >>>>>>>> Curtis and Team DevEco >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> erlang-questions mailing list >>>>>>>> erlang-questions@REDACTED >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From otp@REDACTED Thu Nov 21 13:27:53 2019 From: otp@REDACTED (Erlang/OTP) Date: Thu, 21 Nov 2019 13:27:53 +0100 (CET) Subject: Patch Package OTP 22.1.8 Released Message-ID: <20191121122753.5D99C2546DC@hel.cslab.ericsson.net> Patch Package: OTP 22.1.8 Git Tag: OTP-22.1.8 Date: 2019-11-21 Trouble Report Id: OTP-16301 Seq num: ERL-1079 System: OTP Release: 22 Application: erts-10.5.6 Predecessor: OTP 22.1.7 Check out the git tag OTP-22.1.8, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.5.6 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.5.6 application can be applied independently of other applications on a full OTP 22 installation. --- Fixed Bugs and Malfunctions --- OTP-16301 Application(s): erts Related Id(s): ERL-1079 Large amounts of quickly executed dirty work could cause heavy contention on an internal spin lock. The spin lock was replaced by a mutex which behaves much better under these conditions. Full runtime dependencies of erts-10.5.6: kernel-6.1, sasl-3.3, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From v@REDACTED Thu Nov 21 14:19:15 2019 From: v@REDACTED (Valentin Micic) Date: Thu, 21 Nov 2019 15:19:15 +0200 Subject: INET SETOPTS Message-ID: <97F9C8E2-5DD8-466C-904E-AD31DE75043C@micic.co.za> Hi all, At the present moment (and this seems to have been the case in the past as well), any process that have an access to 'Sock? may issue a successful call to inet:setopts( Sock, [{active, Flag}] ). Is it safe to presume that this is going to stay this way going forward, or, are there plans to change this behaviour to allow, say, only ?controlling process? to issue such command? Thanks in advance. V/ From raimo+erlang-questions@REDACTED Thu Nov 21 15:11:33 2019 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 21 Nov 2019 15:11:33 +0100 Subject: INET SETOPTS In-Reply-To: <97F9C8E2-5DD8-466C-904E-AD31DE75043C@micic.co.za> References: <97F9C8E2-5DD8-466C-904E-AD31DE75043C@micic.co.za> Message-ID: <20191121141133.GA32837@erix.ericsson.se> On Thu, Nov 21, 2019 at 03:19:15PM +0200, Valentin Micic wrote: > Hi all, > > At the present moment (and this seems to have been the case in the past as well), any process that have an access to 'Sock? may issue a successful call to inet:setopts( Sock, [{active, Flag}] ). > Is it safe to presume that this is going to stay this way going forward, or, are there plans to change this behaviour to allow, say, only ?controlling process? to issue such command? We will not change existing API:s such as inet:setopts/2, but a new API, for example socket + net, may benefit from protecting especially options that control process communication such as {active, Flag} to only be used by reasonable processes (the controlling process). > > Thanks in advance. > > V/ -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From v@REDACTED Thu Nov 21 15:31:25 2019 From: v@REDACTED (Valentin Micic) Date: Thu, 21 Nov 2019 16:31:25 +0200 Subject: INET SETOPTS In-Reply-To: <20191121141133.GA32837@erix.ericsson.se> References: <97F9C8E2-5DD8-466C-904E-AD31DE75043C@micic.co.za> <20191121141133.GA32837@erix.ericsson.se> Message-ID: <0E8407A4-8A6C-44A1-8C70-815A17FC1019@micic.co.za> Thank you Raimo. Much appreciated. Kind regards V/ > On 21 Nov 2019, at 16:11, Raimo Niskanen wrote: > > On Thu, Nov 21, 2019 at 03:19:15PM +0200, Valentin Micic wrote: >> Hi all, >> >> At the present moment (and this seems to have been the case in the past as well), any process that have an access to 'Sock? may issue a successful call to inet:setopts( Sock, [{active, Flag}] ). >> Is it safe to presume that this is going to stay this way going forward, or, are there plans to change this behaviour to allow, say, only ?controlling process? to issue such command? > > We will not change existing API:s such as inet:setopts/2, but a new > API, for example socket + net, may benefit from protecting especially options > that control process communication such as {active, Flag} to only be used > by reasonable processes (the controlling process). > >> >> Thanks in advance. >> >> V/ > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB From essen@REDACTED Thu Nov 21 16:17:20 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 21 Nov 2019 16:17:20 +0100 Subject: Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> Message-ID: <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> On 19/11/2019 17:16, Jesper Louis Andersen wrote: > On Mon, Nov 18, 2019 at 2:56 PM Fred Hebert > wrote: > > I do not know about the OTP team's plans with regards to supporting > HTTP/2, but I can say this: HTTP/2 is sufficiently complex and > different from HTTP/1.1 that it's not just a workaround to add > support to it; it essentially asks people to reimplement something > reminiscent of SCTP over TCP in order to provide multiple channels, > and includes things like reimplementing the sliding window > mechanisms that exist within individual TCP connections. The > workaround to "wanting to use HTTP/2" is "use an HTTP/2 library"; > it's too complex to just add rapidly. If not Cowboy and Gun, then > something like Chatterbox can be an acceptable library, and if these > do not work, then using some sort of HTTP2-to-HTTP1 proxy, which > some service providers in the cloud can do for you, generally. > > > And HTTP/3 requires you to implement a dialect of QUIC on top of UDP. > That is, you "just" have to write a complete TCP-like protocol and get > it to run fast. The complexity of this is through the roof. In > principle, this effort should have been modular, so you have a chance to > catch up, but alas, no. I'm not sure I like the direction this is going > in, because the resource investment is so heavy. Mind you, last time > such a thing happened we had XML and then Douglas Crockford comes by > with a hack. The hack then won, despite all its flaws. I'm afraid there isn't going to be a hack for this one. No need to take on the entire complexity all at once though. I've contemplated starting work on a QUIC implementation but I've queued it back to second place so I might only start in 6 months if nobody does before me. I would go at it in this order: * Implement QPACK (easy and more or less independent) * Write the NIF for the QUIC state machine (shortcut! for example the CloudFlare implementation) * Write the network layer. The hard part is TLS 1.3 (including 0-RTT!), but it can be skipped by using a custom cleartext "encryption" mechanism so you can still progress even if you don't have TLS 1.3 yet. Since it's being worked on, waiting is good! * Implement HTTP/3 * Use TLS 1.3 * Get rid of the NIF For the network layer you're never going to have good performance with the current gen_udp interface. You pretty much have to go with socket. Then there's priorities. The good news there is that you can implement them for HTTP/2 first (if the draft is adopted, that is): https://tools.ietf.org/html/draft-kazuho-httpbis-priority-04 I think a "not so bad" full-Erlang implementation is entirely possible by the end of 2020, not too long after the protocol becomes a standard. Then there's the very long optimization step. As to whether an Erlang implementation would perform well enough compared to C, that's another question entirely. But it's on risk I'd be willing to take. Cheers, -- Lo?c Hoguin https://ninenines.eu From luis.rascao@REDACTED Thu Nov 21 16:51:13 2019 From: luis.rascao@REDACTED (=?UTF-8?B?THVpcyBSYXNjw6Nv?=) Date: Thu, 21 Nov 2019 15:51:13 +0000 Subject: Common Tests and logs In-Reply-To: References: Message-ID: Hi, I've seen that partisan has a test suite that uses slave nodes each producing a set of logs (lager) in their own directory, maybe this helps lr On Sun, Nov 17, 2019 at 1:03 PM Roberto Ostinelli wrote: > Hey Andras, > This still doesn't print logs for other nodes AFAIK. I've ended up > redirecting all logs to disk during tests, with something like error_logger:logfile({open, > atom_to_list(node())}). This makes individual log files be created in the > test result's directory... > > If there is a better solution, I'd love to hear it. > > On Sat, Nov 16, 2019 at 11:08 PM Boroska Andr?s > wrote: > >> Hi Roberto, >> >> Did you try io:format/3 where the first argument is the 'user' atom? >> >> Br, >> Andras >> >> On Sat, Nov 16, 2019 at 9:37 PM Roberto Ostinelli >> wrote: >> >>> All, >>> I'm having a hard time to get the logs of slave nodes in Common Tests, >>> started with ct_slave:start/1,2,3. More specifically, when I use ct:pal >>> I only see the messages on the main ct@ node. >>> >>> How can I capture logs running on slave nodes? Given that io:format >>> also does not work in the CT environment, I'm not really sure how to >>> proceed. >>> >>> Thank you, >>> r. >>> >> -- PGP fingerprint: F708 E141 AE8D 2D38 E1BC DF3D 1719 3EA0 647D 7260 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Nov 21 17:22:59 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 21 Nov 2019 17:22:59 +0100 Subject: Inets http2 support In-Reply-To: <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> Message-ID: On Thu, Nov 21, 2019 at 4:17 PM Lo?c Hoguin wrote: > I've contemplated starting work on a QUIC implementation but I've queued > it back to second place so I might only start in 6 months if nobody does > before me. We have also considered doing this. I've already written a nif wrapper for the CloudFlare rust code that can do the very basics, however the steps from basic to fully-fledged implementation seems huge. Also, from what I understand, the cloud flare implementation is not ready for production use, but that will hopefully change in the future. So far my impression has been more similar to what Jesper describes. It will be a lot of work to get correct and fast enough in Erlang. Using some other stack (like the mvfst for instance) would cut out a lot of the complexity, but we lose all of the control of what is going on. Though I suppose that is the same as what happens with TCP today. I've been changing opinion on every other week for a while whether I think it would be a good idea for us to implement it or not, this week I'm against :) The things that I think make this job huge is the flow control, congestion control things and the constant changes to the protocol that seem to be planned. However, if there are more people than us at the OTP team that are interested in helping out to build good support for QUIC, it becomes a lot more interesting. One of the use-cases (besides the obvious HTTP/3 use-case) that we have been thinking about would be to allow the Erlang distribution to run over QUIC. > * Write the network layer. The hard part is TLS 1.3 (including 0-RTT!), > but it can be skipped by using a custom cleartext "encryption" mechanism > so you can still progress even if you don't have TLS 1.3 yet. Since it's > being worked on, waiting is good! > We were thinking about re-using the ssl applications TLS 1.3 implementation. It is not trivial as transport and TLS are upsidedown for QUIC, but should be doable sais Ingela and Peter :p Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Nov 21 17:50:39 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 21 Nov 2019 17:50:39 +0100 Subject: Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> Message-ID: Mind you I also change my mind on the complexity every other week... Ultimately it's just something that has to be done so it doesn't really matter how big the task is. Gotta get started first. From what I've seen of the progress June 2020 or so would be a good time to start as the protocol should be close to completion by then. On 21/11/2019 17:22, Lukas Larsson wrote: > One of the use-cases (besides the obvious HTTP/3 use-case) that we have > been thinking about would be to allow the Erlang distribution to run > over QUIC. That's what I want for RabbitMQ and based on experiments emulating some aspects of QUIC, this is an insanely promising prospect. I expect QUIC to triple the throughput for mirrored RabbitMQ queues *at the very least*. Some aspects of the distribution do not play well with QUIC though, like `monitor(process, {Name, Node})`. This is the one thing in the distribution protocol for which you can't guarantee ordering over multiple QUIC streams because you don't have the pid that you have for all other messages. Right now you also must parse the message in order to figure out on which stream you have to send it, it would probably be better to have this info readily available. Of course you could always use a single QUIC channel but then what's the point. :-) > We were thinking about re-using the ssl applications TLS 1.3 > implementation. It is not trivial as transport and TLS are upsidedown > for QUIC, but should be doable sais Ingela and Peter :p If we can have a good interface that has TLS return the encrypted/decrypted binary data rather than write it directly to the socket, I could also use it in Gun to support TLS connections over TLS proxies. Right now it uses the callback module and while it works it's fairly complex. -- Lo?c Hoguin https://ninenines.eu From lukas@REDACTED Thu Nov 21 18:21:27 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 21 Nov 2019 18:21:27 +0100 Subject: QUIC was Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> Message-ID: On Thu, 21 Nov 2019, 17:50 Lo?c Hoguin, wrote: > Mind you I also change my mind on the complexity every other week... > Ultimately it's just something that has to be done so it doesn't really > matter how big the task is. Indeed. The question however is if it is best to do it like we've done for crypto or for ssl/ssh. Gotta get started first. From what I've seen > of the progress June 2020 or so would be a good time to start as the > protocol should be close to completion by then. > I don't think it is too bad an idea to start now either. From what I can tell the rfc:s are pretty stable already. > On 21/11/2019 17:22, Lukas Larsson wrote: > > One of the use-cases (besides the obvious HTTP/3 use-case) that we have > > been thinking about would be to allow the Erlang distribution to run > > over QUIC. > > That's what I want for RabbitMQ and based on experiments emulating some > aspects of QUIC, this is an insanely promising prospect. I expect QUIC > to triple the throughput for mirrored RabbitMQ queues *at the very least*. > Do you? Why? What aspect of it do you think would do that? > Some aspects of the distribution do not play well with QUIC though, like > `monitor(process, {Name, Node})`. This is the one thing in the > distribution protocol for which you can't guarantee ordering over > multiple QUIC streams because you don't have the pid that you have for > all other messages. Hmm, yes. Not trivial to solve but should be doable. Right now you also must parse the message in order > to figure out on which stream you have to send it, it would probably be > better to have this info readily available. Fixing this should be very simple. The information is there we just need to put it in the tuple returned. Of course you could always > use a single QUIC channel but then what's the point. :-) > > > We were thinking about re-using the ssl applications TLS 1.3 > > implementation. It is not trivial as transport and TLS are upsidedown > > for QUIC, but should be doable sais Ingela and Peter :p > > If we can have a good interface that has TLS return the > encrypted/decrypted binary data rather than write it directly to the > socket, I could also use it in Gun to support TLS connections over TLS > proxies. Right now it uses the callback module and while it works it's > fairly complex. > Maybe this would be a good starting point for us. Building an API that makes this possible. > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Nov 21 19:00:45 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 21 Nov 2019 19:00:45 +0100 Subject: QUIC was Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> Message-ID: <1b298abb-5ff0-dbd8-db0a-d918fc98b6ef@ninenines.eu> On 21/11/2019 18:21, Lukas Larsson wrote: > I don't think it is too bad an idea to start now either. From what I can > tell the rfc:s are pretty stable already. The QUIC ones yes, minus the priority mechanism. But HTTP/3 depends on both priorities being figured out and also the pushed mechanism. I'm not sure how far the latter got or if it's going to be supported at all initially. So for my use case it's not as stable as it could be. > On 21/11/2019 17:22, Lukas Larsson wrote: > > One of the use-cases (besides the obvious HTTP/3 use-case) that > we have > > been thinking about would be to allow the Erlang distribution to run > > over QUIC. > > That's what I want for RabbitMQ and based on experiments emulating some > aspects of QUIC, this is an insanely promising prospect. I expect QUIC > to triple the throughput for mirrored RabbitMQ queues *at the very > least*. > > > Do you? Why? What aspect of it do you think would do that? I said too much. :-) > Some aspects of the distribution do not play well with QUIC though, > like > `monitor(process, {Name, Node})`. This is the one thing in the > distribution protocol for which you can't guarantee ordering over > multiple QUIC streams because you don't have the pid that you have for > all other messages. > > > Hmm, yes. Not trivial to solve but should be doable. My current solution is "don't use it". rpc:call(whereis) plus monitor is almost the same as well. > Right now you also must parse the message in order > to figure out on which stream you have to send it, it would probably be > better to have this info readily available. > > > Fixing this should be very simple. The information is there we just need > to put it in the tuple returned. I'd love to have this today. :-) > Of course you could always > use a single QUIC channel but then what's the point. :-) > > > We were thinking about re-using the ssl applications TLS 1.3 > > implementation. It is not trivial as transport and TLS are > upsidedown > > for QUIC, but should be doable sais Ingela and Peter :p > > If we can have a good interface that has TLS return the > encrypted/decrypted binary data rather than write it directly to the > socket, I could also use it in Gun to support TLS connections over TLS > proxies. Right now it uses the callback module and while it works it's > fairly complex. > > > Maybe this would be a good starting point for us. Building an API that > makes this possible. So much to do so little time. :-) -- Lo?c Hoguin https://ninenines.eu From ingela.andin@REDACTED Thu Nov 21 21:16:00 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 21 Nov 2019 21:16:00 +0100 Subject: Inets http2 support In-Reply-To: <1451515160.1108713.1574073606814@mail.yahoo.com> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> Message-ID: Hi! Regarding the roadmap for inets. The first goal is to strip a lot legacy stuff from inets, to make it easier to maintain. We think that its use case should be small embedded uses and there are other webservices for those who want all the belles and whistles. But as others have pointed out there is no quick fix for HTTP/2 or HTTP/3. At the moment fully supported TLS-1.3 is our priority. And also other underlaying features protocols will have to come first. Regards Ingela Erlang/OTP team - Ericsson AB Den m?n 18 nov. 2019 kl 13:56 skrev josh zinny : > Hi guys, > > I was wondering if http2/3 is one the roadmap for Inets? I know there's is > Cowboy and Gun. > However, I'm interested in the Inets httpd(http2). I have read the Erlang > 22.1 documentation, but it only references http/1.1. > > Also, any suggestions on how to achieve this or a workaround will be > greatly appreciated. > > Many thanks, > Josh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Nov 21 21:23:30 2019 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 21 Nov 2019 21:23:30 +0100 Subject: QUIC was Inets http2 support In-Reply-To: <1b298abb-5ff0-dbd8-db0a-d918fc98b6ef@ninenines.eu> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <1b298abb-5ff0-dbd8-db0a-d918fc98b6ef@ninenines.eu> Message-ID: On Thu, 21 Nov 2019, 19:00 Lo?c Hoguin, wrote: > On 21/11/2019 18:21, Lukas Larsson wrote: > > I don't think it is too bad an idea to start now either. From what I can > > tell the rfc:s are pretty stable already. > > The QUIC ones yes, minus the priority mechanism. But HTTP/3 depends on > both priorities being figured out and also the pushed mechanism. I'm not > sure how far the latter got or if it's going to be supported at all > initially. So for my use case it's not as stable as it could be. > Ah yes, I've not been following those at all as they don't matter for the distribution case. > > On 21/11/2019 17:22, Lukas Larsson wrote: > > > One of the use-cases (besides the obvious HTTP/3 use-case) that > > we have > > > been thinking about would be to allow the Erlang distribution to > run > > > over QUIC. > > > > That's what I want for RabbitMQ and based on experiments emulating > some > > aspects of QUIC, this is an insanely promising prospect. I expect > QUIC > > to triple the throughput for mirrored RabbitMQ queues *at the very > > least*. > > > > > > Do you? Why? What aspect of it do you think would do that? > > I said too much. :-) > > > Some aspects of the distribution do not play well with QUIC though, > > like > > `monitor(process, {Name, Node})`. This is the one thing in the > > distribution protocol for which you can't guarantee ordering over > > multiple QUIC streams because you don't have the pid that you have > for > > all other messages. > > > > > > Hmm, yes. Not trivial to solve but should be doable. > > My current solution is "don't use it". rpc:call(whereis) plus monitor is > almost the same as well. > Mhm, it is rare enough that using some slow fall back would work. > > > Right now you also must parse the message in order > > to figure out on which stream you have to send it, it would probably > be > > better to have this info readily available. > > > > > > Fixing this should be very simple. The information is there we just need > > to put it in the tuple returned. > > I'd love to have this today. :-) > I'll see what we can do. > > Of course you could always > > use a single QUIC channel but then what's the point. :-) > > > > > We were thinking about re-using the ssl applications TLS 1.3 > > > implementation. It is not trivial as transport and TLS are > > upsidedown > > > for QUIC, but should be doable sais Ingela and Peter :p > > > > If we can have a good interface that has TLS return the > > encrypted/decrypted binary data rather than write it directly to the > > socket, I could also use it in Gun to support TLS connections over > TLS > > proxies. Right now it uses the callback module and while it works > it's > > fairly complex. > > > > > > Maybe this would be a good starting point for us. Building an API that > > makes this possible. > > So much to do so little time. :-) > Indeed. > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Nov 21 22:02:22 2019 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 21 Nov 2019 22:02:22 +0100 Subject: QUIC was Inets http2 support In-Reply-To: <1b298abb-5ff0-dbd8-db0a-d918fc98b6ef@ninenines.eu> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <1b298abb-5ff0-dbd8-db0a-d918fc98b6ef@ninenines.eu> Message-ID: Hi! > > > > > We were thinking about re-using the ssl applications TLS 1.3 > > > implementation. It is not trivial as transport and TLS are > > upsidedown > > > for QUIC, but should be doable sais Ingela and Peter :p > > > I think it is doable. I think the big job is implementing the QUIC state machine. But maybe we need a QUIC flavoured TLS state machine that reuses much of the same code. Similar to that we have a TLS statemachine, a TLS-1.3 statmachine and DTLS statemachine at the moment. > If we can have a good interface that has TLS return the > > encrypted/decrypted binary data rather than write it directly to the > > socket, I could also use it in Gun to support TLS connections over > TLS > > proxies. Right now it uses the callback module and while it works > it's > > fairly complex. > > > Maybe this would be a good starting point for us. Building an API that > > makes this possible. > > We can think about it. This will probably not do it for QUIC as it wants to handle the encryption part itself. But I think there are many places where can make possible new APIs. So much to do so little time. :-) > > Like always, but I do not think QUIC will replace TCP and day soon... I think the trick is to find small steps. Regards Ingela Erlang/OTP team - Ericsson AB > -- > Lo?c Hoguin > https://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michaelviveros@REDACTED Thu Nov 21 23:00:59 2019 From: michaelviveros@REDACTED (Michael Viveros) Date: Thu, 21 Nov 2019 17:00:59 -0500 Subject: [erlang-questions] SSL Out of Order Cert Chain Question (9.2) In-Reply-To: References: <9ZVU-GXPquw0qkkxp1hrLF5qWBG0x8GWKm-ytdhfqeeQIl4dqt_pheziUaWjsXsQBcAhlWFRkur6-qAGXkLkyUgTMU7Y3QL5afob0aN_9Pg=@ram9.cc> <0gZbavXqk2pscGQB8vdgUqxiIBbjyDCO0OaHc5st0MuCZ4Sae37BOpjuVetffteASs2QxjDnK3keE10HoqaGKfgOWlNNsV5ctxTb5peH0wQ=@ram9.cc> Message-ID: Heard back from Ingela and she confirmed it will be part of OTP-22.2 and OTP-23.0 as opposed to being a patch to OTP-21 (since it's not a vulnerability or critical bug). On Wed, Nov 20, 2019 at 9:22 PM Michael Viveros wrote: > Any update about next steps, Ingela? > > On Tue, Nov 12, 2019 at 5:47 PM Michael Viveros > wrote: > >> Just confirmed your fix worked! >> >> On Mon, Nov 11, 2019 at 6:16 PM Michael Viveros >> wrote: >> >>> Great news, thanks Ingela! I tried confirming the fix on my end but I'm >>> an Erlang noob so it's taking me awhile, will try again tomorrow. >>> >>> On Mon, Nov 11, 2019 at 5:17 AM Ingela Andin >>> wrote: >>> >>>> Hi! >>>> >>>> I have a patch that solves the problem. I have pushed it to my gitrepo >>>> https://github.com/IngelaAndin/otp/tree/ingela/ssl/unorded-chain >>>> >>>> I had to set the options {depth, 2} and {customize_hostname_check, >>>> [{match_fun, CustomFun}] >>>> where CustomFun = public_key:pkix_verify_hostname_match_fun(https) as >>>> the peer certificate is a wildcard cert. >>>> >>>> >>>> Regard Ingela Erlang/OTP team >>>> >>>> >>>> Den fre 8 nov. 2019 kl 21:46 skrev Curtis J Schofield : >>>> >>>>> Oh this is wonderful news!! Glad you were able to identify the code >>>>> not reached !! >>>>> >>>>> Deeply appreciate your support and expertise! >>>>> >>>>> Best, >>>>> Curtis >>>>> >>>>> Sent from ProtonMail Mobile >>>>> >>>>> >>>>> On Fri, Nov 8, 2019 at 12:15 PM, Ingela Andin >>>>> wrote: >>>>> >>>>> Hi! >>>>> >>>>> I think I am on to the problem, we have only whitebox tested the >>>>> unorded chain functionality (we intend to create a blackbox but that is a >>>>> bigger job as we need to find some introp software that can create such >>>>> chains or create a simulation modle), >>>>> so I am positive I found that we do not reach the code for sorting >>>>> the chain. I will try to fix it next week. It is easier now that I at >>>>> least I got a server that I can manually blackbox test with :) >>>>> >>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>> >>>>> Den tors 7 nov. 2019 kl 20:53 skrev Ingela Andin < >>>>> ingela.andin@REDACTED>: >>>>> >>>>>> Hi! >>>>>> >>>>>> Den tors 7 nov. 2019 kl 19:35 skrev Michael Viveros < >>>>>> michaelviveros@REDACTED>: >>>>>> >>>>>>> Hi Ingela, >>>>>>> >>>>>>> Curtis' example server from his first message, hooks.glip.com, >>>>>>> presents its certificates out-of-order. The correct order is Peer -> >>>>>>> Intermediate CA 1 - > Intermediate CA 2 -> Root CA but they get presented >>>>>>> as Peer -> Root CA -> Intermediate CA 2 -> Intermediate CA 1 and this >>>>>>> returns the "Unknown CA" error. You can confirm this by running `openssl >>>>>>> s_client -connect hooks.glip.com:443`. >>>>>>> >>>>>>> >>>>>> Yes I agree that this is an out of order chain, in contrast to the >>>>>> social.fluffel.io. I will look into it at work tomorrow. >>>>>> >>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>> >>>>>> >>>>>> >>>>>>> On Thu, Nov 7, 2019 at 1:23 PM Curtis J Schofield >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Ingela >>>>>>>> >>>>>>>> Thank you for your attention- perhaps Micheal can explain this >>>>>>>> better.. >>>>>>>> >>>>>>>> Sent from ProtonMail Mobile >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Nov 7, 2019 at 6:55 AM, Ingela Andin < >>>>>>>> ingela.andin@REDACTED> wrote: >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> I tried this out and it is not out of order, it sends the peer cert >>>>>>>> followed by the intermediate cert repeated, that is the chain looks like >>>>>>>> [Peer, CA1, CA1]. >>>>>>>> Looking at TLS-1.3 RFC it looks like extra certs should ignored >>>>>>>> too, so I suppose we need to add that. >>>>>>>> >>>>>>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>>>>>> >>>>>>>> Den l?r 2 nov. 2019 kl 15:24 skrev Mark Reynolds : >>>>>>>> >>>>>>>>> Hey, >>>>>>>>> >>>>>>>>> I confirm that out of order certs does not seems to be fixed, and >>>>>>>>> it fails with 'Unknown CA' error: >>>>>>>>> >>>>>>>>> >>>>>>>>> iex(2)> :hackney.get("https://social.fluffel.io") >>>>>>>>> {:error, >>>>>>>>> {:tls_alert, {:unknown_ca, 'received CLIENT ALERT: Fatal - Unknown >>>>>>>>> CA'}}} >>>>>>>>> >>>>>>>>> >>>>>>>>> the only issue with this server TLS certificates is the chain >>>>>>>>> order (CA is Letsencrypt): >>>>>>>>> https://www.ssllabs.com/ssltest/analyze.html?d=social.fluffel.io >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Nov 2, 2019, at 01:12, Curtis J Schofield wrote: >>>>>>>>> >>>>>>>>> Hi! >>>>>>>>> >>>>>>>>> Just curious if there is an update on out of order certs. >>>>>>>>> >>>>>>>>> The example has id0, id1, id2, id3 certs with id1 being the >>>>>>>>> natural >>>>>>>>> root of id2 who is the root of id3, who is the root of id0. >>>>>>>>> >>>>>>>>> We can correct the out of order problem by including id1,id2,id3 >>>>>>>>> certs >>>>>>>>> in our chain. >>>>>>>>> >>>>>>>>> It would be nice to hear from the erlang maintainers around what >>>>>>>>> kind of >>>>>>>>> "out of order" erlang can handle nicely and if there is planned >>>>>>>>> support for >>>>>>>>> our case! >>>>>>>>> >>>>>>>>> Thank you again, >>>>>>>>> >>>>>>>>> Curtis. >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent through ProtonMail Encrypted Email >>>>>>>>> Channel. >>>>>>>>> >>>>>>>>> >>>>>>>>> ??????? Original Message ??????? >>>>>>>>> On Saturday, October 19, 2019 4:34 PM, Curtis J Schofield >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi! Thank you. >>>>>>>>> >>>>>>>>> >>>>>>>>> I included the root cert in the example. The root cert is id1 in >>>>>>>>> cert chain - this is evident in the other file. >>>>>>>>> >>>>>>>>> It seems because the root cert is out of order - the cert chain is >>>>>>>>> invalid - IIRC this may be true for tls1.2 - however the negotiation is at >>>>>>>>> TLS1.2 >>>>>>>>> >>>>>>>>> >>>>>>>>> Thank you for your consideration! >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent from ProtonMail Mobile >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Oct 19, 2019 at 10:51 AM, Ingela Andin < >>>>>>>>> ingela.andin@REDACTED> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi! >>>>>>>>> >>>>>>>>> "Unknown CA" means that you did not have the ROOT certificate of >>>>>>>>> the chian in your "trusted store" (cacerts option). >>>>>>>>> If you do not own the ROOT certificate you can not trust the chain. >>>>>>>>> >>>>>>>>> Regards Ingela Erlang/OTP Team - Ericsson AB >>>>>>>>> >>>>>>>>> Den fre 18 okt. 2019 kl 21:52 skrev Curtis J Schofield >>>>>>>>> : >>>>>>>>> >>>>>>>>> Dear Erlang Questions: >>>>>>>>> >>>>>>>>> >>>>>>>>> SSL 9.0.2 mentions a patch to fix out of order cert chains >>>>>>>>> >>>>>>>>> In SSL 9.2 we have a root CA and an out of order cert chain >>>>>>>>> for host hooks.glip.com. >>>>>>>>> >>>>>>>>> When we try to verify peer with the out of order cert >>>>>>>>> chain we get 'Unknown CA'. >>>>>>>>> >>>>>>>>> Is this expected behaviour for Erlang SSL 9.2 with verify_peer ? >>>>>>>>> >>>>>>>>> The http://erlang.org/doc/apps/ssl/notes.html#ssl-9.0.2 notes >>>>>>>>> mention that other care may need to be taken to ensure >>>>>>>>> compatibility. >>>>>>>>> >>>>>>>>> Reproduce error: >>>>>>>>> >>>>>>>>> https://github.com/robotarmy/out-of-order-ssl >>>>>>>>> >>>>>>>>> Thank you, >>>>>>>>> Curtis and Team DevEco >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Sent through ProtonMail Encrypted Email Channel. >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> erlang-questions mailing list >>>>>>>>> erlang-questions@REDACTED >>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Fri Nov 22 21:03:38 2019 From: dch@REDACTED (Dave Cottlehuber) Date: Fri, 22 Nov 2019 20:03:38 +0000 Subject: Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> Message-ID: <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> On Thu, 21 Nov 2019, at 16:50, Lo?c Hoguin wrote: > Mind you I also change my mind on the complexity every other week... > Ultimately it's just something that has to be done so it doesn't really > matter how big the task is. Gotta get started first. From what I've seen > of the progress June 2020 or so would be a good time to start as the > protocol should be close to completion by then. > > On 21/11/2019 17:22, Lukas Larsson wrote: > > One of the use-cases (besides the obvious HTTP/3 use-case) that we have > > been thinking about would be to allow the Erlang distribution to run > > over QUIC. Had you considered something like nng instead of HTTP/3 for this? - already available[1] - much much simpler than HTTP/3/QUIC (I would hate to count the lines of code) - the survey pattern[2] looks ideal for maintaining heartbeats I've fiddled with enm[3] https://github.com/basho/enm which is wire compatible, and it's very fast indeed. [1]: https://github.com/nanomsg/nng [2]: https://nanomsg.org/gettingstarted/nng/survey.html [3]: https://github.com/basho/enm From essen@REDACTED Sat Nov 23 09:04:43 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 23 Nov 2019 09:04:43 +0100 Subject: Inets http2 support In-Reply-To: <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> Message-ID: <37281c1c-696a-b6fc-9337-b2bfc6d2be71@ninenines.eu> On 22/11/2019 21:03, Dave Cottlehuber wrote: >> On 21/11/2019 17:22, Lukas Larsson wrote: >>> One of the use-cases (besides the obvious HTTP/3 use-case) that we have >>> been thinking about would be to allow the Erlang distribution to run >>> over QUIC. > > Had you considered something like nng instead of HTTP/3 for this? You mean QUIC, no need for HTTP/3 for the distribution. > - already available[1] > - much much simpler than HTTP/3/QUIC (I would hate to count the lines of code) > - the survey pattern[2] looks ideal for maintaining heartbeats > > I've fiddled with enm[3] https://github.com/basho/enm which is wire compatible, > and it's very fast indeed. > > [1]: https://github.com/nanomsg/nng > [2]: https://nanomsg.org/gettingstarted/nng/survey.html > [3]: https://github.com/basho/enm From what I understand it's still built on top of TCP. The main advantage of QUIC is that you can map messages onto different streams. Each stream's messages are ordered but the whole connection is not. Streams don't block each other. Different streams may have different priorities (we could have a process_flag that indicates this for very important processes and the distribution driver would send their messages before others). Cheers, -- Lo?c Hoguin https://ninenines.eu From rtp11@REDACTED Sat Nov 23 10:47:00 2019 From: rtp11@REDACTED (rtp) Date: Sat, 23 Nov 2019 10:47:00 +0100 Subject: rebar3 compile dependencies separated from project code Message-ID: Hi, I'm build up a docker pipeline for a erlang project. Currently I build the whole project in a new erlang-Docker-Container by a rebar3 as prod tar This task downloads and compiles the dependencies every time I build; because the erlang-container is plain and empty. I can now call a rebar3 get-deps in a previous container (or step) to get rid of the downloads. But I want to get a step further an compile the dependencies before building the application, to compile only the app-code and to make the build faster. Does anybody know how to compile dependencies separated from app-code ? I can't find anything. Perhaps there's a plugin I don't know!? Thanks a lot Ralf From rtp11@REDACTED Sat Nov 23 11:04:40 2019 From: rtp11@REDACTED (rtp) Date: Sat, 23 Nov 2019 11:04:40 +0100 Subject: rebar3 compile dependencies separated from project code In-Reply-To: References: Message-ID: Found a solution just now :-D Just copy only the rebar.conf-file in a first step and compile it (I didn't expect to work ... but it does) : FROM erlang:22 WORKDIR /BUILD/ # get deps in a first step - to use cache in next build ;-) COPY erlang/aw/rebar.config . RUN rebar3 compile # now get whole code COPY erlang/aw/ . RUN rebar3 as prod tar Perhaps it helps someone :-) Cheerio Ralf On 23.11.19 10:47, rtp wrote: > Hi, > > I'm build up a docker pipeline for a erlang project. Currently I build > the whole project in a new erlang-Docker-Container by a > > rebar3 as prod tar > > This task downloads and compiles the dependencies every time I build; > because the erlang-container is plain and empty. > > I can now call a > > rebar3 get-deps > > in a previous container (or step) to get rid of the downloads. > > But I want to get a step further an compile the dependencies before > building the application, to compile only the app-code and to make the > build faster. > > Does anybody know how to compile dependencies separated from app-code ? > > I can't find anything. Perhaps there's a plugin I don't know!? > > > Thanks a lot > > Ralf > From eric.pailleau@REDACTED Sat Nov 23 13:38:06 2019 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Sat, 23 Nov 2019 13:38:06 +0100 Subject: Inets http2 support In-Reply-To: <37281c1c-696a-b6fc-9337-b2bfc6d2be71@ninenines.eu> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> <37281c1c-696a-b6fc-9337-b2bfc6d2be71@ninenines.eu> Message-ID: QUIC : "To solve this problem, QUIC includes a connection identifier which uniquely identifies the connection to the server regardless of source. This allows the connection to be re-established simply by sending a packet, which always contains this ID, as the original connection ID will still be valid even if the user's IP address changes." Wow... Nice security issues in perspective! Envoy? depuis mon mobile ---- Lo?c Hoguin a ?crit ---- >On 22/11/2019 21:03, Dave Cottlehuber wrote: >>> On 21/11/2019 17:22, Lukas Larsson wrote: >>>> One of the use-cases (besides the obvious HTTP/3 use-case) that we have >>>> been thinking about would be to allow the Erlang distribution to run >>>> over QUIC. >> >> Had you considered something like nng instead of HTTP/3 for this? > >You mean QUIC, no need for HTTP/3 for the distribution. > >> - already available[1] >> - much much simpler than HTTP/3/QUIC (I would hate to count the lines of code) >> - the survey pattern[2] looks ideal for maintaining heartbeats >> >> I've fiddled with enm[3] https://github.com/basho/enm which is wire compatible, >> and it's very fast indeed. >> >> [1]: https://github.com/nanomsg/nng >> [2]: https://nanomsg.org/gettingstarted/nng/survey.html >> [3]: https://github.com/basho/enm > > From what I understand it's still built on top of TCP. The main >advantage of QUIC is that you can map messages onto different streams. >Each stream's messages are ordered but the whole connection is not. >Streams don't block each other. Different streams may have different >priorities (we could have a process_flag that indicates this for very >important processes and the distribution driver would send their >messages before others). > >Cheers, > >-- >Lo?c Hoguin >https://ninenines.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Nov 23 17:20:43 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 23 Nov 2019 17:20:43 +0100 Subject: uri_string weird inconsistency Message-ID: <91625bb8-47a4-bd68-ef52-c86e90cae28f@ninenines.eu> Hello, Why does uri_string:normalize/1 catch a throw and return an error tuple, but uri_string:normalize/2 does not? Both are documented functions and this behavior difference is not documented, besides the 2-arity not having error() as a returned type: http://erlang.org/doc/man/uri_string.html#normalize-1 Is this an oversight perhaps? Cheers, -- Lo?c Hoguin https://ninenines.eu From matthias@REDACTED Sun Nov 24 00:51:53 2019 From: matthias@REDACTED (Matthias Lang) Date: Sun, 24 Nov 2019 00:51:53 +0100 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> References: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> Message-ID: <20191123235153.GA12317@hec.corelatus.se> Jumping into a thread that's a few weeks old. My first thought was to see what the 'Barklund' draft says: | An empty list precedes a cons ... and conses are ordered first by | their heads, then by their tails. (Thus a longer list may preced a | shorter list even though a shorter list tuple always precedes a | longer tuple.) For example, [] < [a|2] <[a|b] < [a] < [a, a], [b]. | | http://www.erlang.org/download/erl_spec47.ps.gz So far no surprises, but then it says | Binaries are ordered first by their size, then according to their | elements lexicographically. (That is, the same as the order | between tuples of integers. So I think the Barklund draft doesn't agree with the implementation. I went back to OTP R8B. Even that doesn't agree with the Barklund draft: 1> init:script_id(). {"OTP APN 181 01","R8B"} // I think R8B is from about 2002 2> <<0,0,1>> < <<1,0>>. true I'm not sure if the Barklund draft never agreed with the implementation, or if the implementation changed some time before 2003. Also, it seems that the current reference manual defines what happens for lists, but not binaries and bit strings: http://erlang.org/doc/reference_manual/expressions.html#term-comparisons Matthias ---------------------------------------------------------------------- Date: 28. October 2019 From: Raimo Niskanen To erlang-questions@REDACTED Subject: Re: Binary, List and Tuple Inequalities (Paradox?) > I'd like to defend the current term order of lists, tuples and > binaries. > > Lists are compared a'la string comparison, which avoids having to > traverse the whole list when comparing. Binaries are compared as lists > since both are used for handling strings, and also to be compared like > memcmp does it. > > Tuples are used for general complex terms. They can be compared size > first whithout having to be fully traversed, as opposed to lists, which > is useful in other contexts. You can thereby choose your data > structure to select comparision. > -- > Raimo Niskanen > From v@REDACTED Sun Nov 24 08:24:48 2019 From: v@REDACTED (Valentin Micic) Date: Sun, 24 Nov 2019 09:24:48 +0200 Subject: Binary, List and Tuple Inequalities (Paradox?) In-Reply-To: <20191123235153.GA12317@hec.corelatus.se> References: <180ae745ba2acccfadafebe278a45155a68ab461.camel@ericsson.com> <20191123235153.GA12317@hec.corelatus.se> Message-ID: Thank you for jumping in. :-) Am I alone in thinking that Barklund makes sense? V/ > On 24 Nov 2019, at 01:51, Matthias Lang wrote: > > Jumping into a thread that's a few weeks old. > > My first thought was to see what the 'Barklund' draft says: > > | An empty list precedes a cons ... and conses are ordered first by > | their heads, then by their tails. (Thus a longer list may preced a > | shorter list even though a shorter list tuple always precedes a > | longer tuple.) For example, [] < [a|2] <[a|b] < [a] < [a, a], [b]. > | > | http://www.erlang.org/download/erl_spec47.ps.gz > > So far no surprises, but then it says > > | Binaries are ordered first by their size, then according to their > | elements lexicographically. (That is, the same as the order > | between tuples of integers. > > So I think the Barklund draft doesn't agree with the implementation. > > I went back to OTP R8B. Even that doesn't agree with the Barklund draft: > > 1> init:script_id(). > {"OTP APN 181 01","R8B"} // I think R8B is from about 2002 > 2> <<0,0,1>> < <<1,0>>. > true > > I'm not sure if the Barklund draft never agreed with the implementation, > or if the implementation changed some time before 2003. > > Also, it seems that the current reference manual defines what happens > for lists, but not binaries and bit strings: > > http://erlang.org/doc/reference_manual/expressions.html#term-comparisons > > Matthias > > ---------------------------------------------------------------------- > Date: 28. October 2019 From: Raimo Niskanen > To erlang-questions@REDACTED Subject: > Re: Binary, List and Tuple Inequalities (Paradox?) > > >> I'd like to defend the current term order of lists, tuples and >> binaries. >> >> Lists are compared a'la string comparison, which avoids having to >> traverse the whole list when comparing. Binaries are compared as lists >> since both are used for handling strings, and also to be compared like >> memcmp does it. >> >> Tuples are used for general complex terms. They can be compared size >> first whithout having to be fully traversed, as opposed to lists, which >> is useful in other contexts. You can thereby choose your data >> structure to select comparision. >> -- >> Raimo Niskanen >> From ostinelli@REDACTED Sun Nov 24 20:28:57 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 24 Nov 2019 20:28:57 +0100 Subject: global lock Message-ID: All, In my cluster some nodes can generate events when a consistency issue is detected. When this happens, I'd like one node (and only one) to resolve this conflict. global:trans/4 allows to set the number of retries, so one approach could be to set retries to 0 so *if* my understanding is correct, only 1 node would do the conflict resolution (the one that grabs the lock). If some kind soul can confirm that this is correct, thank you in advance. :) Otherwise, what other options do I have? For example, can I set a flag somehow so that the nodes that run the code in the global:trans know that this has already been done by another node? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peterdmv@REDACTED Mon Nov 25 10:56:01 2019 From: peterdmv@REDACTED (Peter Dimitrov) Date: Mon, 25 Nov 2019 10:56:01 +0100 Subject: uri_string weird inconsistency Message-ID: Hi, This is a bug in the type spec and as far as I see the catch is unnecessary in uri_string:normalize/1 as exceptions are already caught in all other functions that are called by normalize. The intended behavior is to always return an error tuple in case of an error. BR/Peter -----Original Message----- From: erlang-questions On Behalf Of Lo?c Hoguin Sent: den 23 november 2019 17:21 To: Erlang Questions Subject: uri_string weird inconsistency Hello, Why does uri_string:normalize/1 catch a throw and return an error tuple, but uri_string:normalize/2 does not? Both are documented functions and this behavior difference is not documented, besides the 2-arity not having error() as a returned type: https://protect2.fireeye.com/v1/url?k=8b83a40f-d70986e6-8b83e494-0cc47ad93e2e-315c3a8c3f88d867&q=1&e=db14f46e-9908-4d65-828f-a5804e01ef18&u=http%3A%2F%2Ferlang.org%2Fdoc%2Fman%2Furi_string.html%23normalize-1 Is this an oversight perhaps? Cheers, -- Lo?c Hoguin https://protect2.fireeye.com/v1/url?k=3f9e31b6-6314135f-3f9e712d-0cc47ad93e2e-d1a6f2e3a8affc9e&q=1&e=db14f46e-9908-4d65-828f-a5804e01ef18&u=https%3A%2F%2Fninenines.eu%2F -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Mon Nov 25 17:21:05 2019 From: paulperegud@REDACTED (Paul Peregud) Date: Mon, 25 Nov 2019 16:21:05 +0000 Subject: global lock In-Reply-To: References: Message-ID: > global:trans/4 allows to set the number of retries, so one approach could be to set retries to 0 so *if* my understanding is correct, only 1 node would do the conflict resolution (the one that grabs the lock). If some kind soul can confirm that this is correct, thank you in advance. :) That's not enough. Alice may detect the issue, grab the lock, resolve the issue and release the lock before Bob receives the signal and attempts to grab the lock. You need some kind of ConflictId -> resolved :: boolean() mapping. From ostinelli@REDACTED Mon Nov 25 17:43:32 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Mon, 25 Nov 2019 17:43:32 +0100 Subject: global lock In-Reply-To: References: Message-ID: Ok, so indeed the flag I was referring to seems to be a necessity. Thank you, r. On Mon, Nov 25, 2019 at 5:21 PM Paul Peregud wrote: > > global:trans/4 allows to set the number of retries, so one approach > could be to set retries to 0 so *if* my understanding is correct, only 1 > node would do the conflict resolution (the one that grabs the lock). If > some kind soul can confirm that this is correct, thank you in advance. :) > > That's not enough. Alice may detect the issue, grab the lock, resolve > the issue and release the lock before Bob receives the signal and > attempts to grab the lock. You need some kind of ConflictId -> > resolved :: boolean() mapping. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Tue Nov 26 01:39:51 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 26 Nov 2019 07:39:51 +0700 Subject: Hackney cannot access HTTPS - CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data Message-ID: Hi, I use hackney to fetch a web page and i get error when accessing website with https with following error: TLS client: In state certify at tls_connection.erl:966 generated CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data ** exception error: no match of right hand side value {error, {tls_alert, {handshake_failure, "received CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data"}}} in function handler_test_test_test:fetch_page/1 (/Users/okaprinarjaya/Oprek/Erlang-Oprek-Tiga/idea_execute/src/handler_test_test_test.erl, line 37) Then I found this: https://github.com/benoitc/hackney/issues/362 . I try to implement that github issue but still failed and still get same error message. My code based on github issue: https://pastebin.com/QeCc0tab My environment: 1. MacOS Mojave v10.14.6 2. Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] 3. I install hackney with this way through rebar3: {hackney, ".*", {git, "git://github.com/benoitc/hackney.git", {branch, "master"}}} and my current hackney version is 1.15.2 ssl:cipher_suites() [{ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, {ecdhe_rsa,aes_256_cbc,sha384,sha384}, {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, {ecdh_rsa,aes_256_cbc,sha384,sha384}, {dhe_rsa,aes_256_cbc,sha256}, {dhe_dss,aes_256_cbc,sha256}, {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, {ecdhe_rsa,aes_128_cbc,sha256,sha256}, {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, {ecdh_rsa,aes_128_cbc,sha256,sha256}, {dhe_rsa,aes_128_cbc,sha256}, {dhe_dss,aes_128_cbc,sha256}, {ecdhe_ecdsa,aes_256_cbc,sha}, {ecdhe_rsa,aes_256_cbc,sha}, {dhe_rsa,aes_256_cbc,sha}, {dhe_dss,aes_256_cbc,sha}, {ecdh_ecdsa,aes_256_cbc,sha}, {ecdh_rsa,aes_256_cbc,sha}, {ecdhe_ecdsa,aes_128_cbc,sha}, {ecdhe_rsa,aes_128_cbc,sha}, {dhe_rsa,aes_128_cbc,sha}, {dhe_dss,aes_128_cbc,sha}, {ecdh_ecdsa,aes_128_cbc,sha}, {ecdh_rsa,aes_128_cbc,sha}] Please enlightenment Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From bram.verburg@REDACTED Tue Nov 26 08:00:40 2019 From: bram.verburg@REDACTED (Bram Verburg) Date: Tue, 26 Nov 2019 09:00:40 +0200 Subject: Hackney cannot access HTTPS - CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data In-Reply-To: References: Message-ID: Which exact Erlang/OTP version are you on? It seems to be 21.3, and if you're not on the very latest patch level you might get bitten by ssl bugs such as https://bugs.erlang.org/browse/ERL-968. That particular one was fixed in 21.3.8.5, and even though it was originally reported with a slightly different error response, the root cause does make handshake data appear malformed to the ssl state machine. If that's not the issue, can you perhaps share a URL that triggers the issue? On Tue, 26 Nov 2019 at 02:40, I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> wrote: > Hi, > > I use hackney to fetch a web page and i get error when accessing website > with https with following error: > > TLS client: In state certify at tls_connection.erl:966 generated CLIENT > ALERT: Fatal - Handshake Failure - malformed_handshake_data > > ** exception error: no match of right hand side value {error, > {tls_alert, > {handshake_failure, > "received CLIENT > ALERT: Fatal - Handshake Failure - malformed_handshake_data"}}} > in function handler_test_test_test:fetch_page/1 > (/Users/okaprinarjaya/Oprek/Erlang-Oprek-Tiga/idea_execute/src/handler_test_test_test.erl, > line 37) > > Then I found this: https://github.com/benoitc/hackney/issues/362 . I try > to implement that github issue but still failed and still get same error > message. My code based on github issue: https://pastebin.com/QeCc0tab > > My environment: > 1. MacOS Mojave v10.14.6 > 2. Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] > [async-threads:1] > 3. I install hackney with this way through rebar3: > {hackney, ".*", {git, "git://github.com/benoitc/hackney.git", {branch, > "master"}}} > and my current hackney version is 1.15.2 > > ssl:cipher_suites() > > [{ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, > {ecdhe_rsa,aes_256_cbc,sha384,sha384}, > {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, > {ecdh_rsa,aes_256_cbc,sha384,sha384}, > {dhe_rsa,aes_256_cbc,sha256}, > {dhe_dss,aes_256_cbc,sha256}, > {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, > {ecdhe_rsa,aes_128_cbc,sha256,sha256}, > {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, > {ecdh_rsa,aes_128_cbc,sha256,sha256}, > {dhe_rsa,aes_128_cbc,sha256}, > {dhe_dss,aes_128_cbc,sha256}, > {ecdhe_ecdsa,aes_256_cbc,sha}, > {ecdhe_rsa,aes_256_cbc,sha}, > {dhe_rsa,aes_256_cbc,sha}, > {dhe_dss,aes_256_cbc,sha}, > {ecdh_ecdsa,aes_256_cbc,sha}, > {ecdh_rsa,aes_256_cbc,sha}, > {ecdhe_ecdsa,aes_128_cbc,sha}, > {ecdhe_rsa,aes_128_cbc,sha}, > {dhe_rsa,aes_128_cbc,sha}, > {dhe_dss,aes_128_cbc,sha}, > {ecdh_ecdsa,aes_128_cbc,sha}, > {ecdh_rsa,aes_128_cbc,sha}] > > Please enlightenment > > Thank you > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From okaprinarjaya@REDACTED Tue Nov 26 09:02:48 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Tue, 26 Nov 2019 15:02:48 +0700 Subject: Hackney cannot access HTTPS - CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data In-Reply-To: References: Message-ID: Hi, Thank you for your reply, Unfortunately for now, I don't know exactly how to identify exact detail patch level version for my installed Erlang/OTP. I've try to run: erlang:display(erlang:system_info(otp_release)). but that command only displaying "21" {ok, Version} = file:read_file(filename:join([code:root_dir(), "releases", erlang:system_info(otp_release), "OTP_VERSION"])), io:fwrite(Version). that command only displaying 21.3 And, how to update my Erlang/OTP version to latest Erlang/OTP 21.3 patch release using kerl ? I install Erlang/OTP distribution using kerl at first. so i hope kerl also able to update existing installation. Thank you Pada tanggal Sel, 26 Nov 2019 pukul 14.00 Bram Verburg < bram.verburg@REDACTED> menulis: > Which exact Erlang/OTP version are you on? It seems to be 21.3, and if > you're not on the very latest patch level you might get bitten by ssl bugs > such as https://bugs.erlang.org/browse/ERL-968. That particular one was > fixed in 21.3.8.5, and even though it was originally reported with a > slightly different error response, the root cause does make handshake data > appear malformed to the ssl state machine. > > If that's not the issue, can you perhaps share a URL that triggers the > issue? > > > On Tue, 26 Nov 2019 at 02:40, I Gusti Ngurah Oka Prinarjaya < > okaprinarjaya@REDACTED> wrote: > >> Hi, >> >> I use hackney to fetch a web page and i get error when accessing website >> with https with following error: >> >> TLS client: In state certify at tls_connection.erl:966 generated CLIENT >> ALERT: Fatal - Handshake Failure - malformed_handshake_data >> >> ** exception error: no match of right hand side value {error, >> {tls_alert, >> >> {handshake_failure, >> "received CLIENT >> ALERT: Fatal - Handshake Failure - malformed_handshake_data"}}} >> in function handler_test_test_test:fetch_page/1 >> (/Users/okaprinarjaya/Oprek/Erlang-Oprek-Tiga/idea_execute/src/handler_test_test_test.erl, >> line 37) >> >> Then I found this: https://github.com/benoitc/hackney/issues/362 . I try >> to implement that github issue but still failed and still get same error >> message. My code based on github issue: https://pastebin.com/QeCc0tab >> >> My environment: >> 1. MacOS Mojave v10.14.6 >> 2. Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] >> [async-threads:1] >> 3. I install hackney with this way through rebar3: >> {hackney, ".*", {git, "git://github.com/benoitc/hackney.git", {branch, >> "master"}}} >> and my current hackney version is 1.15.2 >> >> ssl:cipher_suites() >> >> [{ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, >> {ecdhe_rsa,aes_256_cbc,sha384,sha384}, >> {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, >> {ecdh_rsa,aes_256_cbc,sha384,sha384}, >> {dhe_rsa,aes_256_cbc,sha256}, >> {dhe_dss,aes_256_cbc,sha256}, >> {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, >> {ecdhe_rsa,aes_128_cbc,sha256,sha256}, >> {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, >> {ecdh_rsa,aes_128_cbc,sha256,sha256}, >> {dhe_rsa,aes_128_cbc,sha256}, >> {dhe_dss,aes_128_cbc,sha256}, >> {ecdhe_ecdsa,aes_256_cbc,sha}, >> {ecdhe_rsa,aes_256_cbc,sha}, >> {dhe_rsa,aes_256_cbc,sha}, >> {dhe_dss,aes_256_cbc,sha}, >> {ecdh_ecdsa,aes_256_cbc,sha}, >> {ecdh_rsa,aes_256_cbc,sha}, >> {ecdhe_ecdsa,aes_128_cbc,sha}, >> {ecdhe_rsa,aes_128_cbc,sha}, >> {dhe_rsa,aes_128_cbc,sha}, >> {dhe_dss,aes_128_cbc,sha}, >> {ecdh_ecdsa,aes_128_cbc,sha}, >> {ecdh_rsa,aes_128_cbc,sha}] >> >> Please enlightenment >> >> Thank you >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From hello@REDACTED Tue Nov 26 10:39:54 2019 From: hello@REDACTED (Adam Lindberg) Date: Tue, 26 Nov 2019 10:39:54 +0100 Subject: Inets http2 support In-Reply-To: References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> <37281c1c-696a-b6fc-9337-b2bfc6d2be71@ninenines.eu> Message-ID: <0B236C6B-A80F-44AE-A52F-345750347F28@alind.io> @Eric: It can have upsides for Erlang distribution as well (security considerations aside). For example, quick reconnection in case of net splits and changing network environments (e.g. cloud servers getting new IPs because of some network issue). Perhaps even the possibility to mask some network issues that would tear down a TCP connection. Or just roaming Erlang nodes on mobile devices :-D @Lukas, @Lo?c: Working on QUIC separately would still be beneficial though, right? Or is there something in HTTP/3 that requires tighter integration with QUIC that a separate implementation would not provide? Cheers, Adam > On 23. Nov 2019, at 13:38, Eric Pailleau wrote: > > QUIC : > "To solve this problem, QUIC includes a connection identifier which uniquely identifies the connection to the server regardless of source. This allows the connection to be re-established simply by sending a packet, which always contains this ID, as the original connection ID will still be valid even if the user's IP address changes." > > Wow... Nice security issues in perspective! > > > Envoy? depuis mon mobile > > > ---- Lo?c Hoguin a ?crit ---- > > On 22/11/2019 21:03, Dave Cottlehuber wrote: > > >> On 21/11/2019 17:22, Lukas Larsson wrote: > > >>> One of the use-cases (besides the obvious HTTP/3 use-case) that we have > > >>> been thinking about would be to allow the Erlang distribution to run > > >>> over QUIC. > > > > > > Had you considered something like nng instead of HTTP/3 for this? > > > > You mean QUIC, no need for HTTP/3 for the distribution. > > > > > - already available[1] > > > - much much simpler than HTTP/3/QUIC (I would hate to count the lines of code) > > > - the survey pattern[2] looks ideal for maintaining heartbeats > > > > > > I've fiddled with enm[3] https://github.com/basho/enm which is wire compatible, > > > and it's very fast indeed. > > > > > > [1]: https://github.com/nanomsg/nng > > > [2]: https://nanomsg.org/gettingstarted/nng/survey.html > > > [3]: https://github.com/basho/enm > > > > From what I understand it's still built on top of TCP. The main > > advantage of QUIC is that you can map messages onto different streams. > > Each stream's messages are ordered but the whole connection is not. > > Streams don't block each other. Different streams may have different > > priorities (we could have a process_flag that indicates this for very > > important processes and the distribution driver would send their > > messages before others). > > > > Cheers, > > > > -- > > Lo?c Hoguin > > https://ninenines.eu > From essen@REDACTED Tue Nov 26 11:09:57 2019 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 26 Nov 2019 11:09:57 +0100 Subject: Inets http2 support In-Reply-To: <0B236C6B-A80F-44AE-A52F-345750347F28@alind.io> References: <1451515160.1108713.1574073606814.ref@mail.yahoo.com> <1451515160.1108713.1574073606814@mail.yahoo.com> <59833099-10cd-3b0d-0826-5ec2c6952dba@ninenines.eu> <79b70056-8568-4bca-bbbf-eb5d00db85c2@www.fastmail.com> <37281c1c-696a-b6fc-9337-b2bfc6d2be71@ninenines.eu> <0B236C6B-A80F-44AE-A52F-345750347F28@alind.io> Message-ID: <7869b299-d41c-33bd-f0b2-813d33c0b332@ninenines.eu> On 26/11/2019 10:39, Adam Lindberg wrote: > @Lukas, @Lo?c: Working on QUIC separately would still be beneficial though, right? Or is there something in HTTP/3 that requires tighter integration with QUIC that a separate implementation would not provide? I am definitely talking about separate implementations, QUIC could be in OTP or third party and HTTP/3 would just sit in Cowboy/Cowlib. What HTTP/3 would provide to QUIC development is a use case that uses most of the features from QUIC and would allow optimizing QUIC. I'm thinking for example of the synchronization part of QPACK, of the different types of streams QUIC is using, and so on. I don't think HTTP/3 would have a significant impact on the features of the QUIC library though. There might be an exception to that, and that's currently an unknown due to the lack of consensus, and that is the priority mechanism. It's arguably more into optimization territory as well though. So yes it's fine to work on them separately. I just think about the full picture since I'm mostly interested in HTTP myself. If I started working on this today I would focus on getting the HTTP/3 part before QUIC because I'm more interested, not because of any coupling between the two. Cheers, -- Lo?c Hoguin https://ninenines.eu From okaprinarjaya@REDACTED Tue Nov 26 18:17:04 2019 From: okaprinarjaya@REDACTED (I Gusti Ngurah Oka Prinarjaya) Date: Wed, 27 Nov 2019 00:17:04 +0700 Subject: Hackney cannot access HTTPS - CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data In-Reply-To: References: Message-ID: Hi, Yes You are correct @Bram Verburg, I got bitten by ssl bugs. So now I use Erlang/OTP 22.1.8 and problem solved! Thank you Pada tanggal Sel, 26 Nov 2019 pukul 15.02 I Gusti Ngurah Oka Prinarjaya < okaprinarjaya@REDACTED> menulis: > Hi, > > Thank you for your reply, Unfortunately for now, I don't know exactly how > to identify exact detail patch level version for my installed Erlang/OTP. > > I've try to run: > erlang:display(erlang:system_info(otp_release)). but that command only > displaying "21" > {ok, Version} = file:read_file(filename:join([code:root_dir(), "releases", > erlang:system_info(otp_release), "OTP_VERSION"])), io:fwrite(Version). > that command only displaying 21.3 > > And, how to update my Erlang/OTP version to latest Erlang/OTP 21.3 patch > release using kerl ? > I install Erlang/OTP distribution using kerl at first. so i hope kerl > also able to update existing installation. > > Thank you > > > > > Pada tanggal Sel, 26 Nov 2019 pukul 14.00 Bram Verburg < > bram.verburg@REDACTED> menulis: > >> Which exact Erlang/OTP version are you on? It seems to be 21.3, and if >> you're not on the very latest patch level you might get bitten by ssl bugs >> such as https://bugs.erlang.org/browse/ERL-968. That particular one was >> fixed in 21.3.8.5, and even though it was originally reported with a >> slightly different error response, the root cause does make handshake data >> appear malformed to the ssl state machine. >> >> If that's not the issue, can you perhaps share a URL that triggers the >> issue? >> >> >> On Tue, 26 Nov 2019 at 02:40, I Gusti Ngurah Oka Prinarjaya < >> okaprinarjaya@REDACTED> wrote: >> >>> Hi, >>> >>> I use hackney to fetch a web page and i get error when accessing website >>> with https with following error: >>> >>> TLS client: In state certify at tls_connection.erl:966 generated CLIENT >>> ALERT: Fatal - Handshake Failure - malformed_handshake_data >>> >>> ** exception error: no match of right hand side value {error, >>> {tls_alert, >>> >>> {handshake_failure, >>> "received >>> CLIENT ALERT: Fatal - Handshake Failure - malformed_handshake_data"}}} >>> in function handler_test_test_test:fetch_page/1 >>> (/Users/okaprinarjaya/Oprek/Erlang-Oprek-Tiga/idea_execute/src/handler_test_test_test.erl, >>> line 37) >>> >>> Then I found this: https://github.com/benoitc/hackney/issues/362 . I >>> try to implement that github issue but still failed and still get same >>> error message. My code based on github issue: >>> https://pastebin.com/QeCc0tab >>> >>> My environment: >>> 1. MacOS Mojave v10.14.6 >>> 2. Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] >>> [async-threads:1] >>> 3. I install hackney with this way through rebar3: >>> {hackney, ".*", {git, "git://github.com/benoitc/hackney.git", {branch, >>> "master"}}} >>> and my current hackney version is 1.15.2 >>> >>> ssl:cipher_suites() >>> >>> [{ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, >>> {ecdhe_rsa,aes_256_cbc,sha384,sha384}, >>> {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, >>> {ecdh_rsa,aes_256_cbc,sha384,sha384}, >>> {dhe_rsa,aes_256_cbc,sha256}, >>> {dhe_dss,aes_256_cbc,sha256}, >>> {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, >>> {ecdhe_rsa,aes_128_cbc,sha256,sha256}, >>> {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, >>> {ecdh_rsa,aes_128_cbc,sha256,sha256}, >>> {dhe_rsa,aes_128_cbc,sha256}, >>> {dhe_dss,aes_128_cbc,sha256}, >>> {ecdhe_ecdsa,aes_256_cbc,sha}, >>> {ecdhe_rsa,aes_256_cbc,sha}, >>> {dhe_rsa,aes_256_cbc,sha}, >>> {dhe_dss,aes_256_cbc,sha}, >>> {ecdh_ecdsa,aes_256_cbc,sha}, >>> {ecdh_rsa,aes_256_cbc,sha}, >>> {ecdhe_ecdsa,aes_128_cbc,sha}, >>> {ecdhe_rsa,aes_128_cbc,sha}, >>> {dhe_rsa,aes_128_cbc,sha}, >>> {dhe_dss,aes_128_cbc,sha}, >>> {ecdh_ecdsa,aes_128_cbc,sha}, >>> {ecdh_rsa,aes_128_cbc,sha}] >>> >>> Please enlightenment >>> >>> Thank you >>> >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostinelli@REDACTED Tue Nov 26 18:33:22 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Tue, 26 Nov 2019 18:33:22 +0100 Subject: dirty_select a limited number of records Message-ID: All, I've got a mnesia table that I create with: mnesia:create_table(my_table, [ {type, set}, {attributes, record_info(fields, my_table)}, {index, [#my_table.pid]}, {storage_properties, [{ets, [{read_concurrency, true}, {write_concurrency, true}]}]} ]). If I want to get all records for the secondary index, I do: mnesia:dirty_index_read(my_table, Pid, #my_table.pid). This operation if very fast. Now, sometimes I just need the *first* matching record and if there are many entries with the secondary index, the operation becomes a bottleneck. So I'm trying: S = fun() -> MatchHead = #my_table{pid = Pid, _ = '_'}, Guards = [], Result = '$_', mnesia:select(my_table, [{MatchHead, Guards, [Result]}], 1, read) end, case mnesia:activity(sync_dirty, S) of {[Entry], _} -> Entry; _ -> undefined end. This works, however this operation does not seem to be using the secondary index as it becomes extremely slow. I also tried: S = fun() -> MatchHead = #my_table{pid = '$1', _ = '_'}, Guard = {'=:=', '$1', Pid}, Result = '$_', mnesia:select(my_table, [{MatchHead, [Guard], [Result]}], 1, read) end, case mnesia:activity(sync_dirty, S) of '$end_of_table' -> undefined; {[Entry], _} -> Entry end. And this is also slow. What can I do to dirty_select a limited number of records from a mnesia table? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Tue Nov 26 22:26:47 2019 From: t@REDACTED (Tristan Sloughter) Date: Tue, 26 Nov 2019 14:26:47 -0700 Subject: rebar3 compile dependencies separated from project code In-Reply-To: References: Message-ID: Yes, only copy the rebar.config and rebar.lock files and rebar3 compile will do the right thing. You might find https://adoptingerlang.org/docs/production/docker/ useful. It goes beyond this form of layer caching to what is now possible with the latest versions of docker, including things like caching the hex packages outside of a layer so they don't get redownloaded after a cache bust. Tristan On Sat, Nov 23, 2019, at 03:04, rtp wrote: > Found a solution just now :-D > > Just copy only the rebar.conf-file in a first step and compile it (I > didn't expect to work ... but it does) : > > FROM erlang:22 > > WORKDIR /BUILD/ > > # get deps in a first step - to use cache in next build ;-) > > COPY erlang/aw/rebar.config . > RUN rebar3 compile > > # now get whole code > > COPY erlang/aw/ . > RUN rebar3 as prod tar > > > Perhaps it helps someone :-) > > Cheerio > > Ralf > > > On 23.11.19 10:47, rtp wrote: > > Hi, > > > > I'm build up a docker pipeline for a erlang project. Currently I build > > the whole project in a new erlang-Docker-Container by a > > > > rebar3 as prod tar > > > > This task downloads and compiles the dependencies every time I build; > > because the erlang-container is plain and empty. > > > > I can now call a > > > > rebar3 get-deps > > > > in a previous container (or step) to get rid of the downloads. > > > > But I want to get a step further an compile the dependencies before > > building the application, to compile only the app-code and to make the > > build faster. > > > > Does anybody know how to compile dependencies separated from app-code ? > > > > I can't find anything. Perhaps there's a plugin I don't know!? > > > > > > Thanks a lot > > > > Ralf > > > From ostinelli@REDACTED Wed Nov 27 10:03:15 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Wed, 27 Nov 2019 10:03:15 +0100 Subject: dirty_select a limited number of records In-Reply-To: References: Message-ID: Here's another try, this one too is very slow: QH = qlc:q([E || E <- mnesia:table(my_table), E#my_table.pid == Pid]), mnesia:async_dirty(fun() -> QC = qlc:cursor(QH), R = qlc:next_answers(QC, N), qlc:delete_cursor(QC), R end). Any ideas? On Tue, Nov 26, 2019 at 6:33 PM Roberto Ostinelli wrote: > All, > I've got a mnesia table that I create with: > > mnesia:create_table(my_table, [ > {type, set}, > {attributes, record_info(fields, my_table)}, > {index, [#my_table.pid]}, > {storage_properties, [{ets, [{read_concurrency, true}, > {write_concurrency, true}]}]} > ]). > > If I want to get all records for the secondary index, I do: > > mnesia:dirty_index_read(my_table, Pid, #my_table.pid). > > This operation if very fast. Now, sometimes I just need the *first* > matching record and if there are many entries with the secondary index, the > operation becomes a bottleneck. So I'm trying: > > S = fun() -> > MatchHead = #my_table{pid = Pid, _ = '_'}, > Guards = [], > Result = '$_', > mnesia:select(my_table, [{MatchHead, Guards, [Result]}], 1, read) > end, > case mnesia:activity(sync_dirty, S) of > {[Entry], _} -> Entry; > _ -> undefined > end. > > This works, however this operation does not seem to be using the secondary > index as it becomes extremely slow. I also tried: > > S = fun() -> > MatchHead = #my_table{pid = '$1', _ = '_'}, > Guard = {'=:=', '$1', Pid}, > Result = '$_', > mnesia:select(my_table, [{MatchHead, [Guard], [Result]}], 1, read) > end, > case mnesia:activity(sync_dirty, S) of > '$end_of_table' -> undefined; > {[Entry], _} -> Entry > end. > > And this is also slow. What can I do to dirty_select a limited number of > records from a mnesia table? > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Aleksander.Nycz@REDACTED Wed Nov 27 10:29:04 2019 From: Aleksander.Nycz@REDACTED (Aleksander Nycz) Date: Wed, 27 Nov 2019 10:29:04 +0100 Subject: dirty_select a limited number of records In-Reply-To: References: Message-ID: <237d9b3f-1ef7-4ec9-e0c4-dbea35a442ab@comarch.pl> Hello, Please check: https://github.com/nyczol/otp/tree/mnesia_new_index2 for dirty_index_read/4 with limit in mnesia module. Branch was created in 2013 but unfortunately PR was rejected by OTP. Branch contains 2 features: - new index type for mnesia - set of functions that operate on secondary indexes and return limited number of records Used on prod envs from 2013 untli now. Maybe its good time to create PR again for 2nd part only? Regards Aleksander Nycz W dniu 27.11.2019 o?10:03, Roberto Ostinelli pisze: > Here's another try, this one too is very slow: > > QH = qlc:q([E || E <- mnesia:table(my_table), E#my_table.pid == Pid]), > mnesia:async_dirty(fun() -> > ? ? QC = qlc:cursor(QH), > ? ? R = qlc:next_answers(QC, N), > ? ? qlc:delete_cursor(QC), > ? ? R > end). > > Any ideas? > > > > On Tue, Nov 26, 2019 at 6:33 PM Roberto Ostinelli > wrote: > > All, > I've got a mnesia table that I create with: > > mnesia:create_table(my_table, [ > ? ? {type, set}, > ? ? {attributes, record_info(fields, my_table)}, > ? ? {index, [#my_table.pid]}, > ? ? {storage_properties, [{ets, [{read_concurrency, true}, > {write_concurrency, true}]}]} > ]). > > If I want to get all records for the secondary index, I do: > > mnesia:dirty_index_read(my_table, Pid, #my_table.pid). > > This operation if very fast. Now, sometimes I just need the > /first/ matching record and if there are many entries with the > secondary index, the operation becomes a bottleneck. So I'm trying: > > S = fun() -> > ? ? MatchHead = #my_table{pid = Pid, _ = '_'}, > ? ? Guards = [], > ? ? Result = '$_', > ? ? mnesia:select(my_table, [{MatchHead, Guards, [Result]}], 1, read) > end, > case mnesia:activity(sync_dirty, S) of > ? ? {[Entry], _} -> Entry; > ? ? _ -> undefined > end. > > This works, however this operation does not seem to be using the > secondary index as it becomes extremely slow. I also tried: > > S = fun() -> > ? ? MatchHead = #my_table{pid = '$1', _ = '_'}, > ? ? Guard = {'=:=', '$1', Pid}, > ? ? Result = '$_', > ? ? mnesia:select(my_table, [{MatchHead, [Guard], [Result]}], 1, read) > end, > case mnesia:activity(sync_dirty, S) of > ? ? '$end_of_table' -> undefined; > ? ? {[Entry], _} -> Entry > end. > > And this is also slow. What can I do to dirty_select a limited > number of records from a mnesia table? > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From mjtruog@REDACTED Fri Nov 29 22:01:04 2019 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 29 Nov 2019 13:01:04 -0800 Subject: [ANN] CloudI 1.8.0 Released! Message-ID: Download 1.8.0 from https://osdn.net/dl/cloudi/cloudi-1.8.0.tar.gz CloudI (https://cloudi.org/) is a "universal integrator" using an Erlang core to provide fault-tolerance with efficiency and scalability. The CloudI API provides a minimal interface to communicate among services so programming language agnostic and protocol agnostic integration can occur.? CloudI currently integrates with the programming languages: C/C++, Elixir, Erlang, Go, Haskell, Java, JavaScript/node.js, OCaml, PHP, Perl, Python, and Ruby,? Many reusable services are included that rely on the CloudI service bus. The details for this release are below: ? * backwards compatibility difference: ??? * ACL strings do not get a "*" automatically appended on ????? non-patterns, so exact strings may be used as an ACL string ? * Unicode support was added for external service configuration ??? (file_path, args, env) and any other string or function use ? * CloudI API now has a shutdown function for external services ??? (internal services always had the shutdown return value) ? * Fixed CloudI Service API services_add failure with multiple ??? service configurations to handle data properly ? * Fixed service configuration option count_process_dynamic ??? so it doesn't influence a pending service termination ? * Erlang services were added: ??? * cloudi_service_api_batch executes services with batch queues ??? * cloudi_service_cron sends service requests for cron expressions ??? * cloudi_service_shell executes service request shell commands ? * Java service cloudi_service_htmlunit was created ??? (see https://github.com/CloudI/cloudi_service_htmlunit/ ) ? * Haskell GHC 8.8.1 support was added ? * Bugs were fixed and other improvements were added ??? (see the ChangeLog for more detail) Please mention any problems, issues, or ideas! Thanks, Michael SHA256 CHECKSUMS cloudi-1.8.0.tar.gz? (16092165 bytes) a9b690289d5db6239bfe847d27cfe81f5a05194e9732f45d60344ef95afc4f4d cloudi-1.8.0.tar.bz2 (13315002 bytes) 80c6a46fdf2c1b1903b3062afda2961c3cfcbfdaacb4f67ca3e875f3e7f767fa From ostinelli@REDACTED Fri Nov 29 23:46:49 2019 From: ostinelli@REDACTED (Roberto Ostinelli) Date: Fri, 29 Nov 2019 23:46:49 +0100 Subject: gen_server locked for some time Message-ID: All, I have a gen_server that in periodic intervals becomes busy, eventually over 10 seconds, while writing bulk incoming data. This gen_server also receives smaller individual data updates. I could offload the bulk writing routine to separate processes but the smaller individual data updates would then be processed before the bulk processing is over, hence generating an incorrect scenario where smaller more recent data gets overwritten by the bulk processing. I'm trying to see how to solve the fact that all the gen_server calls during the bulk update would timeout. Any ideas of best practices? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Nov 30 00:21:13 2019 From: g@REDACTED (Guilherme Andrade) Date: Fri, 29 Nov 2019 23:21:13 +0000 Subject: gen_server locked for some time In-Reply-To: References: Message-ID: Hello Roberto, If copying the data to a second process is not (too) costly, you can do just that: have a second process responsible for writing the data - the I/O bound component - and the original one act as a coordinator, directing the second one asynchronously but always conscious of what the secondary process is doing. This means it will remain available to handle incoming calls (unless seriously overloaded) and, if need be, reject incoming requests preemptively for back pressure, whether instantaneously or after a timeout measured on its own (rather than on the caller) by replying through `gen_server:reply/2` rather than through `{reply, Reply, State}`. Alternatively, you can also switch the problem around and use some sort of broker to match write requests with the writer process - this has the advantage of making it trivial to scale to multiple writer processes unless there are hard serialization constraints. With this approach, and in the simplest setup, a single broker process never blocks and can forward the write requests to an available writer process which has explicitly declared itself available for writing (and it only does this between write requests), as well as manage timeouts. The `sbroker` library[1], although no longer maintained, is a true wonder for implementing this sort of pattern. [1]: https://github.com/fishcakez/sbroker On Fri, 29 Nov 2019 at 22:47, Roberto Ostinelli wrote: > All, > I have a gen_server that in periodic intervals becomes busy, eventually > over 10 seconds, while writing bulk incoming data. This gen_server also > receives smaller individual data updates. > > I could offload the bulk writing routine to separate processes but the > smaller individual data updates would then be processed before the bulk > processing is over, hence generating an incorrect scenario where smaller > more recent data gets overwritten by the bulk processing. > > I'm trying to see how to solve the fact that all the gen_server calls > during the bulk update would timeout. > > Any ideas of best practices? > > Thank you, > r. > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Nov 30 00:23:42 2019 From: g@REDACTED (Guilherme Andrade) Date: Fri, 29 Nov 2019 23:23:42 +0000 Subject: gen_server locked for some time In-Reply-To: References: Message-ID: (I know see you actually mentioned using separate processes, but my suggestion should still apply, depending on constraints.) On Fri, 29 Nov 2019 at 23:21, Guilherme Andrade wrote: > Hello Roberto, > > If copying the data to a second process is not (too) costly, you can do > just that: have a second process responsible for writing the data - the I/O > bound component - and the original one act as a coordinator, directing the > second one asynchronously but always conscious of what the secondary > process is doing. > This means it will remain available to handle incoming calls (unless > seriously overloaded) and, if need be, reject incoming requests > preemptively for back pressure, whether instantaneously or after a timeout > measured on its own (rather than on the caller) by replying through > `gen_server:reply/2` rather than through `{reply, Reply, State}`. > > Alternatively, you can also switch the problem around and use some sort of > broker to match write requests with the writer process - this has the > advantage of making it trivial to scale to multiple writer processes unless > there are hard serialization constraints. > With this approach, and in the simplest setup, a single broker process > never blocks and can forward the write requests to an available writer > process which has explicitly declared itself available for writing (and it > only does this between write requests), as well as manage timeouts. The > `sbroker` library[1], although no longer maintained, is a true wonder for > implementing this sort of pattern. > > [1]: https://github.com/fishcakez/sbroker > > On Fri, 29 Nov 2019 at 22:47, Roberto Ostinelli > wrote: > >> All, >> I have a gen_server that in periodic intervals becomes busy, eventually >> over 10 seconds, while writing bulk incoming data. This gen_server also >> receives smaller individual data updates. >> >> I could offload the bulk writing routine to separate processes but the >> smaller individual data updates would then be processed before the bulk >> processing is over, hence generating an incorrect scenario where smaller >> more recent data gets overwritten by the bulk processing. >> >> I'm trying to see how to solve the fact that all the gen_server calls >> during the bulk update would timeout. >> >> Any ideas of best practices? >> >> Thank you, >> r. >> > > > -- > Guilherme > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From karlsson.rm@REDACTED Sat Nov 30 06:42:44 2019 From: karlsson.rm@REDACTED (Mikael Karlsson) Date: Sat, 30 Nov 2019 06:42:44 +0100 Subject: gen_server locked for some time In-Reply-To: References: Message-ID: Hi Roberto, For the smaller data updates using the gen_server:cast, which returns immediate, is an option if you do not need to check any reply values. Increasing the timeout using gen_server:call/3 is another if your clients accept to be blocked for the complete time during bulk update. In that case you could consider using the gen_statem instead which has a default timeout of infinity for the call function. I guess the aboves are not possible? so when offloading bulk write to a separate process you can maybe add some state handling in your gen_server (gen_statem) so that you process the smaller updates into an internal queue during the bulk write and postpone processing them until the write is finished. The gen_statem has postpone functions (which will still block your client) as well as insertion of internal events but I am not sure how useful they are in your case. Mikael Den fre 29 nov. 2019 kl 23:47 skrev Roberto Ostinelli : > All, > I have a gen_server that in periodic intervals becomes busy, eventually > over 10 seconds, while writing bulk incoming data. This gen_server also > receives smaller individual data updates. > > I could offload the bulk writing routine to separate processes but the > smaller individual data updates would then be processed before the bulk > processing is over, hence generating an incorrect scenario where smaller > more recent data gets overwritten by the bulk processing. > > I'm trying to see how to solve the fact that all the gen_server calls > during the bulk update would timeout. > > Any ideas of best practices? > > Thank you, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sat Nov 30 11:50:07 2019 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 30 Nov 2019 11:50:07 +0100 Subject: gen_server locked for some time In-Reply-To: References: Message-ID: On Fri, Nov 29, 2019 at 11:47 PM Roberto Ostinelli wrote: > > All, > I have a gen_server that in periodic intervals becomes busy, eventually over 10 seconds, while writing bulk incoming data. This gen_server also receives smaller individual data updates. > > I could offload the bulk writing routine to separate processes but the smaller individual data updates would then be processed before the bulk processing is over, hence generating an incorrect scenario where smaller more recent data gets overwritten by the bulk processing. > > I'm trying to see how to solve the fact that all the gen_server calls during the bulk update would timeout. If there is more logic in the gen_server for incoming data, have it offload all writes to the separate process, using its message queue as a buffer. Otherwise make the sends to the gen_server asynchronous.